The quality of online research

From: Tim Lee <TLee_at_webcmo.com>
Date: Wed, 19 May 1999 08:50:09 -0500 (CDT)

As web marketers, we need marketing research information to
support our decisions. But, can we trust those online
researches?

Recently, Geoffrey Ramsey at emarketer.com criticized the
quality of online research in an advertising conference. He
claimed that the poor quality of online research came from:
1) those researches are paid by certain organizations -
results are biased; and 2) those researches are conducted
through surveys--information is exaggerated. Although I
agree with Geoffrey that the quality of online research is
poor, I am afraid I don't think he gave the right reasons of
such poor quality.

Geoffrey's statements just wanted to demonstrate the
superiority of his research. However, from his statements,
it seems to me that he doesn't quite understand the research
process and statistic analysis. I have asked him: if he
thinks those research results are invalid (biased or
exaggerated), how could he make the valid analysis based on
such invalid data? What statistic technique they used to
conduct analyses (Because I don't think there is any
statistic theory or methods supporting his analyses.) But no
one from emarketer tries to answer these questions.

The real reasons for poor quality of online research, I
believe, come from: 1) limited marketing knowledge; 2)
limited stats knowledge; and 3) irresponsible attitudes.

1. Limited marketing knowledge: while we have so many online
research reports, how many different kinds of information we
get? Not much! So far, most online research are about "how
many online" or "how many bought online". While we have so
many marketing issues that need our marketing research
professionals to explore, why on the earth do almost all of
them explore such "how many online" subject again and again,
and again?

The answer is: they don't have much marketing knowledge to
explore the Internet issues. Have we ever heard from these
online research firms discussing our marketing issues? If a
marketing researcher has less marketing knowledge than
his/her clients, how can he/she help the clients understand
their markets?

2. Limited states knowledge: It is forgivable if a marketing
researcher has limited marketing knowledge but knows
statistical techniques needed for daily marketing research.
Unfortunately most marketing researchers don't meet such
minimum requirement. Let's look at the marketing research
reports conducted by the "leading" online marketing research
firms (you know the names), what portion of their work is
not bivariate analysis? Almost zero!

I don't mean any marketing research shall use multivariate
analysis. Sometimes (only sometimes) a bivariate analysis is
the best technique for a research project. However, one
CHOOSES to use bivariate analysis is totally different from
one only knows bivariate analysis. For most marketing issues
(such as market segmentation, market targeting, brand choice
behavior, brand equity, etc.) cannot be researched by
bivariate analysis. If our "leading" research companies are
such, what do we expect from the rest?

3. Irresponsible attitudes: I always believe that research
is a very serious business. A researcher shall be
responsible for whatever he/she does. But it is not the case
in the reality. Nowadays, many marketing research companies
(especially those "leading" companies) are addicted to
online forecasting. However, only occasionally we heard that
their forecast was merely GUESS, we have never been informed
about how they made their projections and how reliable their
results are.

Do they know many of us are making our marketing decisions
based on their forecasts? Do they know how reliable their
forecasts are?

I once did a test to see how unreliable these forecasts are.
The minimum error from my different forecasting models is
300%! (Please read "Let's forecast!" at
http://www.webcmo.com/forum/fortop4.htm.)

As a matter of fact, the most serious problem about the
quality of online research is not how poor the information
we get. IT IS WHAT INFORMATION WE COULD NOT GET WHILE WE
NEED IT! As web marketers, we need quality online research.
However, can we trust online researchers?

I hope one day marketing research can become an honest and
serious business. A business we web marketers can trust.
Regards,

Tim Lee
Editor -- Journal of Web Marketing Research
http://www.WebCMO.com
Innovations in web marketing research & strategy
Participate in "Email marketing Strategy" survey to receive
a FREE report
http://www.webcmo.com/mis/survey.htm

========================================================================
------------------------------------------------------------------------
        This week's Online Advertising Discussion List sponsors:
               Flycast, Peopledesign,and Iconocast

WEB ADVERTISING THAT MAKES YOU A HERO. Our Web experts optimize your
buy across 922 sites. All for what you'd normally pay for placement on
one. Call it superhuman. http://WWW.FLYCAST.COM/OA

                                ---

What % of the companies who have worked with an Interactive Services
firm have stopped using a firm before contract completion?
The answer and more at www.PeopleDesignTechnology.com/1999answers

                                ---

*** We're saying "HASTA LA VISTA BABY" to Boring Conferences! ***
Register now for Web Attack! -- The Ultimate Internet Marketing Event
June 17-18, San Francisco http://www.iconocast.com/webattack/tenagra

------------------------------------------------------------------------
========================================================================



Received on Wed May 19 1999 - 09:54:13 CDT