NONE: Re: ONLINE-ADS>> Online marketing research dilemmas
Re: ONLINE-ADS>> Online marketing research dilemmas
Steve Runfeldt (stever_at_CustomerSat.com)
Sun, 21 Jun 1998 00:37:53 -0400
Tim,
I think that what you say in general about analytical
techniques is true. Blind reliance on any particular
statistical technique or inappropriate use of multivariate
analysis is always problematic, not only on the Internet
but in any research endeavour.
My training in multivariate techniques came as an
undergrad in Psychology at Brandeis University and in grad
school in Behavioral Sciences at Georgia State and The
Rockefeller University. This was followed by nine years
of statistical analysis at one of the leading market
research firms. I still prefer a combination of Factor
(to remove the multicolinearity) and Multiple Regression
modeling. I have seen it work and produce very helpful
results. I have also seen it misused when sample sizes
are too small or when the range of questions is too
limited, as, for example, when too little of the variance
in the overall measure is attributable to the factors.
This is not to say that Causal Path Analysis, Logit,
Neural Network Modeling, etc are not also a useful
techniques. When there is a robust result to report and
when the right questions have been asked, in my
experience, all of these advanced techniques usually give
the same ultimate result, that is, they lead to the same
actions taken. Factor and Multiple Regression can usually
be explained satisfactorily to most clients. The approach
makes intuitive sense. More sophisticated approaches are
not usually as easy to explain. The client is left
believing the statistical analyist simply on the weight of
his or her reputation. At that point we become something
like high priests. I believe that much of the debate over
which multivariate techniuque to use should be left to
more esoteric statistical discussions. Claims that
technique X is useless and that only technique Y should
ever be used, are, IMHO irresponsible, especially when
there is so much experience out there proving the validity
of all of these approaches.
Multivariate techniques do often reveal connections that
are not apparent from crosstabs or frequencies alone. I
also think that it is somewhat unfortunate that our
industry has an over-reliance on crosstabs. I once gave a
workshop on statistical methods to a group of market
researchers where I gave them a crosstab of about 200
cells in pairs of columns. Ten of these cells showed a
statistically significant difference at the 95% confidence
level. I then explained to the class that the table was
of completely random numbers and that what the 95%
confidence level means is that 5% of the time you will get
differences this large by randon chance alone. They were
very surprised to see an example this clearly. I think
that there is definitely too great a reliance on the
details of crosstabs at the cellular level. This is
difficult to explain to most laymen.
Regarding popup survey invitations, these may be placed
anywhere on a website. In some cases it can be helpful to
let the respondent know that they will be surveyed. In
other cases it is a disadvantage. It can be a very useful
tool, but like any tool it can also be misused. It does,
however, lessen the effect of self selection. When
measuring opinions from visitors to a website, linking the
survey to a simple link or banner ad means that the
respondents are wholly self-selected. By using popup
technology, we regain some control over which visitors are
sampled. The popup window cannot be ignored like the
banner ad. The visitor has to make an active choice to
either take or not take the survey. They are required to
click on either "Yes" or "No". It is not as intrusive as
a phone call during dinner, but it does require a
response. This alone reduces self selection bias.
I am not exactly sure to what you are referring when you
say "We are merely some marketing researchers who cannot
suffer from the current situation of online marketing
research." If you are referring to the large number of
forms posted on websites with little or no marketing
research expertise behind them, I agree 100%. People can
spot a poorly designed survey a mile off. There is a very
good article in the current issue of Marketing Research by
Bearden, Madden and Uscategui on problems within the
market research and direct marketing industries, in
general and not just on the Internet, with consumers being
over surveyed and oversold. I think that we need to be
very careful how we approach consumers online. I also
think that it is going to be very difficult, if not
impossible, to control how surveys are deployed on the
Net. Afterall, all you need to post a survey is someone
who can write an HTML forms page. I hope that the need
for experienced survey designers and analyists will become
more evident.
Regards,
Steve Runfeldt
CustomerSat.com
P.S. AS far as I know, 'Online Advertising Masters' is not yet taken. <g>
========================================================================
------------------------------------------------------------------------
This week's Online Advertising Discussion List sponsor:
24/7 Media
"We chose 24/7 Media because of their proven
success selling online advertising."
-- Howard Lefkowitz, VP Business Development, EarthLink Network
24/7 Media -- Always Serving You
http://www.247media.com/join.html
------------------------------------------------------------------------
========================================================================
Online Advertising Discussion List To Unsubscribe send UNSUBSCRIBE
http://www.o-a.com/ to online-ads-request_at_o-a.com
