NONE: Re: ONLINE-ADS>> Gatech survey is self-selected, not a real survey
Re: ONLINE-ADS>> Gatech survey is self-selected, not a real survey
Steve Runfeldt (Steve_at_CustomerSat.com)
Wed, 18 Feb 1998 01:08:29 -0800
Glenn Fleishman wrote:
> In the discussion on cookies, Michael Hammons wrote:
>
> >Check out the GVU survey:
> >
> >http://www.gvu.gatech.edu/user_surveys/survey-1997-10/
> >
> >It specifically asks users about cookies and I think you
> >will find this information interesting.
>
> I'd like to continue my own private GVU screed, which is - although I like
> the GVU folks and think there is some benefit in what they're doing, it's
> not a survey. It's a self-selected, non-sampled group of people who fill it
> out.
Glenn has a point here which is very well taken, but I think that it goes a
little
too far. The GVU survey can be very useful as an indicator, provided one
is aware
of the limitations. I think that it is important to note too that all
surveys are
in essence self selected. Even telephone surveys allow the respondent to
decline.
The fact is that no consumer survey escapes this bias to one degree or another.
The question is, to what degree...?
One one reason that we named our company CustomerSat.com is that when we started
out the only really accurate way of collecting online survey data was to use an
established customer base for our sample. Similarly, online employee
satisfaction
surveys do not suffer the same sampling problems as competely self selected open
surveys.
At the AMA market research meeting in Orlando last October, Teri Mortola from
Netscape remarked that there was presently no way to control sample on a Web
survey. That was not entirely correct. We have two techniques, one which we
routinely use and another which we are about to introduce. We have been using
Positive Respondent Identification (TM) (PRI) for almost six months now.
With PRI
we send Email survey invitations to a preselected respondent sample. Each
invitation contains a unique clickable PRI address. Respondents click on the PRI
address and are brought to a Web survey. Each respondent may visit and complete
the survey only once. No one without a valid PRI code can gain access to the
survey. We recently completed a consumer survey where personalized
invitations were
sent to 1800 prescreened respondents (no spam, these people were called
first). We
had 900 responses in 7 days!
PRI requires a balanced email sample list before the survey is conducted. While
this method is not yet appropriate for generalized market research, it is
entirely
appropriate if your target market is on the Internet (and your sample
contains only
preapproved email addresses, for example from a customer or employee list). I
believe that this approach gives us level of sample control comparable to CATI
survey methods.
We will be demo'ing PRI at the ARF conference in NY in March and will be
using it
as the offiicial survey partners of the conference.
But what about open Web surveys where you don't have email addresses?
Suppose you
want to measure the effectivness of your Web site and do not want to ask
visitors
to register. The latest approach uses Pop!Up(TM) Windows. With Pop!Up we
select
1/n visitors to a Web site to receive a Java based window containing an
invitation
to complete a survey. If the site visitor agrees, the window reduces to an icon
which moves to the top of the screen. When the visitor is done browsing
the site,
he/she may click on the icon and be brought to the survey page. This approach,
too, provides much better sampling than a simple open invitation to a survey.
I would like to get some feedback on what list members think of these
approaches.
In particular, do you think that the Pop!Up approach would provide
surveyors like
GVU a more balanced sample, if, for example, they could place pop-up windows on
some of the major search engines. Regarding PRI, one issue we are constantly
addressing is email sample. I wonder how many of you have ways of
collecting email
sample?
Regards,
Steve Runfeldt
VP Research
CustomerSat.com
----------------------------------------------------------------------
This week's Online Advertising Discussion List sponsor:
HotBot - http://www.hotbot.com - The WIRED Search Center
HotBot, rated the #1 search engine by CNET, has keywords available now
in these categories: computers, software, shareware, Internet, games,
music, and more. Contact Rick Boyce at 415.276.8440 or rick_at_wired.com.
-----------------------------------------------------------------------
Online Advertising Discussion List To Unsubscribe send UNSUBSCRIBE
http://www.o-a.com/ to online-ads-request_at_o-a.com
