Statistical Significance

Steve Roth (72260.3100_at_CompuServe.COM)
09 Aug 96 14:00:59 EDT

• Messages sorted by: [ date ][ thread ][ subject ][ author ]
• Archive Home Page
• Search
• Next message: Madvulcan: "Re: Ad exposure"
• Previous message: Di-ann Eisnor: "Re: what belongs on a banner"

re: Tony Hsieh's posting:

>We compare some of the click-thru ratios of banners that we have tested to
>advertise our service, the Internet Link Exchange. Each banner has been
>shown over
>16,000 times, so we believe the results to be statistically significant.

As a direct-mail marketer, we're incredibly interested in statistical
significance (and click-through rate). If we mail 10,000 pieces and get a
one-percent response rate, what will we get on the next mailing?

I've put up a little excel spreadsheet (Mac 4.x format; most current and
later versions should open it) on our web site that calculates
statistical significance (we hired a UW math student to create it for
us).

http://www.thunderlizard.com/misc/statsign.html

Here's what it does:

Test Mailing Next Mailing
Size of list 20,000 64 percent chance response
will be
between:
Percentage Response 0.50% High Response 0.55%
Low Response 0.45%

95 percent chance response
will be
between:
High Response 0.60%
Low Response 0.40%
Standard D 0.05%

Hope you find this useful! It should be equally valid for
banners--impressions versus clicks.

The basics:

As you increase your mailing size, statistical significance goes
up--standard deviation goes down (unfortunately, slowly), so your next
mailing is more predictable.

As your response rate increases, the standard deviation goes up as well,
but not as fast as the response rate. So with higher response rates,
future mailings are more predictable.

While you're at our site (my plug here), check out our conference, Web
Advertising '96, running October 31-November 1 in New York. You'll see
our sponsorship on this list in the next week or so.

Steve Roth
Thunder Lizard Productions
Seattle, WA
sroth_at_thunderlizard.com
www.thunderlizard.com

• Archive Home Page
• Search
• Next message: Madvulcan: "Re: Ad exposure"
• Previous message: Di-ann Eisnor: "Re: what belongs on a banner"

With an archive of more than 14,000 postings, since 1996 the Online Advertising Discussion List has been the Internet's leading forum focused on professional discussion of online advertising and online media buying and selling strategies, results, studies, tools, and media coverage. If you wish to join the discussion list, please use this link to sign up on the home page of the Online Advertising Discussion List.