Re: Online-ads>> Reporting robot activiy in web statistics

Daniel Pfarrer (dan_at_gsp.com)
Tue, 1 Jul 1997 08:38:29 -0400 (EDT)

• Messages sorted by: [ date ][ thread ][ subject ][ author ]
• Archive Home Page
• Search
• Next message: Jim Sterne: "Re: ONLINE-ADS>> Reporting robot activity in web statistics"
• Previous message: FitnessLink: "Re: ONLINE-ADS>> Banners: should we pack our bags?"

On Mon, 30 Jun 1997, Josh McCormack wrote:

> > My thought: shouldn't ALL ad tracking software provide a mechanism to
> > separate out "commercial robots" and "personal robots" for reporting of
> > ad activity?
>
> I question the efficacy of screening out robots and spiders. Unless the
> robot or spider identifies itself as such, and not as Netscape, or
> another (as they usually seem to) I don't see how they would be logged
> as what they are.

It is quite easy to detect robots (as long as their are identified ahead
of time. For ex: In my web stats page (http://www.gsp.com/analog/#Browser)
you can see that "ArchitextSpider" was identified itself as a spider.
Another robot identified itself as using the server "crawl2.atext.com".
You could put this domain into a spider file to filter out.

You can see an example of all the information that is transmitted for each
http request by loading (http://www.gsp.com/cgi-bin/printenv). ttyl!

Daniel Pfarrer <dan_at_gsp.com>

• Archive Home Page
• Search
• Next message: Jim Sterne: "Re: ONLINE-ADS>> Reporting robot activity in web statistics"
• Previous message: FitnessLink: "Re: ONLINE-ADS>> Banners: should we pack our bags?"

With an archive of more than 14,000 postings, since 1996 the Online Advertising Discussion List has been the Internet's leading forum focused on professional discussion of online advertising and online media buying and selling strategies, results, studies, tools, and media coverage. If you wish to join the discussion list, please use this link to sign up on the home page of the Online Advertising Discussion List.