NONE: Re: ONLINE-ADS>> Weird Log Data
Re: ONLINE-ADS>> Weird Log Data
Robert J. Woodhead (trebor_at_animeigo.com)
Thu, 3 Dec 1998 06:50:17 -0600 (CST)
RICHARD_at_TENAGRA.COM WROTE:
> Since seeing this rather dramatic demonstration of how a
> spider can screw up click counts, I've begun wondering just
> how serious a problem this is. How much of the traffic
> generated by any site is really spiders? And is there a way
> to protect yourself?
>
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
First, a properly configured robots.txt file will prevent
most well-behaved spiders from accessing the URLs. They
check it before retrieving the target URL.
Second, make sure that all the encoded data is encoded as
arguments to the URL, not in the URL itself.
eg:
http://myserver.com/adcounter.cgi?src=excite&dest=mycustomer.com
instead of this type of encoding.
http://myserver.com/adcounter/excite/mycustomer.com
Most (but not all) spiders either don't follow URLs with ?
or $ parameters in them, or if they do, do so only once.
Finally, keep logs by user_agent, and you can easily
determine if you're being spidered, unless the spider is
seriously nasty and is masquerading as a browser.
Best
R
-----------------------------------------------------------------------
Woodhead's Law: "The further you are from your server, the more likely
it is to crash." The Webma<DEL><DEL>slave's Motto: "We Live To Serve."
http://selfpromotion.com/ The Net's only URL registration SHARESERVICE
mailto:0_at_0-0.ac My incredibly short alternate email address!
============= Mac OS X Carbon over Linux = a Coup D'Porte! ============
Alex, my 3-year-old, has a Y2K bug in his operating system. According
to him, the year after 1999 is "nineteen-ninety-ten."
========================================================================
------------------------------------------------------------------------
This week's Online Advertising Discussion List sponsors:
MarketWave and The Mining Co.
Which produces better DOLLAR returns? Advertising or Direct Mail?
Hit List can tell you. Click here to find out how.
http://www.marketwave.com/default.htm?AdName=OA6&AdSource=OA
---
NOBODY DELIVERS ONLINE BUYERS LIKE THE MINING CO. _at_Plan says Mining Co.
users are more likely to buy online than users of the average navigation
site. To reach them, contact Alan Wragg (awragg_at_miningco.com)
------------------------------------------------------------------------
========================================================================
Online Advertising Discussion List To Unsubscribe send UNSUBSCRIBE
http://www.o-a.com/ to online-ads-request_at_o-a.com
