Re: Log analysis in the era of cache servers...

From: John Charlesworth <jcharlesworth_at_bellacoola.com>
Date: Thu 27 Jan 2000 12:28:01 -0800

IAN LEICHT WROTE:
> I believe that considerable amounts of traffic to our
> site is never making to our servers due to caching
> technology. It is important for us to be able to have
> as accurate as possible of an understanding of our
> traffic patterns.

Ian is quite right--we have been tracking the effect of
caching for over 2-1/2 years. While browser- and
proxy-caching is very effective at improving the
apparent speed of the Internet, it has the unfortunate
side-effect of distorting the usage data that a web
server can collect. Many page requests never make it
as far as your server and therefore won't appear in
your server logs.

Our measurements show the effects of caching /
undercounting have been steadily increasing--currently
the undercounting ranges from 11% to 66%, depending on
where a page fits into a site's traffic pattern. For
instance, a "hub" page will experience much more
undercounting than a "leaf" page.

For those interested in learning more, our white paper
"Caching Explained: A Simple Guide to why you're
undercounting your traffic by up to 40%" covers this
topic in detail, including:
- What browser- and proxy-caching is,
- What the problem is, and why it's important to you, and
- Three solutions, their pros and their cons

http://www.bellacoola.com/html/caching_explained.htm

Ian proposed three approaches of his own and asked:

> Does anyone have any recommendations for solving
> this problem?

Once you dig in to it, the problem is a little more
complex than Ian may realize. We should know--we've
been tackling this problem for longer than anyone else
we know.

And, although self-promotion is discouraged on this
list, Ian did ask for recommendations :-)

BellaCoola has been fine-tuning its cache-correct log
file service for 2-1/2 years. We produce a family of
"Sniffers" that work together to measure the true
viewership (or click-through) of Internet content,
including:
- web pages (even if spread across multiple servers,
  or on remotely-hosted/secure e-commerce servers
  like Digital River)
- outbound text links
- banners (either on your site, or on external sites)
- rich-media content (real audio streams etc)
- newsgroup postings
- FTP downloads
- mailto tags

Our Sniffers are used by many well-known companies who
are serious about their tracking and want a proven,
scalable solution that integrates with their favorite
in-house analysis tools.

I hope this helps--if I can answer any of your
questions, feel free to contact me off list. If I get
enough questions about a given topic, I'll put
together another white paper so everyone can learn.

Cheers,
.../j

---------------- <mailto:jcharlesworth_at_bellacoola.com> ----------------
John Charlesworth, President BellaCoola Software Corporation
          Do you know which promotions are *really* working??
       Sign up for BellaCoola's all-in-one web tracking service!
Complete cache-corrected measurement of email, banners, and page-views.
         Everything you need to track marketing effectiveness.
-------------------- <http://www.bellacoola.com/?prw3> ------------------





Received on Thu Jan 27 2000 - 14:28:01 CST