Re: Tracking and Looksmart

From: Alexis D. Gutzman <>
Date: Tue 23 Jul 2002 20:41:11 -0500

Rather than trying to fool the search engines, why not do what they
recommend, and put a site map link on the home page, which the spider will
find and crawl. This will put all or most of your content within two clicks
of the home page, and make all of it readily accessible to the search
engine. As long as you make human navigation clear, most visitors won't
click on the site map. Putting the site map link below the fold will also
help, since a site we recently profiled found that ~90% of visitors never
even scrolled! If you need to be absolutely sure that only spiders make it
to the protected pages without getting the form, you can use server-side
detection to identify the footprint of the spider (there aren't that many
that really matter) and let the server show these agents the content behind
the barrier without seeing the form. Try the site map first, though and see
whether any humans go to the site map and find the content without going
through the form.

Make sure that if you don't want the content available without the visitor
seeing a form that you (a) set your meta tag to <META NAME="ROBOTS"
CONTENT="NOARCHIVE"> so that the SEs don't keep a copy of the page for
viewing directly from the SE (as cached content), (b) intercept traffic that
clicks on any deep links into your site to show them the barrier page
instead of the protected content.

Alexis D. Gutzman
Managing Editor, _MarketingSherpa's Buyer's Guide to Search Engine
Optimization Firms_
Managing Editor, Reports
MarketingSherpa <== Free "Top 10 eMarketing Mistakes" Report

Received on Tue Jul 23 2002 - 20:41:11 CDT


With an archive of more than 14,000 postings, since 1996 the Online Advertising Discussion List has been the Internet's leading forum focused on professional discussion of online advertising and online media buying and selling strategies, results, studies, tools, and media coverage. If you wish to join the discussion list, please use this link to sign up on the home page of the Online Advertising Discussion List.


Online Advertising Industry Leaders:

Local SEO with Video
Houston SEO
Austin Web Design

Add your company...

Local SEO with Video


Online Advertising Discussion List Archives: 2003 - Present
Online Advertising Discussion List Archives: 2001 - 2002
Online Advertising Discussion List Archives: 1999 - 2000
Online Advertising Discussion List Archives: 1996 - 1998

Online Advertising Home | Guidelines | Conferences | Testimonials | Contact Us | Sponsorship | Resources
Site Access and Use Policy | Privacy Policy

2323 Clear Lake City Blvd., Suite 180-139, Houston, TX 77062-8120
Phone: 281-480-6300
Copyright 1996-2007 The Online Advertising Discussion List, a division of ADASTRO Incorporated.
All Rights Reserved.