Re: Tracking and Looksmart

From: John Smart <JSmart_at_InternetDesign.com>
Date: Tue 23 Jul 2002 20:38:50 -0500

"Jill Paskoff" <jillp_at_specsource.net> wrote:

>Our site is database driven and I believe spiders don't crawl it because
>some selection is required by the user to continue (a form action). Once
>the initial selection is made, the links are infinite. Is there a way to
>get the spiders past the initial selection process so that they can crawl
>the remaining pages?

There are two ways to get past this issue. The one is to include a static
link, something like <a
href=http://www.domain.com/script.cgi?hi=there&how=are_you>link</a>
Now, this still has a question mark, but there is a way round that - use
some php code. Have the page as, I don't know "/phplink.php". Set
phplink.php to read:

<?php
header("Location: http://www.domain.com/script.cgi?hi=there&how=are_you");
?>

This will get the spider or the visitor to the dynamic page without them
knowing that they are at a dynamic page. (Or at least without knowing they
have used a link with question marks in it).

Now, before you accuse me of not reading your mail carefully, here is where
I am going with this:
Give the static link a name such as "Example query" (Name this to what ever

is appropriate!) and fill in the data to give dummy results. This will allow
the search engines to pick up on the dummy data, and still let users fill in
their own data to access it.

And finally, for those who are concerned that search engines don't like
.php, there are two things to do. One - go to your favorite search engine
and perform a search for "index.php" (about 2.2 million results on Google).
2. Don't call it a php file! The following is for Apache servers (which most
are!) I don't know how to set this on others, but it should be well
documented. Create a file in your web site root folder called .htaccess - in
here put the following line:

AddType application/x-httpd-php .html

What this does is tells the server that any .html file could be a .php file,
and it should check them for php tags. If you have a HUGE site, full of
static pages, make sure you only put this in the folders where you are
placing cloaked php files otherwise you may see some reduction in
performance (though not much unless it really is a huge and busy site).

I hope this helps you get the results you want.

Kindest regards,
John Smart
InternetDesign.com - A Human Touch in a Digital World (TM)
JSmart_at_InternetDesign.com
http://www.InternetDesign.com






Received on Tue Jul 23 2002 - 20:38:50 CDT