robots.txt

[marain]

Here is my robots.txt file:

User-agent: *
#
# See Wikipedia article for specs
#
Disallow: /test.*
Disallow: /pgp_key.php
Disallow: /googlef110bd7f27a60db7.html

My web guru set up my site structure such that efffective URL for individual pages are in the form of marainlaw.com/page.php?here=[pagename]. I wanted spiders to not visit marainlaw.com/page.php?here=test, but I see that googlebot went there anyway. Should my robots.txt file be specified differently?

[bernie1227]

Well /*? Is anything with a session id, so you could do something like


User-agent: *
#
# See Wikipedia article for specs
#
Disallow: /test.*
Disallow: /pgp_key.php
Disallow: /googlef110bd7f27a60db7.html
Disallow: /*?here=test