robots/crawls

[neilkw]

hi

I've just taken over the design of a website and one of the tasks I have is to increase the traffic to the site.

The previous designer has installed a robots disallow file in the root. Would deleting this help roboets etc crawl the site and increrase my google rating?

The site doesn't have any password protected areas or any e-commerce areas so from that point of view I'm not worried about certain pages being cached.

thanks

[jscheuer1]

Are you talking about robots.txt? It all depends upon how this or other similar type files are configured. If it is denying access to the entire site for no reason, yes - get rid of it or at least modify it. However, usually these type of files are used to limit spiders and robots to certain pages that would be logical starting off points in the site and to deny access to pages that should be displayed in frames just for example. I used something like this once to make sure a site was indexed the way I wanted it to be with files that were linked to from many pages only being represented as linked from one or two pages that were the most logical lead ins for them.

[neilkw]

ok yes, sorry, it is a robots.txt file
I have no frameset, so I am thinking that I may pull the .txt file and let the robots have a good crawl as currently our google rating is not amazingly good.

[jscheuer1]

One of the best ways to boost your Google though, is to have other sites link to yours.