Log in

View Full Version : Twitter's robots.txt question:



Hemat
12-24-2010, 11:02 AM
Twitter's robots.txt, It shows everything is disallowed, but surprisingly search engines are crawling and indexing everybody's profiles pages, Why?