Work at Home Forum - An online community of people working at home.


This is a read-only forum. To Register and post topics, visit our main page.


 robots.txt


 
Forum Index -> Internet Marketing
Author Message
Gobala Krishnan
Pro
Pro


Joined: 17 Jun 2004
Posts: 58
Location: Malaysia
PostPosted: Wed Jun 30, 2004 4:56 pm    Post subject: robots.txt
Hi everyone,

Could anyone advice me whether it's necessary to have a robots.txt file in your site directory?

If it is, then what should it contain?

Thanks,

Bala
Back to top
Vishal P. Rao
Forum Admin
Forum Admin


Joined: 02 May 2004
Posts: 112
PostPosted: Thu Jul 01, 2004 4:05 am    Post subject:
Hi Bala,

It's not at all necessary to have a robots.txt file in your site directory. Robots.txt is simply used to specify which areas of your site should be spidered by search engines and which should not be. You can also explicitly exclude certain spiders from spidering your site.

To learn more about it, visit the following url:
http://www.searchengineworld.com/robots/robots_tutorial.htm

Best wishes Very Happy

Vishal
Back to top
Display posts from previous:   
Forum Index -> Internet Marketing All times are GMT
Page 1 of 1

 



© 2004-2005 Work at Home Forum. Powered by phpBB © 2001, 2005 phpBB Group