Webmasters Fight
free resources and tools in your hands
 

Rombots.txt Making Tool

1.
Allow all web spiders :
Or choose webspiders that you want to allow :
Google:
Yahoo
Msn
2.
If have a problem with Yahoo spider (Slurp) and you want to slow it :
3.
Folders names to avoid , such as (admin) and (cgi-bin) folders :
//
//
//
//
//
//
//
4.
Do you want to prevent spam bot from crwoling :

© webmastersfight.com all rights reserved | graphicdesign by snop