How do I restrict all bots to crawl over one forum
disallow through robots.txt file !!
Make a file called robots.txt in your forum's root folder and add the following code in it;
User-agent: *
Disallow: /forumdisplay.php?fid=X
Replace X with the disallowed forum ID.
(2011-10-12, 03:20 PM)Yaldaram Wrote: [ -> ]Make a file called robots.txt in your forum's root folder and add the following code in it;
User-agent: *
Disallow: /forumdisplay.php?fid=X
Replace X with the disallowed forum ID.
This will stop all bots? or I have to add their user-agent?
It will stop all of them.