MyBB Community Forums

Full Version: Spider / Bots Feature
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
I'm not sure this is the correct place to post this but I'm having a bit of difficulty finding this.

The feature called "Spiders / Bots" says that, "This section allows you to manage the search engine spiders & bots automatically detected by your forum. You're also able to see when a particular bot last visited."

Does this automatically block them out or allow them to access the forum?
Or, does that depend on which permissions their user-group is allowed?

If it doesn't block them out, is there a plugin available which will block them?
Fassim Anti Forum Spam
Quote:this plug-in will block known spammers/bots from registering via IP, email address, anonymous proxy, top email domains, top ISP - (Internet ...


if you want to block access to known bots then you may disallow them thru robots.txt file

see also : Add Search Engine Bots List
Out of interested why do you want to block bots from visiting your website?
Whatever bots/spiders are listed in the ACP, you can edit each one to put them into a particular group. Create a "Banned Bots" group, set it as a "Banned Group" and then put the bots you don't want to access your site into that group.
(2011-07-25, 07:05 PM)pompano Wrote: [ -> ]Out of interested why do you want to block bots from visiting your website?

Using it as an internal forum. No registered users, all users are made by the "Add New User" button.
Where is the robots.txt file located?
robots.txt can be added to your site root directory (here is a simple example which disallows bots using download links here)