How? :p
I'd rather not have that little bugger crawling around the forum looking for email addresses to spam, so any way of banning it? or perhaps adding a ban by ip address function if banning isn't currently possible?
Yes. Banning serchbot could be an idea. What you might have to do is (dont quote me on this) wait until it comes online on your forum then find the I.P adress of the bot by logging onto your admin and looking on users online, this should show every user onlines I.P adress and ban it. By doing this, the serchbot will be banned from your site.
Why do you have email addresses readily available on your forum?
Like any good crawler, the MSN search bot listens to robots.txt. All you need to do is disallow the bot's user agent with a robots.txt and you're done.
Info about robots.txt:
http://www.robotstxt.org/wc/norobots.html
BTW: The MSN Search bot does not look for emailaddresses to spam. It just indexes your site for
http://search.msn.com. There are hundreds of bots that do the same thing like the MSN bot, all do it for their own searchengine or index site. The problem is not searchbots crawling your site; this is in fact a good thing if you want people to visit your site.
Oh, banning the bot by IP isn't really effective. MSN probably has hundreds or even more machines that are crawling as MSN Search bot. That would mean you have to wait for every one of them to visit your forum, and ban each of their IP. A god robots.txt will keep them all away at once.
If you are afraid that emailaddresses will get picked up by spamcrawlers (MSN Search in
not a spam crawler), then just make sure you don't have any emailaddresses on your site. Either mask them like "menthix AT hotmail.com" or something like that, or just put your emailaddress in your MyBB profile (MyBB hides emailaddresses from bots). The emailaddress in the MyBB profile for your MSN account is not hidden btw.
I finally put a
--
User-agent: *
Disallow: /forum/
--
in my robots.txt to be sure. I don't want them neither, I don't need guests on my Board
And I don't want to see the bots in my online-list.
So I just have to wait what happens, I guess, they will still try to access.
FraGe
I like to have the MSN SEARCH Bot on my board, can I "let it come" ?
Thanks!
Where can I add my site for some other search engines?
(Yahoo Slurp, ...)
http://search.yahoo.com/info/submit.html
http://www.google.com/addurl/
Just look around on the search pages for things like "Submit your site" and "Webmaster info" and you'll find it.
You do not always need to submit your site btw. Search engine bots will come sooner or later when other sites, which are already listed on search engines, link to your site.