MyBB Community Forums

Hello,

Question: I've had a MyBB forum up and running successfully for quite some time and upgraded as and when new releases come available. All works absolutely brilliantly. For some reason there's now permanetly a GoogleBot logged into my forum and it's been there for about three weeks. Literally ALWAYS there looking at calendar, posts or "no permission" pages. I've attempted to block some access through "robots.txt" (like to calendar.php) but it still stays in place. Now I'm starting to think it's caught up in some sort of weird loop inside my forum and can't get out - as you can probably see, I'm not an expert at any of this at all. Should I just ignore it or do you recommend banning all bots from MyBB forums to avoid this sort of thing? I didn't really want to do that so that it's still referenced on search engines - what do you recommend?

You don't have to worry about it, it will visit your forums every now and then. If you block bots using robots.txt they just see a 'no permission' page.

destroyer Wrote:You don't have to worry about it, it will visit your forums every now and then. If you block bots using robots.txt they just see a 'no permission' page.

OK, thank you. I'll put my "ignoring" head on and pay no attention to it.

...GoogleBot? ...What GoogleBot?

PeteSa Wrote:GoogleBot logged into my forum and it's been there for about three weeks.

You are lucky to have GoogleBot only. Wait till you get the other two: Yahoo! Slurp, and MSN. Those three are like parasites, they never leave my forum. What is strange to me is when I see them "using pm system", and the inactive forums...believe it or not!

Regards

maatty Wrote:

PeteSa Wrote:GoogleBot logged into my forum and it's been there for about three weeks.

You are lucky to have GoogleBot only. Wait till you get the other two: Yahoo! Slurp, and MSN. Those three are like parasites, they never leave my forum. What is strange to me is when I see them "using pm system", and the inactive forums...believe it or not!

Regards

They try to access every single url possible and retrieve a copy of that page - However, they will just recieve a no permissions error trying to access those pages

Tikitiki Wrote:They try to access every single url possible and retrieve a copy of that page - However, they will just recieve a no permissions error trying to access those pages

Thanks for the clarification Tikitiki.

Regards

If you want to stop them from accessing pages they don't have permission to use (and therefore wasting your bandwidth) you can use a robots.txt file.
Above is an example of what your robots.txt file would look like - that is MyBB's file.

More information on robots.txt files: http://en.wikipedia.org/wiki/Robots.txt