MyBB Community Forums

Hi all, I have been reading the posts on robots. I have a private forum, for serious high end art collectors, that is closed to the general public, and I do not want google bots or any other bots for that matter probing around. How do I stop this ?

I noted a txt file called ro_bots but all the links to this seem to be broken.

cheers

Richard

---

Just some more on this, from what I can gather, the Bot's cannot see anything if they do not have permission ?. My guest settings are such that they cannot access anything. When looking at "who's online" however, I saw the following

Googlebot
IP: 66.249.71.67 [lookup] 	02:13 pm 	Emailing a User

Forgive my stupidity, but that surely could not be the case could it.

Cheers

Richard

All they're doing is clicking the email link, but they will just see a no permissions page.

Not sure about the ro_bots, do you mean creating a robots.txt file? If so, see this website: The Web Robots Pages

Your Googlebot is just looking at that page - it will be seeing an permission error message... the bot just follows the guest permissions...

If you want to just stop them coming altogether, create a file called robots.txt and put in it:

User-agent: *
Disallow: /

Thanks all, excellent work as usual.

Much appreciated

Richard

(2008-12-04, 09:21 AM)Matt_ Wrote: [ -> ]...If you want to just stop them coming altogether...

You can restrict what Google/bots see by using Meta tags. For example:

<meta name="robots" content="index, nofollow">

... will index the page, but will not follow anymore links on the page (essentially indexing just your frontpage). Or at least, in theory. Google somehow finds a way to index your pages - likely through links on other websites.

To prevent content from appearing in Google - see http://www.google.com/support/webmasters...topic=8459