MyBB Community Forums

Full Version: Google versus memberlist
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Is it possible to disable Google to crawl memberlist.

Eriva
You can do any of these:
1. Use robots.txt to disallow memberlist.php
2. Hide the memberlist from guests
3. Put the bots in their own usergroup and deny that group view permissions to the memberlist
(2010-12-31, 09:35 AM)- G33K - Wrote: [ -> ]You can do any of these:
1. Use robots.txt to disallow memberlist.php
2. Hide the memberlist from guests
3. Put the bots in their own usergroup and deny that group view permissions to the memberlist

The following text wil be succesfull?

User-agent: Google
Allow: /
User-agent: *
Disallow: /forum/memberlist.php

Allow: /

---
Where do I place the file.
In the root on the server?


Erica

(2010-12-31, 10:54 AM)erica Wrote: [ -> ]
(2010-12-31, 09:35 AM)- G33K - Wrote: [ -> ]You can do any of these:
1. Use robots.txt to disallow memberlist.php
2. Hide the memberlist from guests
3. Put the bots in their own usergroup and deny that group view permissions to the memberlist

The following text wil be succesfull?

User-agent: Google
Allow: /
User-agent: *
Disallow: /forum/memberlist.php

Allow: /

---
Where do I place the file.
In the root on the server?


Erica

Allows should come at the end since you need to tell the bots what to disallow first.
A simple one like this would do:

User-Agent: *
Disallow: /forum/memberlist.php
Allow: /

Yes you would put it in the website root
(2010-12-31, 11:07 AM)- G33K - Wrote: [ -> ]
(2010-12-31, 10:54 AM)erica Wrote: [ -> ]
(2010-12-31, 09:35 AM)- G33K - Wrote: [ -> ]You can do any of these:
1. Use robots.txt to disallow memberlist.php
2. Hide the memberlist from guests
3. Put the bots in their own usergroup and deny that group view permissions to the memberlist

The following text wil be succesfull?

User-agent: Google
Allow: /
User-agent: *
Disallow: /forum/memberlist.php

Allow: /

---
Where do I place the file.
In the root on the server?


Erica

Allows should come at the end since you need to tell the bots what to disallow first.
A simple one like this would do:

User-Agent: *
Disallow: /forum/memberlist.php
Allow: /

Yes you would put it in the website root
Thanks