MyBB Community Forums

Full Version: Blocking "pid=" from Googlebot?
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Hi there,

I'm just wondering if it's wise to prevent Google from indexing multiple copies of a MyBB thread by using robots.txt to disallow links that end with "pid=". So basically adding this to robots.txt:

Disallow: /forum/*pid=

Is this a good idea?

Thanks.
Also, is it smart to use robots.txt to block user profile pages? Again, seems like a good idea to me but I'm not an expert.
I think it's a good idea because sometimes the lastpost link is showing first instead of the real thread and it's really annoying to manually going to the first post.

Not sure if it's going to work:
User-agent: *
Disallow: /forum/*?*pid=*
Disallow: /forum/*&action=lastpost
Disallow: /forum/*&action=newpost

If you use Search Friendly URLs you need to use these:
User-agent: *
Disallow: /forum/*-post-*.html
Disallow: /forum/*-lastpost.html
Disallow: /forum/*-newpost.html