2011-06-08, 01:55 AM
Hi there,
I'm just wondering if it's wise to prevent Google from indexing multiple copies of a MyBB thread by using robots.txt to disallow links that end with "pid=". So basically adding this to robots.txt:
Disallow: /forum/*pid=
Is this a good idea?
Thanks.
Also, is it smart to use robots.txt to block user profile pages? Again, seems like a good idea to me but I'm not an expert.
I'm just wondering if it's wise to prevent Google from indexing multiple copies of a MyBB thread by using robots.txt to disallow links that end with "pid=". So basically adding this to robots.txt:
Disallow: /forum/*pid=
Is this a good idea?
Thanks.
Also, is it smart to use robots.txt to block user profile pages? Again, seems like a good idea to me but I'm not an expert.