MyBB Community Forums
[How To?] Blocking Google Bots for Some Pages like "search.php, memberlist.php" - Printable Version

+- MyBB Community Forums (https://community.mybb.com)
+-- Forum: 1.8 Support (https://community.mybb.com/forum-175.html)
+--- Forum: General Support (https://community.mybb.com/forum-176.html)
+--- Thread: [How To?] Blocking Google Bots for Some Pages like "search.php, memberlist.php" (/thread-163010.html)



Blocking Google Bots for Some Pages like "search.php, memberlist.php" - diskopat - 2014-11-18

How can we do it?
I dont use seo plugin, I just use SEF URL feature. So i haven't robot.txt or sitemap..
I can do this from htaccess i think. But i don't know how to do this. 
I  appreciate your help.


RE: Blocking Google Bots for Some Pages like "search.php, memberlist.php" - StefanT - 2014-11-18

Set up a robots.txt. That's the way you control search engine bots. Wink


RE: Blocking Google Bots for Some Pages like "search.php, memberlist.php" - .m. - 2014-11-18

here is a sample robots.txt file (assuming that forum is at public_html folder on files server)

User-Agent: *
Disallow: /captcha.php
Disallow: /editpost.php
Disallow: /memberlist.php
Disallow: /misc.php
Disallow: /modcp.php
Disallow: /moderation.php
Disallow: /newreply.php
Disallow: /newthread.php
Disallow: /online.php
Disallow: /printthread.php
Disallow: /private.php
Disallow: /ratethread.php
Disallow: /report.php
Disallow: /reputation.php
Disallow: /sendthread.php
Disallow: /search.php
Disallow: /task.php
Disallow: /usercp.php
Disallow: /usercp2.php
Disallow: /calendar.php
Disallow: /newpoints.php
Disallow: /*action=emailuser*
Disallow: /*action=nextnewest*
Disallow: /*action=nextoldest*
Disallow: /*action=emailuser*
Disallow: /*year=*
Disallow: /*action=weekview*
Disallow: /*action=nextnewest*
Disallow: /*action=nextoldest*
Disallow: /*sort=*
Disallow: /*order=*
Disallow: /*mode=*
Allow: /



RE: Blocking Google Bots for Some Pages like "search.php, memberlist.php" - diskopat - 2014-11-18

Thanks a lot both big guys..