MyBB Community Forums

Full Version: MyBB robot.txt
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Pages: 1 2
I have my own honeypot. My robot.txt file contains Disallow lines whose target records the visitor as a bad robot. Bad bots are blocked via .htaccess
This is what I use.
Sitemap: http://yuvapix.com/sitemap-index.xml

User-agent: *
Disallow: /captcha.php
Disallow: /editpost.php
Disallow: /misc.php
Disallow: /modcp.php
Disallow: /moderation.php
Disallow: /member.php?action=emailuser
Disallow: /member.php?action=login
Disallow: /member.php?action=logout
Disallow: /member.php?action=lostpw
Disallow: /member.php?action=register
Disallow: /memberlist.php
Disallow: /misc.php?action=markread
Disallow: /newreply.php
Disallow: /newthread.php
Disallow: /online.php
Disallow: /printthread.php
Disallow: /private.php
Disallow: /ratethread.php
Disallow: /report.php
Disallow: /reputation.php
Disallow: /sendthread.php
Disallow: /stats.php
Disallow: /task.php
Disallow: /usercp.php
Disallow: /usercp2.php
Disallow: /calendar.php
Disallow: /*action=emailuser*
Disallow: /*action=nextnewest*
Disallow: /*action=nextoldest*
Disallow: /*action=emailuser*
Disallow: /*year=*
Disallow: /*action=weekview*
Disallow: /*action=nextnewest*
Disallow: /*action=nextoldest*
Disallow: /*sort=*
Disallow: /*order=*
Disallow: /*mode=*

User-agent: dotbot 
Disallow: /
I use Google SEO plugin robots.txt with a few extras
Pages: 1 2