MyBB Community Forums

What is the best robots.txt for a myBB Forum?

Quote:User-Agent: *
Disallow: /moderation.php
Disallow: /ratethread.php
Disallow: /report.php
Disallow: /reputation.php
Disallow: /sendthread.php
Disallow: /usercp.php
Disallow: /usercp2.php
Disallow: /newreply.php
Disallow: /newthread.php
Disallow: /editpost.php
Disallow: /private.php
Disallow: /search.php
Disallow: /refer.php
Disallow: /myawards.php
Disallow: /stats.php
Disallow: /member.php
Disallow: /memberlist.php
Disallow: /showteam.php
Disallow: /upgrade.php
Disallow: /showratings.php


User-agent: dotbot
Disallow: /

That is mine so far.

Because Google is saying I have this problem http://d.pr/i/wyNQ

Their fix just says to add User-agent: Mediapartners-Google to robots.txt

But dosent User-agent: * mean all?

From the looks of it you've just copied that from HF. There's no "best" robots.txt, you just block what you don't want being indexed by search engines.

Also it doesn't make much sense doing this:

User-agent: Mediapartners-Google
User-Agent: *

That's like putting up traffic cones to divert lorries in the other direction, then further up the road put up traffic cones to divert all other vehicles in the other direction.