MyBB Community Forums

I have uploaded the standard robots.txt file to the root for my forum.

This is what is included:

User-Agent: *
Disallow: /moderation.php
Disallow: /ratethread.php
Disallow: /report.php
Disallow: /reputation.php
Disallow: /sendthread.php
Disallow: /usercp.php
Disallow: /usercp2.php
Disallow: /postings.php
Disallow: /newreply.php
Disallow: /newthread.php
Disallow: /editpost.php
Disallow: /private.php
Disallow: /search.php

That looks fine.

But also, my site is indexed by google and is showing pages like ratethread.php and misc.php, i'm not sure if these should be indexed?

Should i also add these files to the robots.txt aswell to stop them from getting indexed.

Also what about /images
/syndication.php
/archive/
/task.php
/css.php
/attachment.php

Should i also add the above files to the robots.txt?

Let me know which files should i add.

Thanks

There's an example robots.txt included in the Google SEO plugin (but I do not claim it to be perfect, just as another example)

I'll check it out. Thanks

---

Ok, i've checked out your example robots.txt from the Google SEO Plugin. It does include some extra stuff but doesn't include some of the stuff i posted before.

Should i allow these:

/images
/syndication.php
/archive/
/task.php
/css.php
/attachment.php