MyBB Community Forums

I've been reading through the stats on my forum that google webmaster tools provides and noticed that there are a lot of stuff that is listed under the "not found" tab in the crawl error section. all this stuff looks like old links back when the forum was ran on phpbb. should I block them from being indexed since they don't get redirected now?

for example, almost everything in the "not found" tab is url.com/viewtopic.php?f=1&t=1, and etc. should I add /viewtopic.php in my robots.txt file? I don't think mybb uses viewtopic.php for anything, right?

also, in the http tab under crawl errors, I'm seeing a lot for newreply.php and ratethread.php, but those should be getting blocked, per the robots.txt..if I have it set up properly.

here's how my robots.txt file looks:

Sitemap: http://mysite.com/sitemap-index.xml

User-Agent: *
Disallow: /captcha.php
Disallow: /editpost.php
Disallow: /misc.php
Disallow: /modcp.php
Disallow: /moderation.php
Disallow: /newreply.php
Disallow: /newthread.php
Disallow: /online.php
Disallow: /printthread.php
Disallow: /private.php
Disallow: /ratethread.php
Disallow: /report.php
Disallow: /reputation.php
Disallow: /search.php
Disallow: /sendthread.php
Disallow: /task.php
Disallow: /usercp.php
Disallow: /usercp2.php
Disallow: /calendar.php
Disallow: /*action=emailuser*
Disallow: /*action=nextnewest*
Disallow: /*action=nextoldest*
Disallow: /*year=*
Disallow: /*action=weekview*
Disallow: /*action=nextnewest*
Disallow: /*action=nextoldest*
Disallow: /*sort=*
Disallow: /*order=*
Disallow: /*mode=*
Disallow: /*datecut=*
Allow: /

if you block viewtopic nothing but the index would really be read by search engines.

(2011-11-07, 11:56 PM)Alex Smith Wrote: [ -> ]if you block viewtopic nothing but the index would really be read by search engines.

I should of stated that my forum is ran on mybb now, I converted it from phpbb. does mybb use viewtopic.php to display anything at all? don't they use showthread.php? if that's the case, wouldn't it be beneficial to block viewtopic.php so that it doesn't index dead links?

you can add viewtopic.php to robots.txt file ;
you can also add viewtopic as a parameter for the Google bot to ignore such URLs

( image for reference )

(2011-11-08, 03:13 AM)ranjani Wrote: [ -> ]you can add viewtopic.php to robots.txt file ;
you can also add viewtopic as a parameter for the Google bot to ignore such URLs

( image for reference )

thanks, I did both.