MyBB Community Forums

Full Version: Crawl errors, any advice?
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Hi all,

I have tested my crawl errors and wonder if this is normal?

I am using mybb vesion 1.6.1, with google seo plugin

In Sitemaps (4)

The 4 urls are all user profiles and they are all not found through 2 links...

http://the-chess-forum.com/sitemap-users.xml?page=1
Feb 3, 2011

http://the-chess-forum.com/memberlist.ph...username=H

I have 210 users so I dont understand why 4 users would cause a problem? they are still users on my site(not active though)



Not Found (62)

Most of these are calender links (maybe from before I had a robots text)
The problem I am most concerned about is this...

http://the-chess-forum.com/board.html?am...b&fw=p&pi=

I tracked the pages linking to this and the problem lies within the pgn viewer, I dont need the search bots to search this anyway, how would I disallow the pgn viewer in my robots text? If I added something like this...

"Disallow: /board.html" would that do it?

Finally I have Restricted by robots.txt ‎(315)

Within these 315 errors, I have links like this...

http://the-chess-forum.com/Forum-Chess-H...order=desc

Is this restricting bots access to my content? Is the example above normal? or is it stopping content under Forum, Chess Humour?


All this is probably ok, I would just like a second opinion to put my mind at rest, I guess there is no point finding out you have crawl errors if you do nothing about it.

If there is a fix, please help.

Cheers for reading,

Oliver
http://the-chess-forum.com

BUMP

Can anyone please offer some advice?

Thank you,

Oliver
I'm using mybb 1.6.1
http://the-chess-forum.com