2011-02-06, 04:56 PM
Hi all,
I have tested my crawl errors and wonder if this is normal?
I am using mybb vesion 1.6.1, with google seo plugin
In Sitemaps (4)
The 4 urls are all user profiles and they are all not found through 2 links...
http://the-chess-forum.com/sitemap-users.xml?page=1
Feb 3, 2011
http://the-chess-forum.com/memberlist.ph...username=H
I have 210 users so I dont understand why 4 users would cause a problem? they are still users on my site(not active though)
Not Found (62)
Most of these are calender links (maybe from before I had a robots text)
The problem I am most concerned about is this...
http://the-chess-forum.com/board.html?am...b&fw=p&pi=
I tracked the pages linking to this and the problem lies within the pgn viewer, I dont need the search bots to search this anyway, how would I disallow the pgn viewer in my robots text? If I added something like this...
"Disallow: /board.html" would that do it?
Finally I have Restricted by robots.txt (315)
Within these 315 errors, I have links like this...
http://the-chess-forum.com/Forum-Chess-H...order=desc
Is this restricting bots access to my content? Is the example above normal? or is it stopping content under Forum, Chess Humour?
All this is probably ok, I would just like a second opinion to put my mind at rest, I guess there is no point finding out you have crawl errors if you do nothing about it.
If there is a fix, please help.
Cheers for reading,
Oliver
http://the-chess-forum.com
BUMP
Can anyone please offer some advice?
Thank you,
Oliver
I'm using mybb 1.6.1
http://the-chess-forum.com
I have tested my crawl errors and wonder if this is normal?
I am using mybb vesion 1.6.1, with google seo plugin
In Sitemaps (4)
The 4 urls are all user profiles and they are all not found through 2 links...
http://the-chess-forum.com/sitemap-users.xml?page=1
Feb 3, 2011
http://the-chess-forum.com/memberlist.ph...username=H
I have 210 users so I dont understand why 4 users would cause a problem? they are still users on my site(not active though)
Not Found (62)
Most of these are calender links (maybe from before I had a robots text)
The problem I am most concerned about is this...
http://the-chess-forum.com/board.html?am...b&fw=p&pi=
I tracked the pages linking to this and the problem lies within the pgn viewer, I dont need the search bots to search this anyway, how would I disallow the pgn viewer in my robots text? If I added something like this...
"Disallow: /board.html" would that do it?
Finally I have Restricted by robots.txt (315)
Within these 315 errors, I have links like this...
http://the-chess-forum.com/Forum-Chess-H...order=desc
Is this restricting bots access to my content? Is the example above normal? or is it stopping content under Forum, Chess Humour?
All this is probably ok, I would just like a second opinion to put my mind at rest, I guess there is no point finding out you have crawl errors if you do nothing about it.
If there is a fix, please help.
Cheers for reading,
Oliver
http://the-chess-forum.com
BUMP
Can anyone please offer some advice?
Thank you,
Oliver
I'm using mybb 1.6.1
http://the-chess-forum.com