(2012-12-02, 07:53 PM)frostschutz Wrote: [ -> ]If pages are indexed from a time before you disallowed them in robots.txt, you can issue an URL removal request specific to the subset of URLs (same rules as in robots.txt) you want removed. I'd be a bit careful with those removal requests, though.
Yes, these are older threads from the time that the owner started the forum and he had no robots.txt. May I ask why I should be careful? I have the Google 404 page enabled, will that help?
(2012-12-02, 07:53 PM)frostschutz Wrote: [ -> ]Personally, I don't disallow action=lastpost. With the canonical tag provided by Google SEO Meta, Google merges those pages to the canonical name quickly. When doing a site search (site:www.japanisch-netzwerk.de inurl:action=lastpost) it currently finds only one thread page, which has a post written today. In a day or two this should no longer appear but instead be picked up under its canonical name.
I see. Thank you for explaining that.
(2012-12-02, 07:53 PM)frostschutz Wrote: [ -> ]That's a Google thing - they like to list URLs even if they are not allowed to crawl them, if links to those URLs appear on other indexed pages. You could get rid of them with an URL removal request, or by letting Google crawl those pages and specifically serve them with a noindex header
But neither of those solutions are really preferrable...
Yes, there is a link to an old forum (
http://blackcanvas.net/forum-16.html ) and that is returning a 404 page, but I also have an admin forum - and that is showing as a 404 in Webmaster tools. But how do google see it? They are in as guest - that is the setting I have for Spiders.
http://blackcanvas.net/forumdisplay.php?...datecut=20
(2012-12-02, 11:06 PM)Leefish Wrote: [ -> ]But how do google see it?
Ask Google (Fetch as Googlebot feature in the webmaster tools) for a definite answer on that
Removal request: Too much may be removed if your request is not specific enough and/or if there is a problem with your site.
Sorry not to be clear - what I mean is - how does Google see the url to tell me it is a 404?
In webmaster tools it says "404" - but how do they get to see it to say they can't see it? Do you see what I mean? It is an admin only forum and not visible to guests.
EDIT: Doh. Rechecked permissions - guests could view threads but not the forum. Whoops. Now fixed
Is there a link to it anywhere? Was there a link to it anywhere?
I can post links to all mybb.com forums hidden or not: just put in numbers 1..100000
http://community.mybb.com/forum-123.html
But I just noticed that the redirect permission setting doesn't seem to be working as intended. Did it never work, or did it break at some point, that is the question... either way, I'll have to fix it.
There was a link in the Webmaster tools crawl stats, it was a 404.
It said:
Googlebot couldn't crawl this URL because it points to a non-existent page. Generally, 404s don't harm your site's performance in search, but you can use them to help improve the user experience
And in the linked from in the same page it said:
http://www.blackcanvas.net/forumdisplay.php?fid=12 << which is my admin forum. So I was just curious.
I have a question, Google bot keeps viewing calendar.php. I have disabled calendar access and also in the robots.txt but still I keep seeing google on calendar. Thank You!
It may take a while if you've disabled it just recently.
Sorry, I still do not understand how to make parent URLs like I can do in phpBB with phpBB Ultimate SEO Mod in MyBB with Google SEO. If you do not understand what I mean, it's this: "mywebsiteurl.com/section name/discussion-name-with-lower-cases.html" - How can I do that? I want this kind of URLs for discussions and sections/categories!
Please refer to the documentation for that.
I was chatting to a friend about the Google SEO and she was saying it would be better for SEO if the url came first with the forum/thread identifier after it. Is that actually possible though? I did look in the documentation but I cannot figure it out at all.