use a robots.txt file to prevent crawling to those pages !! (
example)
(2011-09-29, 02:34 AM)ranjani Wrote: [ -> ]use a robots.txt file to prevent crawling to those pages !! (example)
Thanks Ranjani is this example robot.txt file exactly match for these errors or it is just an example.
^ though it is an example , it also matches your requirement AND
if you use
Google SEO plugin then it has a perfect robots.txt file
(2011-09-29, 02:54 AM)ranjani Wrote: [ -> ]^ though it is an example , it also matches your requirement AND
if you use Google SEO plugin then it has a perfect robots.txt file
Yes I use google SEO plugin too. But in my google webmaster tools shows that I have no robot.txt file. So I upload a file manually. Then why is that?
(2011-09-29, 02:54 AM)ranjani Wrote: [ -> ]^ though it is an example , it also matches your requirement AND
if you use Google SEO plugin then it has a perfect robots.txt file
Thanks for your quick reply. I really want to get solved this problem. I saw this today. These errors are from long time ago.
^ Google SEO plugin doesn't install robots.txt file.
The plugin has below rules in the example file ( robots.txt.example )
Sitemap: http://yoursite/MyBB/sitemap-index.xml
User-Agent: *
Disallow: /MyBB/captcha.php
Disallow: /MyBB/editpost.php
Disallow: /MyBB/misc.php
Disallow: /MyBB/modcp.php
Disallow: /MyBB/moderation.php
Disallow: /MyBB/newreply.php
Disallow: /MyBB/newthread.php
Disallow: /MyBB/online.php
Disallow: /MyBB/printthread.php
Disallow: /MyBB/private.php
Disallow: /MyBB/ratethread.php
Disallow: /MyBB/report.php
Disallow: /MyBB/reputation.php
Disallow: /MyBB/search.php
Disallow: /MyBB/sendthread.php
Disallow: /MyBB/task.php
Disallow: /MyBB/usercp.php
Disallow: /MyBB/usercp2.php
Disallow: /MyBB/calendar.php
Disallow: /MyBB/*action=emailuser*
Disallow: /MyBB/*action=nextnewest*
Disallow: /MyBB/*action=nextoldest*
Disallow: /MyBB/*year=*
Disallow: /MyBB/*action=weekview*
Disallow: /MyBB/*action=nextnewest*
Disallow: /MyBB/*action=nextoldest*
Disallow: /MyBB/*sort=*
Disallow: /MyBB/*order=*
Disallow: /MyBB/*mode=*
Disallow: /MyBB/*datecut=*
Allow: /
note: you have to modify the path in above as per your forum URL ..
There is no such a xml file in my site although I user Google SEO plugin. This my tobot.txt file Please advice me. Thanks
User-agent: *
Allow: /
same problems here, i have the 'google seo' robots.txt file in the forum's root, but seems to be ignored by google.
I've noticed that by disabling google seo, all the 403 errors vanish, you can check one of your links here:
http://webconfs.com/http-header-check.php and see the status change from 403 to 200, that should resolve the problems we see in webmaster tools ..but i don't understand where to change the code ..i would not disable google seo to avoid those 403 errors, suggestions?
(2011-09-29, 03:35 AM)freeeducation365 Wrote: [ -> ]There is no such a xml file in my site although I user Google SEO plugin.
User-agent: *
Allow: /
Erm, That will stop SEO.
That's telling every bot to "Bugger Off" in a niceish sort of way...
(2012-08-28, 12:48 AM)lunaraurora Wrote: [ -> ]suggestions?
CHMod 644 all your MyBB files.
(2012-08-28, 01:59 AM)bcousins Wrote: [ -> ] (2011-09-29, 03:35 AM)freeeducation365 Wrote: [ -> ]There is no such a xml file in my site although I user Google SEO plugin.
User-agent: *
Allow: /
Erm, That will stop SEO.
That's telling every bot to "Bugger Off" in a niceish sort of way...
(2012-08-28, 12:48 AM)lunaraurora Wrote: [ -> ]suggestions?
CHMod 644 all your MyBB files.
Wrong. That's telling bots they can access everything - notice it says
Allow: /
@lunaraurora post the contents of your robots.txt file please and your URL.
![[Image: image-04F7_4E83D541.jpg]](https://camo.mybb.com/93ad1f505c8712112859db0b03588603121644b1/687474703a2f2f66726565656475636174696f6e3336352e696e666f2f70686f746f73686172652f696d6167652d303446375f34453833443534312e6a7067)