MyBB Community Forums

Full Version: crawling errors
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
google couldnt crawl into my site and i always get errors.
but today i placed this robots.txt in my root directory 
i copied and placed them in a note pad and saved it as robots.txt and placed it the public html of the root directory.
this is safe and okay?


User-Agent: *
Disallow: captcha.php
Disallow: editpost.php
Disallow: misc.php
Disallow: modcp.php
Disallow: moderation.php
Disallow: newreply.php
Disallow: newthread.php
Disallow: online.php
Disallow: printthread.php
Disallow: private.php
Disallow: ratethread.php
Disallow: report.php
Disallow: reputation.php
Disallow: search.php
Disallow: sendthread.php
Disallow: task.php
Disallow: usercp.php
Disallow: usercp2.php
Disallow: calendar.php
Disallow: *action=emailuser*
Disallow: *action=nextnewest*
Disallow: *action=nextoldest*
Disallow: *year=*
Disallow: *action=weekview*
Disallow: *action=nextnewest*
Disallow: *action=nextoldest*
Disallow: *sort=*
Disallow: *order=*
Disallow: *mode=*
Disallow: *datecut=*
Allow: /

i saved it in a notepad as robots.txt and uploaded it in the public_html of my site


is that right?

any help?

hope this prevent my site from crawling?
i wan my site to grow bigger
You're not blocking anything important so it's fine, but don't expect your site to suddenly just rank 1 in Google, it's not as easy as that Smile