2016-12-06, 03:28 PM
google couldnt crawl into my site and i always get errors.
but today i placed this robots.txt in my root directory
i copied and placed them in a note pad and saved it as robots.txt and placed it the public html of the root directory.
this is safe and okay?
User-Agent: *
Disallow: captcha.php
Disallow: editpost.php
Disallow: misc.php
Disallow: modcp.php
Disallow: moderation.php
Disallow: newreply.php
Disallow: newthread.php
Disallow: online.php
Disallow: printthread.php
Disallow: private.php
Disallow: ratethread.php
Disallow: report.php
Disallow: reputation.php
Disallow: search.php
Disallow: sendthread.php
Disallow: task.php
Disallow: usercp.php
Disallow: usercp2.php
Disallow: calendar.php
Disallow: *action=emailuser*
Disallow: *action=nextnewest*
Disallow: *action=nextoldest*
Disallow: *year=*
Disallow: *action=weekview*
Disallow: *action=nextnewest*
Disallow: *action=nextoldest*
Disallow: *sort=*
Disallow: *order=*
Disallow: *mode=*
Disallow: *datecut=*
Allow: /
i saved it in a notepad as robots.txt and uploaded it in the public_html of my site
is that right?
any help?
hope this prevent my site from crawling?
i wan my site to grow bigger
but today i placed this robots.txt in my root directory
i copied and placed them in a note pad and saved it as robots.txt and placed it the public html of the root directory.
this is safe and okay?
User-Agent: *
Disallow: captcha.php
Disallow: editpost.php
Disallow: misc.php
Disallow: modcp.php
Disallow: moderation.php
Disallow: newreply.php
Disallow: newthread.php
Disallow: online.php
Disallow: printthread.php
Disallow: private.php
Disallow: ratethread.php
Disallow: report.php
Disallow: reputation.php
Disallow: search.php
Disallow: sendthread.php
Disallow: task.php
Disallow: usercp.php
Disallow: usercp2.php
Disallow: calendar.php
Disallow: *action=emailuser*
Disallow: *action=nextnewest*
Disallow: *action=nextoldest*
Disallow: *year=*
Disallow: *action=weekview*
Disallow: *action=nextnewest*
Disallow: *action=nextoldest*
Disallow: *sort=*
Disallow: *order=*
Disallow: *mode=*
Disallow: *datecut=*
Allow: /
i saved it in a notepad as robots.txt and uploaded it in the public_html of my site
is that right?
any help?
hope this prevent my site from crawling?
i wan my site to grow bigger