MyBB Community Forums

(2017-10-22, 03:50 PM)sarisisop Wrote: [ -> ]

(2017-10-22, 12:45 PM)ataulhaq Wrote: [ -> ]Can you please tell me which codes should i put in robot.txt file. I just searched my public_html folder and there I could not found it. So I want to create one but don't know how?
Your guidance will be highly appreciated.

You need to create a new file titled

robots.txt

Then you can add places you don't want the Bots to visit, for example I have this in mine.

User-agent: *
Disallow: /online.php
Disallow: /newthread.php
Disallow: /ratethread.php
Disallow: /printthread.php
Disallow: /calendar.php
Disallow: /member.php
Disallow: /usercp.php
Disallow: /usercp2.php
Disallow: /newreply.php
Disallow: /search.php
Disallow: /memberlist.php
Disallow: /private.php
Disallow: /member.php?action=register

You can also block Bots like this.

User-agent: Baiduspider
Disallow: /

User-agent: Yandex
Disallow: /

That's just an example, and you need to tailor it to your requirements. Some people don't bother using them and some bad bots will ignore them, but it can stop Google looking at pages you don't want them to see.

Thank you so much for sharing these very useful tips.

(2017-10-22, 12:42 PM)ziuma Wrote: [ -> ]i use robots.txt to stop search engine

and

<Files robots.txt>
         order allow,deny
         deny from all
 </Files>

to stop robots.txt acces

is it wrong??? i also need suggestion

This is about as wrong as you can get. robots.txt should be at your domain's root folder (eg. http://www.example.com/robots.txt) and reachable by every visitor. Well-behaved robots will fetch that file to see where they can index and how much time they should sleep before requests.

As other posts point out, you should rename your admin directory and update inc/config.php to reflect that change. You may want to enable the ACP secret PIN in that file, too.

After renaming the real admin folder, download and install the ACP Honeypot plugin so anyone attempting to hack your ACP won't get a 404 error, but won't be able to make any real changes. Note that after installing this, you will have to modify any other plugins you upload.

My all users profile indexed on google