MyBB Community Forums
Invisible to search - Printable Version

+- MyBB Community Forums (https://community.mybb.com)
+-- Forum: 1.8 Support (https://community.mybb.com/forum-175.html)
+--- Forum: General Support (https://community.mybb.com/forum-176.html)
+--- Thread: Invisible to search (/thread-167410.html)



Invisible to search - Richard_Prime - 2015-02-19

Is it possible to make my website invisible to searches and search engines? If it is possible, how can I do it?


RE: Invisible to search - Destroy666 - 2015-02-19

Upload robots.txt file to your forum root (where forumdisplay.php, showthread.php etc. are stored) with this content:
User-agent: *
Disallow: /



RE: Invisible to search - Ircher - 2015-02-20

I would imagine removing all the spiders/bots would also work if guests are forced to sign in....


RE: Invisible to search - Josh H. - 2015-02-20

(2015-02-19, 11:52 PM)Destroy666 Wrote: Upload robots.txt file to your forum root (where forumdisplay.php, showthread.php etc. are stored) with this content:
User-agent: *
Disallow: /

This AND I would look up resources on blocking the Yandex search engine. Yandex has a bad reputation of not respecting robots.txt rules. If you don't want your site on a Russian search engine, you'll have to do that extra leg work too Smile.


RE: Invisible to search - Crazycat - 2015-02-20

For yandex (or other bots), you can use in .htaccess:
RewriteEngine on
Options +FollowSymlinks
RewriteBase /

RewriteCond %{HTTP_USER_AGENT} ^Yandex [NC]
RewriteRule ^(.*)$ http://not.an.url/



RE: Invisible to search - Richard_Prime - 2015-02-20

Oh, thanks guys, I was not ready to let people go into my website because it still need some works. Do tell me if any other search engines is not blocked out from the help of Crazycat & Destroy666. Once again, thanks guys for your help!