Is it possible to make my website invisible to searches and search engines? If it is possible, how can I do it?
Upload
robots.txt file to your forum root (where forumdisplay.php, showthread.php etc. are stored) with this content:
User-agent: *
Disallow: /
I would imagine removing all the spiders/bots would also work if guests are forced to sign in....
(2015-02-19, 11:52 PM)Destroy666 Wrote: [ -> ]Upload robots.txt file to your forum root (where forumdisplay.php, showthread.php etc. are stored) with this content:
User-agent: *
Disallow: /
This AND I would look up resources on blocking the Yandex search engine. Yandex has a bad reputation of not respecting robots.txt rules. If you don't want your site on a Russian search engine, you'll have to do that extra leg work too
.
For yandex (or other bots), you can use in .htaccess:
RewriteEngine on
Options +FollowSymlinks
RewriteBase /
RewriteCond %{HTTP_USER_AGENT} ^Yandex [NC]
RewriteRule ^(.*)$ http://not.an.url/
Oh, thanks guys, I was not ready to let people go into my website because it still need some works. Do tell me if any other search engines is not blocked out from the help of Crazycat & Destroy666. Once again, thanks guys for your help!