Current time: 02-09-2012, 05:06 AM Hello There, Guest! (LoginRegister)


Most effective way to block google from indexing certain forums.
01-08-2009, 11:30 PM (This post was last modified: 01-08-2009 11:31 PM by Netbuddy.)
Post: #1
Solved: 1 Year, 4 Months, 1 Week, 3 Days, 15 Hours, 1 Minute, 51 Seconds ago Most effective way to block google from indexing certain forums.
Just wondering if this is achievable at all. I have a friend that runs a VB board, several years after it launched, he still finds it difficult to block google from indexing certain forums on his site.

Im not looking for a fullproof method. I dont think thats possible, but simply the most effective methods people have come up with so far.

Any help is really appreciated. If I can say I've made every effort to make a forum private, and it gets indexed anyway, at least ive covered my own backside. Big Grin

I only want some forums private, on a selective basis, not the entire site.

Any help is really appreciated.
Find all posts by this user
01-09-2009, 04:34 AM
Post: #2
Solved: 1 Year, 4 Months, 1 Week, 3 Days, 15 Hours, 1 Minute, 51 Seconds ago RE: Most effective way to block google from indexing certain forums.
Admin_CP_Config_Spiders
Find all posts by this user
01-09-2009, 09:04 AM
Post: #3
Solved: 1 Year, 4 Months, 1 Week, 3 Days, 15 Hours, 1 Minute, 51 Seconds ago RE: Most effective way to block google from indexing certain forums.
The best probable way is to use a robots.txt to disallow access.

http://www.robotstxt.org/faq/prevent.html

Like it says, this will only block the well-behaved spiders that check the robots.txt file. I guess you can hard code it so that access is denied for specific spider user agents.

[Image: igG319dTu71gT.png]
Find all posts by this user


Forum Jump:


User(s) browsing this thread: 1 Guest(s)

Contact Us | MyBB | Return to Top | Return to Content | Lite (Archive) Mode | RSS Syndication