i was wondering what do they actually do when there on your forum?
Basically crawl through your pages and index you on search engines etc.
oo ok, googlebot has been on mine for hours now, that's why i was wondering, thanks
Proper, that means Googlebot Loves your forum
it must, it's been on alot today and yesterday
I don't want my forum indexed at all.
I've disabled the search engine friendly url's (thanks to Matt) and selected 'yes' for the headers to prevent caching.
Am I on the right track and is there anything else I can do?
Thanks!
The SEO URLs won't stop you getting indexed, they'll just make your links easier to type for your users, and
would help a search bot to index the URL, but disabling that won't stop the bots. To stop them, create a file called robots.txt in your forum root folder, and put this code in it:
User-Agent: *
Disallow: /
and it's goodbye search engines.
Ah, yes, of course. *slaps forehead*
So then...
Under what circumstances would I opt to send 'No Cache Headers'?
I went ahead and left it as 'no' for 'Send No Cache Headers'. Don't know exactly what this means yet, but as long as I'm not indexed by the SE's, I'm cool.
Thanks again, Matt!
SEO friendly URLs are really a joke. Google still indexes URLs that have query strings.
It's just for looks really. I use them because they're much more easier to navigate than query strings.