Hi
I help moderate a site and been assigned the task of how to stop visits by a googlebot.. but there is a IP address so from the other threads I have read it is a adsense? It seems like every time we make a post at the forum the bot shows up and I am able to get the IP and have told her to block that IP.. but it returned with a different one!
we are both new to the mybb forum and I need instructions on how to block them.. There is some links at the forum that she does not want shared, is a password protected but she is still afraid of piracy of her products...
I have searched the community for information on what to do, but I really don't understand it!!
all the stuff about disallow and * I find really confusing.. where do I need to tell my adm were to add the disallow robot text.. is there an instruction Manuel?
Not sure it this makes sense!
Not sure it this makes sense
Sounds like you could make use of a robots.txt file
One we use at MyBB can be found here:
http://community.mybboard.net/robots.txt
It basically tells bots (or search engines) not to visit certain pages.
If you want bots to not read your forums at all, then have the robots.txt file contain this:
User-agent: *
Disallow: /
And place that in your forums directory.
If there are forums that are password protected or require registration to see it, then you need not worry about bots seeing those pages.
Quote:If there are forums that are password protected or require registration to see it, then you need not worry about bots seeing those pages.
thanks for the help
the forum requires registration but it's still there and only the area where the download section is reqires a password!
rh1n0 Wrote:If you want bots to not read your forums at all, then have the robots.txt file contain this:
User-agent: *
Disallow: /
And place that in your forums directory.
Actually, robots.txt should be located in the root directory of your site, not the forum.
Bots are in the "unregistered" usergroup and surf your forum just like a guest. They can see the same things as a guest would see on your forum.
You could always ban them with the .htaccess file located in the forum directory.
I find robot.txt files to be ok for the good bots.. sometimes then even questionable but .htaccess always works.
Something like this
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} ^Googlebot
RewriteRule ^.*$ index.html/ [F,L]
Or something similar.
Google it and you will find the answers you seek.
thanks for all the help but...
how do I get to the root directory?.. I have admin priviledges and have gone through everything but I can not find the root directory... Sorry but really new to this and directions if you could please..
somewhere I saw that you needed a FTP software to get to the root directory? and I do but when I try to log in, I am unable to it keeps saying wrong username and pw.. but I am able to log into CP admin easy enough!
Quote:You could always ban them with the .htaccess file located in the forum directory
Is this under board settings?
again sorry for being so new at this.. try to find the ans myself but find them confusing!!
You need a ftp client to do this. Your ftp password and username will be different to your boards. Your ftp client will use the password and username you used to obtain hosting.
If this is not your forum the speak to owner about this.
And the .htaccess is not available in ACP either. You need to use a ftp client to access this.
Same goes for the robots.txt file. Well ftp client is the easiest.
Make sure you have "show hidden files" or you might not even see the .htaccess file.