2011-12-14, 03:43 PM
Opening Thoughts
I have one hard and fast rule when investigating who is visiting my forum. The rule is any IP address which is not from a legitimate ISP or search engine such as Google, Yahoo, or MSN (Microsoft Bing) should be banned immediately. I recommend you follow this rule too when deciding who the bad guys are.
Introduction
To know if an IP address visiting your forum is an innocent web surfer (not logged in) or a bot - spammer you must be able to investigate them efficiently. That is the task I will describe in this tutorial.
IP Address Investigation Sites
Create a new group of bookmarks in your browser and name it Bot Detection. If you’re using Windows create a new folder in Favorites and name it Bot Detection.
Add the following links to Bot Detection:
http://www.scroogle.org/cgi-bin/scraper.htm
Scroogle Scraper is an ad free front end for Google. This is the first place to check out visitors IP addresses. Paste in an IP address and hit Search.
http://en.utrace.de/
This is the primary web site you will use to determine if an IP address is a bot or spammer. The first thing it tells you is your IP address, geographic location, ISP and / or organization name.
Collecting and Investigating Visitors IP Addresses
To get the IP addresses you will research go to your forum and login.
Scroll to bottom of forum home page and in “Who’s Online” click “[Complete List]”
In the page that opens your administrator username will be at the top of the list and any “Guests” IP address will appear below that.
To see information about a Guest right click on “[lookup]” and open link in a new tab or window.
If the visitor is one of your forum members who’s visiting but not logged in their username will appear to the right of “Last Known IP” If it is not a forum member then only the IP address will appear.
Right click “(Information on This IP)” and open the link in a new tab or window. Examine the “Host Name” information. If it says “NA” this may be your first clue that this is a malicious visitor and you need to investigate further. If host name information says anything that indicates this IP address is from a hosting company then you also need to investigate further as this is definitely a bot. If you can’t ascertain anything from the information you may also want to investigate further.
Go to Utrace (linked in Bot Detection) enter suspicious IP address in search field and click Search. On map that appears note ISP and / or organization name. Click on ISP / organization name. Note any information that might indicate that this is a hosting company. If it’s not obvious that this is an ISP then go to Scroogle Scraper, enter ISP or organization name and search for information that might tell you who this is. You may also use Scroogle Scraper to see if other web sites have reports of malicious activities by this IP addresses. Tip: If sites you find are not in you native language use your browser's translate function.
Also at Utrace note the IP address range so you may ban entire range owned by this organization (recommended).
After you determine that this is not an ISP you may ban either the specific IP address that visited your forum or the IP address range of the host / organization that is hosting the bot.
Closing Thoughts
Identifying a hosting company that hosts bots is the most challenging part of this process, with practice however this task can be quickly mastered.
You now have the tools and knowledge you need to keep bad guys off your forum.
Record Keeping
To keep a permanent record of the IP addresses you have investigated and / or banned I recommend you create a spreadsheet similar to the one attached to this post.
Additional Resources
Here are some web site that may also prove useful to your investigations:
https://ipdb.at/ (use this if Utrace fails)
http://www.domaincrawler.com/ (works with IP addresses too) (use this if Utrace fails)
http://www.botslist.ca/ (identifies bots both good or bad) (use this if Utrace fails) (periodically slow or offline)
http://ip-lookup.net/ (use this if Utrace fails)
http://www.whoishostingthis.com/ (use this if Utrace fails)
http://legacy.zoneedit.com/lookup.html?ad=whois (not as accurate for European IP addresses as Utrace, sometimes better for US addresses, use if other options fail)
http://www.bizimbal.com/odb/search.html (only identifies if a particular IP address has attacked one of their monitoring stations)
I have one hard and fast rule when investigating who is visiting my forum. The rule is any IP address which is not from a legitimate ISP or search engine such as Google, Yahoo, or MSN (Microsoft Bing) should be banned immediately. I recommend you follow this rule too when deciding who the bad guys are.
Introduction
To know if an IP address visiting your forum is an innocent web surfer (not logged in) or a bot - spammer you must be able to investigate them efficiently. That is the task I will describe in this tutorial.
IP Address Investigation Sites
Create a new group of bookmarks in your browser and name it Bot Detection. If you’re using Windows create a new folder in Favorites and name it Bot Detection.
Add the following links to Bot Detection:
http://www.scroogle.org/cgi-bin/scraper.htm
Scroogle Scraper is an ad free front end for Google. This is the first place to check out visitors IP addresses. Paste in an IP address and hit Search.
http://en.utrace.de/
This is the primary web site you will use to determine if an IP address is a bot or spammer. The first thing it tells you is your IP address, geographic location, ISP and / or organization name.
Collecting and Investigating Visitors IP Addresses
To get the IP addresses you will research go to your forum and login.
Scroll to bottom of forum home page and in “Who’s Online” click “[Complete List]”
In the page that opens your administrator username will be at the top of the list and any “Guests” IP address will appear below that.
To see information about a Guest right click on “[lookup]” and open link in a new tab or window.
If the visitor is one of your forum members who’s visiting but not logged in their username will appear to the right of “Last Known IP” If it is not a forum member then only the IP address will appear.
Right click “(Information on This IP)” and open the link in a new tab or window. Examine the “Host Name” information. If it says “NA” this may be your first clue that this is a malicious visitor and you need to investigate further. If host name information says anything that indicates this IP address is from a hosting company then you also need to investigate further as this is definitely a bot. If you can’t ascertain anything from the information you may also want to investigate further.
Go to Utrace (linked in Bot Detection) enter suspicious IP address in search field and click Search. On map that appears note ISP and / or organization name. Click on ISP / organization name. Note any information that might indicate that this is a hosting company. If it’s not obvious that this is an ISP then go to Scroogle Scraper, enter ISP or organization name and search for information that might tell you who this is. You may also use Scroogle Scraper to see if other web sites have reports of malicious activities by this IP addresses. Tip: If sites you find are not in you native language use your browser's translate function.
Also at Utrace note the IP address range so you may ban entire range owned by this organization (recommended).
After you determine that this is not an ISP you may ban either the specific IP address that visited your forum or the IP address range of the host / organization that is hosting the bot.
Closing Thoughts
Identifying a hosting company that hosts bots is the most challenging part of this process, with practice however this task can be quickly mastered.
You now have the tools and knowledge you need to keep bad guys off your forum.
Record Keeping
To keep a permanent record of the IP addresses you have investigated and / or banned I recommend you create a spreadsheet similar to the one attached to this post.
Additional Resources
Here are some web site that may also prove useful to your investigations:
https://ipdb.at/ (use this if Utrace fails)
http://www.domaincrawler.com/ (works with IP addresses too) (use this if Utrace fails)
http://www.botslist.ca/ (identifies bots both good or bad) (use this if Utrace fails) (periodically slow or offline)
http://ip-lookup.net/ (use this if Utrace fails)
http://www.whoishostingthis.com/ (use this if Utrace fails)
http://legacy.zoneedit.com/lookup.html?ad=whois (not as accurate for European IP addresses as Utrace, sometimes better for US addresses, use if other options fail)
http://www.bizimbal.com/odb/search.html (only identifies if a particular IP address has attacked one of their monitoring stations)