Hi,
I don't know how old the the spider/bot list is but in my opinion it's a little but outdated... Can this list be up to date in new MyBB versions?
These are the Spiders suggestions:
GoogleBot > OK
MSN Search > Name is out of date, 'Bing' will be better
Alexa > OK
Altavista > Never seen this Spider, they use Yahoo and don't have any own Spider (please remove)
Archive.org > OK
What You Seek > Never seen this bot. Does it crawl english speaking sites only??
Hot Bot > they use Yahoo and don't have any own Spider (please remove)
Ask Jeeves > they indexed my site but the spider was never on my board. Maybe the bot name is out of date??
Lycos > my opinion: Nobody use Lycos so it's safe to remove...
Yahoo! > OK (until they use Bing...)
What's your opinion?
I'd ask to add spiders to the list
For example:
twiceler - Bot from cuil.com
That is already added to 1.6
I made a list of the Top 5 search engine world wide + Alexa/Archive.org:
Name User Agent
Google Googlebot
Yahoo! Slurp
Bing msnbot
Ask Teoma
Baidu Baiduspider
Alexa ia_archiver
Archive.org and Alexa use the same crawler (ia_archiver)!
Altavista and Hot Bot use Yahoo! search engine.
I'm missing cuil on the list of the Top 5 - Cuil is the biggest search engine in www
Yeah, the biggest wannabe search engine.
(2009-08-18, 09:53 PM)querschlaeger Wrote: [ -> ]Yeah, the biggest wannabe search engine.
It is actually the biggest search engine in terms of content searched. It isn't one of the biggest in terms of popularity though.
(2009-08-02, 06:23 AM)Ryan Gordon Wrote: [ -> ]That is already added to 1.6
As Ryan says, Cuil has already been added. There's a few other changes too with the bot list.
I looked at:
http://dev.mybboard.net/projects/mybb/re...nserts.php
There a the following Bots:
$inserts[] = "INSERT INTO mybb_spiders (name,useragent) VALUES ('GoogleBot','google');";
$inserts[] = "INSERT INTO mybb_spiders (name,useragent) VALUES ('Lycos','lycos');";
$inserts[] = "INSERT INTO mybb_spiders (name,useragent) VALUES ('Ask Jeeves','ask jeeves');";
$inserts[] = "INSERT INTO mybb_spiders (name,useragent) VALUES ('Hot Bot','slurp@inktomi');";
$inserts[] = "INSERT INTO mybb_spiders (name,useragent) VALUES ('What You Seek','whatuseek');";
$inserts[] = "INSERT INTO mybb_spiders (name,useragent) VALUES ('Archive.org','is_archiver');";
$inserts[] = "INSERT INTO mybb_spiders (name,useragent) VALUES ('Altavista','scooter');";
$inserts[] = "INSERT INTO mybb_spiders (name,useragent) VALUES ('Alexa','ia_archiver');";
$inserts[] = "INSERT INTO mybb_spiders (name,useragent) VALUES ('Bing Search','msnbot');";
$inserts[] = "INSERT INTO mybb_spiders (name,useragent) VALUES ('Yahoo!','yahoo! slurp');";
$inserts[] = "INSERT INTO mybb_spiders (name,useragent) VALUES ('Cuil','twiceler');";
1. There is no Bot
is_archiver! Archive.org uses the Alexa service and also there Bot i
a_archiver. So Archive.org Bot = Alexa Bot
2. Hot Bot and Altavista are owned by Yahoo! Search also uses there Search results. They don't have an own Bot anymore.
3. The name "Ask Jeeves" ist out of date. The Company call his Search Engine just "Ask" or "Ask Search".
4. The correct identifier for Ask Bot is "Teoma".
(2009-08-19, 10:40 PM)querschlaeger Wrote: [ -> ]I looked at: http://dev.mybboard.net/projects/mybb/re...nserts.php
There a the following Bots:
$inserts[] = "INSERT INTO mybb_spiders (name,useragent) VALUES ('GoogleBot','google');";
$inserts[] = "INSERT INTO mybb_spiders (name,useragent) VALUES ('Lycos','lycos');";
$inserts[] = "INSERT INTO mybb_spiders (name,useragent) VALUES ('Ask Jeeves','ask jeeves');";
$inserts[] = "INSERT INTO mybb_spiders (name,useragent) VALUES ('Hot Bot','slurp@inktomi');";
$inserts[] = "INSERT INTO mybb_spiders (name,useragent) VALUES ('What You Seek','whatuseek');";
$inserts[] = "INSERT INTO mybb_spiders (name,useragent) VALUES ('Archive.org','is_archiver');";
$inserts[] = "INSERT INTO mybb_spiders (name,useragent) VALUES ('Altavista','scooter');";
$inserts[] = "INSERT INTO mybb_spiders (name,useragent) VALUES ('Alexa','ia_archiver');";
$inserts[] = "INSERT INTO mybb_spiders (name,useragent) VALUES ('Bing Search','msnbot');";
$inserts[] = "INSERT INTO mybb_spiders (name,useragent) VALUES ('Yahoo!','yahoo! slurp');";
$inserts[] = "INSERT INTO mybb_spiders (name,useragent) VALUES ('Cuil','twiceler');";
1. There is no Bot is_archiver! Archive.org uses the Alexa service and also there Bot ia_archiver. So Archive.org Bot = Alexa Bot
2. Hot Bot and Altavista are owned by Yahoo! Search also uses there Search results. They don't have an own Bot anymore.
3. The name "Ask Jeeves" ist out of date. The Company call his Search Engine just "Ask" or "Ask Search".
4. The correct identifier for Ask Bot is "Teoma".
Can you please file a bug report against
http://dev.mybboard.net/issues/259 with those details.
Thanks!
Ryan