2010-08-11, 03:53 AM
2010-08-11, 06:31 AM
When you post a link on Facebook, it crawls your site to obtain the site title, meta description, and an image to post as a kind of "preview."
Facebook also has a Web Search, integrated into their Search function.
So, it's a legitimate Spider, only with different functions from the traditional Search Engine Spiders we know.
You just need to post a link to your site on Facebook, and it should do a quick crawl of your site.
For other spiders, you just need to submit them to the site they come from.
Try this, it's a great tool: http://www.submitexpress.com/free-submission.html
Facebook's user agent is:
facebookexternalhit/1.0 ( http://www.facebook.com/externalhit_uatext.php)
Facebook also has a Web Search, integrated into their Search function.
So, it's a legitimate Spider, only with different functions from the traditional Search Engine Spiders we know.
You just need to post a link to your site on Facebook, and it should do a quick crawl of your site.
For other spiders, you just need to submit them to the site they come from.
Try this, it's a great tool: http://www.submitexpress.com/free-submission.html
Facebook's user agent is:
facebookexternalhit/1.0 ( http://www.facebook.com/externalhit_uatext.php)