Search Engines employ Robots that are virtual crawlers to spider the web, the IP addresses of the Search Engines and their robots/crawlers names they send out to visit your site are as follows:
Yahoo Slurp Yahoo
MSN Robot MSN
Googlebot Google
Inktomi Slurp Inktomi
Ask Jeeves/Teoma Ask Jeeves
Scooter Alta Vista
Wget Webwombat
ArchitextSpider Excite
Internet Explorer Crawler IE
Northern Light Gulliver Northernlight
Lycos_Spider_(T-Rex) Lycos
These are among many other web crawlers that will index the websites and give out the information on their Search Engine Results Page when searched by the users. Among all of the above crawlers Google crawler must be the smartest one. Because of its latest and ever changing techniques to crawl and index the websites around the world.
