User-Agent: ia_archiver User-Agent: fast User-Agent: Zyborg User-Agent: ArchitextSpider User-Agent: girafa User-Agent: sitecheck.internetseer.com User-Agent: NPBot User-Agent: http://www.almaden.ibm.com/cs/crawler User-Agent: Openbot User-Agent:nabot User-Agent:mogimogi User-agent: searchpreview User-agent: W3CRobot User-agent: Baiduspider Disallow: / User-Agent: NaverBot* Disallow: / User-Agent: NaverBot-1.0 Disallow: / User-Agent: Shim-Crawler Disallow: / User-Agent: nhnbot Disallow: / User-Agent: dloader Disallow: / User-Agent: dloader* Disallow: / User-Agent: Cowbot Disallow: / User-Agent: Cowbot* Disallow: / User-Agent*.btnaccess.net.* Disallow: / User-agent: Megalodon Disallow: /