User-agent: "" User-agent: Mozilla/3.0 (compatible; WebCapture 2.0; Auto; Windows) User-agent: Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 4.0; .NET CLR 1.0.2914) User-agent: Mozilla/4.0 (compatible; Powermarks/3.5; Windows 95/98/2000/NT) User-agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) User-agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; Win64; AMD64) User-agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; FREE; .NET CLR 1.1.4322) User-agent: Opera/9.00 (Windows NT 4.0; U; en) User-agent: ^Java User-agent: ^Jakarta User-agent: User-Agent User-agent: compatible ; User-agent: Mozilla User-agent: libwww User-agent: lwp-trivial User-agent: curl User-agent: PHP/ User-agent: urllib User-agent: GT::WWW User-agent: Snoopy User-agent: MFC_Tear_Sample User-agent: HTTP::Lite User-agent: PHPCrawl User-agent: URI::Fetch User-agent: Zend_Http_Client User-agent: http client User-agent: PECL::HTTP User-agent: panscient.com User-agent: IBM EVV User-agent: Bork-edition User-agent: Fetch API Request User-agent: [A-Z][a-z]{3,} [a-z]{4,} [a-z]{4,} User-agent: WEP Search User-agent: Wells Search II User-agent: Missigua Locator User-agent: ISC Systems iRc Search 2.1 User-agent: Microsoft URL Control User-agent: Indy Library User-agent: Nutch User-agent: larbin User-agent: heritrix User-agent: ia_archiver Disallow: /comments/ Disallow: /comments/add/ Disallow: /contact/ User-agent: * Disallow: /application/ Disallow: /css/ Disallow: /flash/ Disallow: /images/ Disallow: /js/ Disallow: /modules/ Disallow: /video/ Disallow: /system/