# Bad Bot User-agent: FindLinks Disallow: / # Bad Bot User-agent: Gaisbot Disallow: / # Bad Bot User-agent: NextGenSearchBot Disallow: / # Unknown - what do they do? User-agent: BecomeBot Disallow: / # ZyBorg/1.0 Dead Link Checker User-agent: ZyBorg Disallow: / # Bad Bot User-agent: grub-client Disallow: / # Bad Bot User-agent: NaverBot Disallow: / # Bad Bot is what's up.. User-agent: QuepasaCreep Disallow: / # Bad Bot User-agent: TurnitinBot Disallow: / # Bad Bot - generates all kinds of iiserrors User-agent: HarvestMan 1.5 Disallow: / # Bad Bot User-agent: HarvestMan Disallow: / # Bad Bot User-agent: HarvestMan 2.0 Disallow: / #What is this? What does it do? and why does it cause so many errors? User-agent: libwww-perl/5.65 Disallow: / #PurityBot.. anything but pure, runs rampped all over the place User-agent: Purebot/1.1 Disallow: / # All Robots should never hit these directories User-Agent: * Disallow: /process_login.aspx Disallow: /es/process_login.aspx Disallow: /en/process_login.aspx Disallow: /library/ Disallow: /private/ Disallow: /members/ Disallow: /errors/ Disallow: /bin/ Disallow: /Web References/ Disallow: /images/ Disallow: /aspnet_client/