# Robots.txt Added 09/03 User-agent: Teoma Crawl-delay: 60 Disallow: User-agent: MSNBot Crawl-delay: 60 Disallow: User-agent: Slurp Crawl-delay: 30 Disallow: User-agent: BecomeBot Crawl-Delay: 300 Disallow: / User-agent: Exabot Crawl-delay: 300 Disallow: User-agent: * Disallow: