# robots.txt for http://code.activestate.com/ User-agent: * Sitemap: http://code.activestate.com/sitemap.xml User-agent: Sosospider Disallow: / # http://en.wikipedia.org/wiki/Robots_exclusion_standard#Crawl-delay_directive Crawl-delay: 10