# robots.txt generated by freeware tool at http://robots-txt.info/ User-agent: * Disallow: #Crawl delay restrictions Crawl-delay: 20 seconds #Sitemap Sitemap: urllist.txt #Directory restrictions Disallow: /images/ Disallow: /rss/ Disallow: /Templates/ Disallow: /_private/ Disallow: /aspnet_client/ #Specific restrictions User-agent: twiceler Disallow: /User-agent: gigabot Disallow: /User-agent: googlebot-image Disallow: /