robots.txt details

c32bd2463b6ec2e3c2c87afd78209163

saved: | size: 132kb | checksum: c32bd2463b6ec2e3c2c87afd78209163

websites using this robots.txt

content

User-agent: 80legs
Disallow: /

User-agent: 008
Disallow: /

User-agent: *
Crawl-delay: 5

# videos page
disallow: /videos/play.php