#Laat ALLE robots ALLES indexeren
User-agent: *
Disallow: 
Crawl-delay: 30
Request-rate: 1/30