User-agent: * Disallow: /search Crawl-delay: 5 # specifies a 5 second timeout