User-agent: * Crawl-delay: 10 Disallow: /search.html Disallow: /search.html?* Disallow: /browse.html Disallow: /browse.html?* #Sitemap: http://franklin.library.upenn.edu/sitemapindex.xml