# This robots.txt file is a simple example, created for # the "HTTP: The Definitive Guide" book. # Ask all user-agents to keep out of the CGI scripts directory, the # testfile.txt file, and the "inventory-check.cgi" file. User-agent: * Disallow: /cgi-bin/ Disallow: /testfile.txt Disallow: /inventory-check.cgi