User-agent: * #Disallow: /cgi-bin/ # updated 2005-09-07 (disallow folders)