User-agent: * Disallow: /include/ Disallow: /js/ Disallow: *.xls Disallow: *.doc Disallow: *.rtf Disallow: *.pdf