# exclude help system from robots User-agent: * Disallow: /sugarcrm/ Disallow: /dotproject/ Disallow: /teltime/ Disallow: /teltools/ Disallow: /stats/ Disallow: /manual/ Disallow: /manual-2.0/ Disallow: /manual-1.3/ Disallow: /addon-modules/ Disallow: /doc/ Disallow: /images/ Disallow: /dotproject.old/ Disallow: /files/ Disallow: /foto1/ Disallow: /ftp/ Disallow: /images/ Disallow: /include/ Disallow: /lists/ Disallow: /lokal/ Disallow: /meeting/ Disallow: /mini/ Disallow: /news/ Disallow: /nuke/ Disallow: /offline/ Disallow: /pro/ Disallow: /PRODUCT24/ Disallow: /projectstyring/ Disallow: /reseller/ Disallow: /skype/ Disallow: /squint/ Disallow: /stats/ Disallow: /sugarcrm*/ Disallow: /telfile/ Disallow: /teltools/ Disallow: /test/ Disallow: /timeline/ Disallow: /unipro/ Disallow: /video/ # the next line is a spam bot trap, for grepping the logs. you should _really_ change this to something else... Disallow: /all_our_e-mail_addresses # same idea here... Disallow: /admin/ # but allow htdig to index our doc-tree #User-agent: htdig #Disallow: # disallow stress test user-agent: stress-agent Disallow: /