# Author: Carole Duttlinger. # Created: 07/18/2005 # Purpose: The robots.txt file is a special file that tells search # engine and specialty robots, spiders, and crawlers what pages # and files they are allowed to index on your website (this is done # by disallowing access to certain files and directories). # PARTIAL access (All Spiders) User-agent: * Disallow: /_derived/ Disallow: /_private/ Disallow: /_vti_bin/ Disallow: /_vti_cnf/ Disallow: /_vti_inf.html Disallow: /_vti_log/ Disallow: /_vti_pvt/ Disallow: /_vti_txt/ Disallow: /asp/ Disallow: /CF/ Disallow: /CFIDE/ Disallow: /cgi-bin/ Disallow: /design/ Disallow: /ModelGlue/ Disallow: /scripts/ Disallow: /styles/ Disallow: /tracking/ Disallow: /video/ Disallow: /error-404.cfm Disallow: /error-401.html Disallow: /error-403.html Disallow: /error-404.html Disallow: /error-500.html