# Allow CARPSpider User-agent: CARPSpider Disallow: # Disallow all robots from this directory structure. User-agent: * Disallow: /