# 1) this filename (robots.txt) must stay lowercase # 2) this file must be in the servers root directory # ex: http://www.mydomain.com/pliggsubfolder/ -- you must move the robots.txt from # /pliggsubfolder/ to the root folder for http://www.mydomain.com/ # you must then add your subfolder to each 'Disallow' below # ex: Disallow: /cache/ becomes Disallow: /pliggsubfolder/cache/ User-agent: * Crawl-delay: 5 Disallow: /story/title/ Disallow: /search/ Disallow: /submit/ Disallow: /login/ Disallow: /admin/ Disallow: /webalizer/ Disallow: /modlogan/ Disallow: /files/ Disallow: /template_c/ Disallow: /user/ Disallow: /3rdparty/ Disallow: /cache/ Disallow: /files/ Disallow: /backup/ Disallow: /avatars/ Disallow: /modules/ Disallow: /js/ Disallow: /libs/ Disallow: /internal/ Disallow: /evb/ Disallow: /install/ Disallow: /generator/ Disallow: /login.php Disallow: /tag/ Disallow: /upcoming/ Disallow: /tagcloud/range/1 Disallow: /tagcloud/range/2 Disallow: /tagcloud/range/3 Disallow: /tagcloud/range/4 Disallow: /published/ Disallow: /scarecrow.php Disallow: /topusers.php Disallow: /story.php Disallow: /user.php Disallow: /topusers/ Sitemap: http://www.macevangelism.com/sitemapindex.xml