1)   Robots.txt 2.0
Robots.txt is a visual editor for Robot Exclusion Files and a log analyzer software. It allows a user to quickly and easily create the Robots.txt files, required to instruct search engine spiders, which parts of a Web site are not to be indexed
site optimization, log file analysis, spider visits, robots.txt editor, spider instructions, search engine indexation, search engine optimization,