A lot has already been written about creating and using the robots.txtfile. However, some of the articles often tend to be too cryptic or “techie” in nature, leaving a lot of newbies and tech unsavvy people totally overwhelmed with all that information. 

Came across an article by Vanessa Fox that seeks to demystify robots.txtin a clear, easy to understand manner, with lots of examples and links. This is a must read for everyone who wanted to know more about robots.txt and the Robots Exclusion Protocol.

Thanks, Vanessa.