Skip to content

Robots.txt demystified

A lot has already been written about creating and using the robots.txtfile. However, some of the articles often tend to be too cryptic or “techie” in nature, leaving a lot of newbies and tech unsavvy people totally overwhelmed with all that information.

Came across an article by Vanessa Fox that seeks to demystify robots.txtin a clear, easy to understand manner, with lots of examples and links. This is a must read for everyone who wanted to know more about robots.txt and the Robots Exclusion Protocol.

Thanks, Vanessa.

shares
Read previous post:
Canada uses cardboard cops to manage speeding traffic

In a bid to reduce speeding and traffic accidents, police in Vancouver, Canada are now using life-size cardboard traffic cops pointing a...

Close