Hide your robot.txt from prying eyes

Robots,txt is one of those very simple yet crucial tools in a website. Its primarily use to give search engine crawlers instructions about what to index and what not to index, what part of a website should be left a line completely.

Just like there are lots of malicious hackers around, there are many prying robots online, some do not obey the robots.txt instructions at all, while some are even more sinister than just a prying eye. There is a simple code to protext your robots.txt file from such robots whilst making it available to valid genuine robots. Except from the code below, you can find the full code at UK Webmaster World forum.

RewriteEngine On RewriteCond %{http_user_agent} !(googlebot|Msnbot|Slurp) [NC] RewriteRule ^robots\.txt$ http://seo.i-connector.com/ [R,NE,L] AddHandler application/x-httpd-php .txt

---

This is some text prior to the author information. You can change this text from the admin section of WP-Gravatar  Test Bio for Temi Read more from this author

---