03-23-2018, 10:46 AM
Reasons you may want to have a robots.txt file:
You have content you want blocked from search engines
You are using paid links or advertisements that need special instructions for robots
You want to fine tune access to your site from reputable robots
You are developing a site that is live, but you do not want search engines to index it yet
They help you follow some Google guidelines in some certain situations
You need some or all of the above, but do not have full access to your webserver and how it is configured
You have content you want blocked from search engines
You are using paid links or advertisements that need special instructions for robots
You want to fine tune access to your site from reputable robots
You are developing a site that is live, but you do not want search engines to index it yet
They help you follow some Google guidelines in some certain situations
You need some or all of the above, but do not have full access to your webserver and how it is configured
![[-]](https://seomotionz.com/themes/cerulean/collapse.png "[-]")
