+- SEO MotionZ Forum (https://seomotionz.com)
+-- Forum: Search Engine Optimization (https://seomotionz.com/forumdisplay.php?fid=7)
+--- Forum: SEO General (https://seomotionz.com/forumdisplay.php?fid=22)
+--- Thread: What is the purpose of robots.txt file? (/showthread.php?tid=3333)
Pages:
1
2
Re: What is the purpose of robots.txt file? - RH-Calvin - 01-12-2018
Robots.txt is a text file that lists webpages which contain instructions for search engines robots. The file lists webpages that are allowed and disallowed from search engine crawling.
Re: What is the purpose of robots.txt file? - sadesh - 01-12-2018
robots.txt is the program that hide important links from the google that is if the website owner dont want to share some information in the website for the public so that they can use the robots.txt by telling that google dont want can crawl that particular link.