01-02-2019, 05:01 AM
Yes It is necessary, A robots.txt file gives instructions to web robots about the pages the website owner doesn’t wish to be crawled. This file is placed in root folder of website.
Users browsing this thread: 1 Guest(s) |
|