01-07-2018, 05:25 PM
01-08-2018, 08:08 AM
Robots.txt is better. Robots.txt is a text file that lists webpages which contain instructions for search engines robots. The file lists webpages that are allowed and disallowed from search engine crawling.
01-09-2018, 06:09 AM
Robots.txt dictates site or directory-wide crawl behavior, whereas meta robots can dictate indexation behavior at the individual page level.