Thread Rating:
  • 0 Vote(s) - 0 Average
Share Thread:
What is the purpose of robots.txt file?
#11
Robots.txt is a text file that lists webpages which contain instructions for search engines robots. The file lists webpages that are allowed and disallowed from search engine crawling.
Cheapest VPS | $1 VPS Hosting
Cheap Dedicated Servers | Free Setup and IPMI | Unmetered Bandwidth
#12
robots.txt is the program that hide important links from the google that is if the website owner dont want to share some information in the website for the public so that they can use the robots.txt by telling that google dont want can crawl that particular link.


Forum Jump:


Users browsing this thread: 1 Guest(s)
[-]
Ad

[-]
Top Articles
100+ Social Bookmarking Sites List
55+ Image Submission Sites List

[-]
Recent Posts
Nexaslim: Skulptera din kropp med lätthet!
Nexaslim Marknaden för viktminskningsprodukter och kosttillskott är enorm och står för en...reviewnexaslim — 10:35 AM
Google SERP Changes & Algorithm Updates - May 2024
@anush Couldn't agree with you more. They are just stealing headlines and calling it an A...UKcart — 02:44 PM
Google SERP Changes & Algorithm Updates - May 2024
Listen people don't panic. In many cases traffic has become zero so consider yourselves l...anush — 02:43 PM
Lucky Affiliate - All in One Thread
Share your thoughts about this network. Your reviews, feedbacks, any suggestion for the o...Aff — 05:45 AM
Google gives a Manual Action Penalty after March 2024 update
It is for "User generated spam". Our content is well hand written and no f...ananittarhib — 03:16 AM

[-]
Follow us on Facebook