SEO MotionZ Forum › Search Engine Optimization › SEO General v
What is the purpose of robots.txt file?

Thread Closed 
 
Thread Rating:
  • 0 Vote(s) - 0 Average
Share Thread:
What is the purpose of robots.txt file?
rachana
  Member
Joined: Dec 2017
Posts: 50
Likes Received: 0
#7
12-04-2017, 11:17 AM
Robots.txt : It is a text file created by webmasters that instruct web robots how to crawl pages in their website.
The main purpose of this file is to indicate whether the pages of their website should be crawled or not. The crawl instructions are “allowing” or “disallowing” which are specified by user agents.
Thread Closed 


Messages In This Thread

Forum Jump:


Users browsing this thread: 1 Guest(s)
[-]
Ad

[-]
Top Articles
100+ Social Bookmarking Sites List
55+ Image Submission Sites List

[-]
Recent Posts
Fitspresso Is it legit or Does it Really Work?
Fitspresso The average parts in FitSpresso work synergistically to assist with fatting ox...PatriciCoving — 07:51 PM
Zappify Pro Reviews-(Consumer Reports)- Is It Worth The Price? Read Za...
The size of the area where you plan to use the bug zapper is perhaps the single most cruc...getzappify2.0 — 07:05 PM
FitSpresso is a characteristic weight reduction supplement
FitSpresso is a characteristic weight reduction supplement made of cautiously organized n...JohorRoson — 06:54 PM
Google SERP Changes & Algorithm Updates - May 2024
American traffic is down for us as well. EU traffic is also down since the last 24 h...lipulili — 02:44 PM
Google SERP Changes & Algorithm Updates - May 2024
American traffic is down for us as well. EU traffic is also down since the last 24 hou...kaplan — 02:42 PM

[-]
Follow us on Facebook
SEO Motionz