Thread Rating:
  • 0 Vote(s) - 0 Average
Share Thread:
Why We Use Robot.txt File In Seo..?
#1
Hello friends,

Why We Use Robot.txt File In Seo..?
#2
Why the robots.txt file is important. First, let's take a look at why the robots.txt file matters in the first place. The robots.txt file, also known as the robots exclusion protocol or standard, is a text file that tells web robots (most often search engines) which pages on your site to crawl.
#3
Robots.txt is a text file which use webmaster creat to instruct web robots how to crawl pages on their website. It is the part of the robots exclusion protocol (REP) that regulate how robots crawl the web access and index page content and serve that content to end users.
" Robots.txt is a text file you put on your site to tell search robots which pages you would like them not to visit."
#4
Hi,
 
Robot.txt allows the google crawler, which page to crawl of your site and which not to, after indexing.

For example, if you specify in your Robots.txt file that you dont want the search engines to be able to access your thank you page, that page wont be able to show up in the search results and web users wont be able to find it.

Search engines send out tiny programs called spidersor robotsto search your site and bring information back to the search engines so that the pages of your site can be indexed in the search results and found by web users. Your Robots.txt file instructs these programs not to search pages on your site which you designate using a disallowcommand. 

For example, the following Robots.txt command:
User-agent: *
Disallow: /images 
would block all search engine robots from visiting the following page on your website


Forum Jump:


Users browsing this thread: 1 Guest(s)
[-]
Ad

[-]
Top Articles
100+ Social Bookmarking Sites List
55+ Image Submission Sites List

[-]
Recent Posts
Growth Matrix Reviews - Warning! Do NOT Buy Until Knowing This!
Growth Matrix Reviews:>> DISCRIPTION =>> Growth Matrix Reviews is an enhance...RoseBenif — 07:48 AM
Empowering the Skills on Kafka Training with Gologica’s IT Expert
Gologica Offers Kafka Online training, it will help you to get  know  more about Apache K...Naresh1 — 07:35 AM
Nike presenterar den Barcelona-inspirerade Phantom 'Barna'-kollektione...
Nike firar sin koppling till den pulserande staden Barcelona med Phantom 'Barna'-kollekti...fotbollsskorfor — 07:11 AM
PUMA představuje druhé vydání kolekce NJR Creativity Collection, inspi...
PUMA se vrací s druhým vydáním NJR Creativity Collection, pulzující řadou inspirovanou el...kopackycz — 07:08 AM
https://www.facebook.com/slimxmederfahrungen/
SlimXMed Erfahrungen SlimXMed Erfahrungen Weight loss is everyone’s hassle those da...PerezFanders — 07:05 AM

[-]
Follow us on Facebook