Thread Rating:
  • 0 Vote(s) - 0 Average
Share Thread:
Is it necessary to use Robots.txt file?
#8
It is not necessary to use Robots.txt file, But sometime we need to tell crawler which are the pages needs to be indexed and which is not, In that case It is necessary to use Robots.txt file to guide crawler.


Messages In This Thread

Forum Jump:


Users browsing this thread: 1 Guest(s)