What's the significance of Robots.txt file in a website?

[websiteee]

A robots.txt file prevents the search engine to follow your website page which is disallowed by using a command.