What is the limit of a robot.txt file?

[jonathan brown]

Hi,

Yes, the limitations are:

1. The google bot can only read-only first 500KB robot.txt file data.
2. The robot.txt file needs to be added on the top of the domain of your website so that its command gets easily sharable with required details.
3.No assurance that a crawler or robot will visit your site and use your robots.txt file. Actually, search engines use and obey robots.txt file, but malware robots will not.
4. Sometimes the robot.txt file elements get ignored by some search engines