robots.txt controls crawling instructions

A robots file uses user-agent groups and allow or disallow rules. It can reference sitemaps, but it does not explain a product, document an API, or provide typed capability schemas.

llms.txt provides orientation

A concise llms file can name a service, summarize it, and link to authoritative documentation. Support varies, so publishing one does not guarantee crawling, citation, or answer visibility.

Keep both consistent

Do not point agents toward documentation that crawler rules block unintentionally. Keep canonical URLs, product names, and machine interface links aligned with visible content and structured data.

Related reading