- robots.txt: tells crawlers what’s allowed to be crawled. Doesn’t guarantee index.
- sitemap.xml: lists canonical URLs and last-modified dates.
Do both. Keep canonical links stable and avoid thin, duplicate pages.
Sitemap.xml vs robots.txt (and how both help SEO)
Robots controls crawling. Sitemap lists your canonical URLs. Use both.