Crawl budget
Crawl budget is the number of URLs a search engine is willing to crawl on a site within a period. It matters mostly for large sites: broken links, redirect chains, and duplicate URLs waste it, leaving important pages crawled less often.
Related terms
robots.txt
A file telling crawlers which parts of a site they may or may not crawl.
Sitemap
An XML file listing a site's URLs to help search engines find them.
Redirect chain
A sequence of redirects where one URL points to another, then another.
Broken link
A link that points to a URL that can no longer be reached.