Technical SEO · AuditJet Glossary
Crawl Budget
The number of pages Googlebot will crawl on a website within a given timeframe, determined by crawl rate limit and crawl demand.
Crawl budget matters for large sites where Googlebot may not crawl every page in each visit. Wasting crawl budget on low-value pages (parameterised URLs, thin content) can delay indexing of important pages. Core Web Vitals and page speed affect crawl budget — faster servers allow Googlebot to crawl more efficiently.
Related terms
A text file at a website's root that instructs search engine crawlers which pages or sections of a site should not be crawled.
A file that lists all the URLs on a website to help search engines discover and index pages more efficiently.
An HTML element that tells search engines which URL is the preferred version of a page when multiple URLs serve the same or similar content.
Monitor Crawl continuously
AuditJet tracks Core Web Vitals on a schedule with revenue impact alerts.
Start Free