Crawl rate control refers to the methods available for influencing how intensely and how often a search engine accesses a site – including Google Search Console settings, robots.txt directives, or intentional server responses (code 429, Too Many Requests) to signal a temporary limitation.
Crawl rate control becomes especially relevant for servers with limited resources that could be overloaded by aggressive crawling, or to protect server resources during periods of planned high traffic.
Why it matters
An uncontrolled crawl rate on a weak server can cause service interruptions for real users, making careful control of this rate a necessary technical practice for sites with limited server resources.