X-Robots-Tag

SEO

The X-Robots-Tag is the HTTP-header equivalent of the robots meta tag, allowing indexing directives (noindex, nofollow, etc.) to be applied at the server level, in the HTTP response header, instead of placing them in the page’s HTML code.

This header is essential for non-HTML files that can’t contain a meta tag in <head>, such as PDFs, images, or other file types, still allowing control over their indexing.

Why it matters

X-Robots-Tag provides flexibility to apply indexing directives at scale, for example automatically excluding all PDF files in a given directory, without manually editing each individual file.

← All terms in SEO Full glossary