Search results
Indexing, links, and short excerpts; AI-generated summaries are outside this category.
CONTENT-SIGNAL GUIDE
Access and use are
different layers.
Allow and Disallow govern whether a crawler should fetch a path. Content-Signal expresses a publisher preference for downstream uses: search, AI input, and training.
Real-time model input
Retrieval, grounding, and other uses that provide content to a model at answer time.
Training
Training or fine-tuning models. The declaration depends on voluntary support.