AI search · Intermediate

Google-Extended

Google-Extended is a robots.txt token that controls whether content Google crawls can be used to train and ground its Gemini models. It doesn't affect Google Search rankings or inclusion in AI Overviews.

By Thryv StudioUpdated

Category
AI search
Level
Intermediate
Operator
Google
Type
A robots.txt rule, not a separate crawler
Doesn't affect
Search ranking or AI Overviews
Check it free
Robots.txt Checker

What is Google-Extended?

Google-Extended isn't a crawler with its own user agent string. It's a product token you can name in robots.txt to tell Google whether pages it has crawled with its normal crawlers may be used for Gemini models and related AI products. Google introduced it in 2023.

What it does and doesn't control

  • Controls: use of your content to train and ground Gemini models outside Search.
  • Doesn't control: crawling and indexing for Google Search, your rankings, or whether your pages appear in AI Overviews and AI Mode, which are part of Search.

How to use it

User-agent: Google-Extended
Disallow: /

Frequently asked questions

Will blocking Google-Extended remove me from AI Overviews?

No. AI Overviews are a Search feature, governed by Googlebot and the usual snippet controls.

Should I block it?

Only if you don't want your content used for Gemini outside Search. It has no effect on your search visibility.

Browse the glossary

Chat on WhatsApp