Google-Extended
AI training crawlerThe robots.txt switch for Gemini training — not a crawler you'll see in logs.
What Google-Extended does
Google-Extended is a robots.txt control token, not a distinct crawler: Googlebot does the fetching, and Google-Extended governs whether those pages may train Gemini models and power grounding. You will rarely see it as a user agent — you declare it in robots.txt.
Fact sheet
- Operator
- robots.txt token
- Google-Extended
- Honors robots.txt
- Yes (documented)
- Executes JavaScript
- No — invisible to browser analytics
- Identity verifiable by IP
- No published ranges
- Official documentation
- Google docs ↗
How it identifies itself
(control token — requests arrive as Googlebot)
User-agent strings are free to fake — any scraper can claim to be Google-Extended. With no published IP ranges, claims by this agent can't be cryptographically verified — treat lookalikes with suspicion.
Allow or block via robots.txt
# Block Google-Extended User-agent: Google-Extended Disallow: / # Or allow it explicitly User-agent: Google-Extended Allow: /
Should you block Google-Extended?
Disallowing Google-Extended opts you out of Gemini training without touching your Google Search indexing — the two are separate. It does not stop Gemini's live URL fetches for users (see our Gemini unmasking write-up on the blog).
Is Google-Extended reading your site?
Google-Extendednever runs JavaScript, so GA4 can't see it. VisitorType's server-side detection can — with per-page counts.
Measure it free