TL;DR
Can one robots.txt line opt your website out of AI training?
On Friday, OpenAI, Google, Microsoft and Anthropic backed one robots.txt line that tells AI companies not to train on your website. You'll be able to opt out with one rule instead of blocking each AI bot by name. Nothing to change yet, because no company honors it today.
What did OpenAI, Google and Microsoft announce?
On Friday, October 30, 2026, OpenAI, Google and Microsoft Bing all said they will respect a shared "policy token" in robots.txt. Robots.txt is the small text file at the root of your site that tells bots what they may visit. The token is a special name you write in that file, and it means "don't use my content to train AI" for every company that signs on.
The idea comes from an IETF draft written by K. Kelley, an Anthropic engineer. The IETF is the group that writes many of the internet's technical rules. The Verge reports that Anthropic has adopted the token too, and that Meta voiced support through spokesperson Andy Stone without committing to it.
What does the robots.txt line look like?
The draft's example blocks training on the whole site but leaves a press-kit folder open:
`` User-Agent: __usage_ai-training Disallow: / Allow: /press-kit/ ``
Two details trip people up. The token starts with two underscores, and some write-ups drop them, so copy it from the draft itself. Your User-Agent: * rules (the catch-all for every bot) don't apply to the token, so it needs its own block. Any rules you already have for named crawlers like GPTBot or ClaudeBot keep working.
When does the AI training opt-out start?
Not yet. Each company gave its own timeline:
| Company | When it starts honoring the line |
|---|---|
| OpenAI | "In the coming months" |
| Targeting 2027 | |
| Microsoft Bing | February 1, 2027 |
| Anthropic | Adopted, per The Verge (no start date given) |
The line costs nothing to add. It's still an individual IETF draft, not a finished standard, and the wording could change before then.
Does blocking AI training remove you from AI search answers?
No. Training and AI answers are separate switches, and small sites can now set them apart.
Training means a company uses your pages to build its AI models. Grounding means an AI looks up your page while answering someone and cites it with a clickable link, which can send you visitors.
- Google: the new line counts as a Google-Extended "disallow" unless you set Google-Extended separately. Google-Extended is the existing robots.txt name Google uses for AI use of your content. Google says explicitly allowing Google-Extended keeps your pages eligible for grounding in Gemini Apps.
- Bing: Microsoft is adding a separate page-level "AI Grounding" preference, explained in its webmaster help page. Bing says the training opt-out does not affect normal Bing search results.
So a blogger can say "don't train on my writing, but do quote me in answers."
The jargon, in one line each
| Term | What it means |
|---|---|
| robots.txt | A text file on your site that tells bots which pages they may visit |
| Crawler | A bot that reads web pages, for search or for AI |
| Policy token | A shared name in robots.txt that sets one rule for many companies |
| Grounding | When an AI answer looks up your page and links to it |
| Google-Extended | Google's robots.txt name for AI use of your content |
X reactions
SEO people spotted the joint launch fast.
Our take
This is a real win for small sites. Keeping up with a growing list of AI crawler names was a chore nobody running a shop or a blog signed up for, and one line that the biggest labs agree on is much easier to get right.
The split between training and grounding matters even more. Many creators want the visitors that AI answers can send without handing over their work to build the next model, and now there's a clear way to say so. Remember that robots.txt is a polite request. Companies that didn't sign on can ignore it, so treat it as a strong signal, not a lock.
— Artificially Nerdy
What to do now
- Decide your stance before the start dates: fine with training, no training but still cited in AI answers, or out of AI entirely.
- Look at your current robots.txt (yoursite.com/robots.txt) so you know which AI bots you already block by name.
- Keep your existing crawler rules. They still work, and the new line won't be honored for months.
- Want AI answers to cite you? Leave Google-Extended allowed and watch for Bing's AI Grounding setting.
- Copy the token from the draft when you add it, two underscores included.
FAQ
Do I need a developer to add this?
Usually not. On most site builders you can edit robots.txt in the SEO settings, and the rule is three short lines.
Will adding the line hurt my Google or Bing rankings?
Bing says the training opt-out doesn't affect normal Bing search. Google's post ties it to Google-Extended, which is separate from regular Google Search crawling.
Does this remove my content from models that were already trained?
None of the announcements say so. The line is about future training use.
Is Meta on board?
Not yet. The Verge reports Meta voiced support but didn't commit to honoring it.
Sources
- OpenAI: Helping websites express AI training preferences — OpenAI's support and "coming months" timing.
- Google Search Central blog: AI training control — Google-Extended handling, grounding in Gemini Apps, 2027 target.
- Bing Webmaster blog: Your content, your choice — Feb 1, 2027 start, no effect on Bing search.
- Bing Webmaster help: New AI preferences — the page-level AI Grounding preference.
- IETF draft: draft-kelley-aipref-training-preference-signal — the token and the example rule.
- The Verge: OpenAI, Google and Microsoft back an AI scraping opt-out — Anthropic's adoption, Meta's response.
- Glenn Gabe on X — October 30, 2026, 1:32pm ET.
- Lily Ray on X — October 30, 2026, 2:16pm ET.
