🔍 Read the full analysis: AI Training Opt-Out Guide: What It Covers And How To Set It on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
OpenAI, Google, Microsoft and Anthropic have announced support for a shared robots.txt signal asking them not to use a website’s content to train generative AI models. The signal is voluntary, does not block AI answers or undo prior training, and will take effect on different schedules; Meta has expressed support without committing to honor it.
OpenAI, Google, Microsoft and Anthropic said on October 30 that they will honor a shared robots.txt signal asking companies not to use a website’s material to train generative AI models. The move could replace the need for publishers to maintain separate training opt-outs for each participating company, but it is a voluntary request, not a technical block, and support will begin at different times.
The proposed rule uses the user-agent label __usage_ai-training followed by Disallow: / in a site’s robots.txt file. It is described in an individual Internet-Draft titled “Machine-Readable Training Preference Signal,” published October 30 by Kevin Kelley of Anthropic. The draft asks participating services to treat the rule as a preference against using the site’s content for foundational generative AI training.
The companies have not set one common start date. Anthropic says it has started implementing support, while noting that support will take time to reach its services. OpenAI says it aims to implement the signal “in the coming months.” Google says it aims to support it in 2027, with its crawler documentation specifying early 2027. Microsoft says it will honor the training preference from February 1, 2027, as it crawls websites.
Meta has voiced support for publishers expressing preferences, but its spokesperson did not say whether the company will honor this particular signal. The source material reports no commitment from Apple, Amazon, xAI or Perplexity. Existing company-specific controls remain relevant while providers roll out the common rule.
Infographic 01 · Who backs it, and when
One line in robots.txt, four companies, four timelines.
A shared signal against AI training. Only Anthropic says it has started; the others name dates or months. The standard itself is still a draft.
- 30 Oct 2026Draft published, four companies announce supportdoneIndividual Internet-Draft by Kevin Kelley (Anthropic)
- 30 Oct 2026Anthropic: “has already started implementing support”startedIts standards lead on the IETF list: “It’ll take us some time to add support across our services.”
- Coming monthsOpenAI aims to implement itplanExisting GPTBot and OAI-SearchBot rules keep applying
- 16 + 19 Nov 2026Working group meets at IETF 127, San FranciscoopenOn 21 Oct a chair said no consensus has been declared yet
- 1 Feb 2027Microsoft honors the new preferences as it crawlsplanTraining in robots.txt or per page; grounding per page
- 1 Mar 2027Bing stops reading NoArchive / NoCache as AI preferencesdeadlinePublish the new settings before this date
- 2027Google plans support; its docs say “early 2027”planA “no” maps to Google-Extended unless you set that token yourself
- 3 May 2027Draft version 00 expires unless updatedInternet-Drafts are valid for at most six months
Binding? No, voluntary
- OpenAI calls it a “voluntary effort”.
- The vocabulary draft does not “ensure that preferences are followed”.
- The draft “has no formal standing in the IETF standards process”.
- Private agreements may override it, the draft says.
Who else?
- Meta supports a common signal, but did not say whether it will honor it.
- Apple, Amazon, xAI, Perplexity: no statement in the sources reviewed for this article.
- The draft’s acknowledgments list contributors “for identification only”, not supporters.
Sources: IETF draft, 30 Oct 2026 · Anthropic on the IETF list · AIPREF minutes, 21 Oct 2026 · Google, 30 Oct 2026 · Google crawler docs · OpenAI, 30 Oct 2026 · Microsoft Bing, Oct 2026 · Meta (Andy Stone on X). Company plans are company statements.
Powered by Thorsten Meyer AI
A Simpler Request, Not a Block
For website owners, a shared signal could make it easier to communicate a preference across several major AI providers. Instead of relying only on separate crawler names and rules, a publisher can add one common instruction for companies that have agreed to recognize it. That may reduce the chance of overlooking a participating service when updating a site’s preferences.
Its limits matter just as much. Robots.txt communicates a preference; it does not enforce access restrictions. The new line does not itself prevent a crawler from fetching pages, remove material from models trained earlier, or govern AI-generated answers that draw on live web content. It also creates no payment or guarantee of traffic. Microsoft said neither the training nor grounding preference guarantees inclusion, traffic or payment.
The distinction gives publishers a practical reason to separate their choices: whether to allow training, whether to allow content to inform AI answers, and whether to allow ordinary search crawling. A site may want different settings for each. Treating the new training rule as a universal “opt out of AI” switch would overstate what the announced signal covers.
Products Worth Considering
Bose TV Speaker Compact Soundbar with Bluetooth and HDMI-ARC, Black
Jackery Explorer 1000 v2 Portable Power Station
Bose QuietComfort Headphones (2nd Gen), Wireless Noise-Cancelling Over-Ear Headphones, Black
As an affiliate, we earn on qualifying purchases.
Before the announcement, publishers commonly used provider-specific instructions. These include controls for OpenAI’s GPTBot, Google-Extended and Anthropic’s ClaudeBot. Microsoft also used page-level tags called NoArchive and NoCache for relevant preferences. The shared draft is intended to give participating companies a common machine-readable instruction, while existing controls continue to operate during the rollout.
The proposal is connected to the Internet Engineering Task Force’s AI Preferences working group, but it is not an adopted internet standard. The source article reports that the draft has no formal standing in the IETF standards process. At a working-group meeting on October 21, a chair said, “We haven’t declared consensus on anything yet.” Company announcements therefore describe support for a draft, not a completed standards decision.
Companies describe the scope differently in their own documentation. Google says that if a site uses the new training signal and has no Google-Extended rule, it plans to interpret the signal as if the publisher had disallowed Google-Extended. Microsoft is also introducing a separate page-level preference for “AI Grounding,” covering content used to inform answers in Bing, Copilot and Microsoft 365. Those details are specific to the companies’ announced plans, rather than guarantees supplied by the draft itself.
““has already started implementing support for this draft””
— Anthropic, in its announcement of the draft
Adoption and Scope Still Vary
Whether every major AI company will honor the rule remains unknown. Meta has expressed support for publishers’ ability to state preferences but has not made a commitment on this signal. The source material also identifies no public commitment from several other companies, including Apple, Amazon, xAI and Perplexity. Participation may change as implementation proceeds.
The timing and practical effect also remain unsettled. Each company has announced its own rollout schedule, and Anthropic says support across its services will take time. The draft is not an adopted standard, and a robots.txt preference cannot technically guarantee that a company will comply. The announcements do not establish how compliance will be independently measured or what recourse a publisher would have if it believed the rule had been ignored.
There is also a boundary between training and other uses. The shared signal concerns training, not every use of web content in AI products. Microsoft’s separate AI Grounding setting addresses some uses in AI answers, while Google says its existing Google-Extended control remains relevant. Publishers should not assume that one training preference controls live retrieval, search eligibility or all AI services.
Rollout Dates and Publisher Choices
Publishers can add the shared user-agent group if they want to express a preference against training, while retaining provider-specific rules until each relevant company has implemented the shared signal. They should check the documentation and dates issued by each service rather than assume the same rule takes effect everywhere at once.
Microsoft says its new training and AI Grounding preferences take effect on February 1, 2027, as it crawls. It says NoArchive and NoCache will stop counting as training or grounding preferences after March 1, 2027. Sites that currently rely on those tags will need to review Microsoft’s replacement controls before that later date.
The next developments to watch are whether Meta and other providers make explicit commitments, when each company completes implementation, and whether the IETF working group advances the draft toward consensus or a formal standards process. Until then, the rule is best understood as a shared request that participating companies say they plan to honor—not a universal or immediate technical opt-out.
Key Questions
What line can a website add to request that its content not be used for AI training?
The draft uses a robots.txt group with User-Agent: __usage_ai-training and Disallow: /. It expresses a preference for participating companies to follow; it does not technically block access.
Does the signal stop AI systems from using a website in answers?
Not by itself. The announced shared signal concerns generative AI training, not all live-web retrieval or AI answers. Microsoft announced a separate AI Grounding preference, and existing provider controls may still apply.
When will the companies honor the signal?
The dates differ. Anthropic says implementation has started, OpenAI aims for the coming months, Google aims for 2027, and Microsoft says February 1, 2027. These are company rollout plans, not a single shared launch date.
Has Meta agreed to honor the new rule?
Meta has expressed support for publishers’ ability to state preferences, but the spokesperson cited in the source material did not confirm that Meta will honor this specific signal.
Does this remove a website from AI models that were already trained?
No such effect is described in the announcements. The signal is a request about future use of website content for training; it does not remove material from models trained earlier.
Source: ThorstenMeyerAI.com
