On 30 October 2026 Google, Microsoft, OpenAI and Anthropic announced that they will support a shared signal in robots.txt that lets website owners opt their content out of AI model training. The signal comes from a draft at the IETF, the body that develops internet standards. It matters to you if your website contains text, images or expertise that you would rather not find in AI models without being asked. It is not in effect yet.
What happened?
A draft by Kevin Kelley of Anthropic describes a so-called policy token named __usage_ai-training. It is used in robots.txt the same way as a crawler name. The draft itself gives the example of a group that refuses training for the entire website and exempts only the /press-kit/ folder. It introduces no new syntax and uses the familiar groups instead. It is a draft by an individual author that the IETF has not adopted, so it has no official standard status yet.
Microsoft set out the details on its Bing blog. According to that post, Microsoft will honour the new directives from 1 February 2027. They replace the previous NoArchive and NoCache directives. There is also a setting that applies to individual pages only: with it you decide whether content may be used to answer AI queries (the post calls this grounding). Microsoft points out that opting out reduces visibility in AI applications but does not affect eligibility for the regular Bing results.
Google has also published a post on the topic on the Search Central blog. We were unable to retrieve its full text. According to consistent reports in trade media, Google intends to support the signal in 2027 and to treat it as a block for Google-Extended if a website does not address Google-Extended separately. Google-Extended controls whether content is used for Gemini. Regular Google Search is not affected. For OpenAI the reports speak of the coming months, for Anthropic of changes already under way; neither comes with a fixed date.
What does this mean for your business?
For most small businesses nothing changes today. Your site will still be found and search works as before. Three points matter:
- It is a statement, not a technical fence. The token expresses a preference about how content is used. It does not stop a crawler from fetching the page. You cannot force every provider to respect it.
- No entry means no preference. If you add nothing, then under the draft you have not declared anything.
- Blocking comes at a price. If you exclude content from AI training or AI answers, you will be named there less often. For businesses that want to appear in AI answers, that can be a disadvantage. This applies in particular to Microsoft's grounding setting, which explicitly concerns visibility in AI experiences.
Whether an opt-out makes sense for you depends on what your website shows. A business that wins customers by being found usually wants to be named. A publisher, a photographer or a consultancy with a lot of original expert writing may see it differently.
What should you do now?
- Do not rush. The signal only takes effect at Google and Microsoft in 2027, and for OpenAI the timing is open.
- Look at your robots.txt. Many site builders and security plugins already list AI crawlers there, often without anyone noticing. Do you know what yours says?
- Decide what you want: to be named in AI answers, not to be used for training, or to handle the two separately.
- Keep existing crawler-specific rules. If you already block certain crawlers, do not swap those rules for the new token as long as the providers do not support it.
- If you use NoArchive or NoCache for Bing: Microsoft gives transition periods until 1 March 2027. The details are on the Bing blog.
How AI systems can be influenced in general is explained in our article on Generative Engine Optimization (GEO).
What is still open?
The draft may still change before it is adopted. Whether and when other providers such as Meta will follow is not confirmed. It is also open how strictly the providers will comply in practice, and whether contracts between a website and an AI provider override the signal. This article is general information and not legal advice; for copyright questions, consult a specialised lawyer. As of 1 November 2026.
How can KSTAQ help?
We check which crawlers your robots.txt allows and set it up the way you want. This is part of SEO and GEO. If you are planning a new website, we put the right rules in place from the start, see web design. Send us a short note about what you want to achieve.
Sources
- Supporting a shared AI training control for websites, Google Search Central Blog, October 2026
- Your content, your choice: clearer preferences for training and grounding, Bing Blog, 30 October 2026
- Machine-Readable Training Preference Signal (Internet-Draft), Kevin Kelley, IETF, 30 October 2026
- Google, Microsoft, OpenAI and Anthropic back one AI training opt-out, The Next Web, October 2026