One signal may keep several AIs from training on your site
In brief
OpenAI, Google, Microsoft and Anthropic say they will adopt a common standard so a site can set its AI-training preference once for several firms. According to The Verge, the signal would live in robots.txt and was proposed at the IETF by an Anthropic standards lead. Anthropic already uses it; OpenAI says it will follow in the coming months, while Google and Microsoft plan support next year.
Source : The Verge - AI - « The biggest names in AI are backing a new way to opt out of scraping »
What it changes for an SME
If you have a company website, this is mainly a technical setting to limit how a few large AI firms use your public pages to train their systems, not a change to how you work day to day. The timetable is uneven: Anthropic already follows the signal, OpenAI points to the coming months, and Google and Microsoft speak of next year. Microsoft, according to the article, also plans a separate choice for training versus using pages as a verified source, while saying listings in Bing would continue. Other crawlers are not covered by this pledge.
What you can do
- Ask whoever manages your website whether robots.txt already blocks AI crawlers and what you want for training versus search.
- Decide whether public pages may be used for AI training; until all four firms honour the shared signal, separate opt-outs may still be needed.
- If you use a service such as Cloudflare, check how it already treats AI crawlers, because that block is a different mechanism.
Let's talk about the time you could save.
In 20 minutes, we find where AI could save you time or money. Free, no commitment.