Our approach to EU text provenance rules

OpenAI laid out how it will meet the EU AI Act's requirement that generated text be identifiable by machines. Its textGrain system adds an invisible statistical signal to a model's word choices, and a detector looks for that signal. Watermarks are coming to eligible ChatGPT and Codex output in the EU over the coming weeks, API customers worldwide can opt in, and detector access opens only to approved researchers and expert organizations.

What sets this post apart is the data on failure. At a target false positive rate of 1%, OpenAI says its detector caught watermarks in about 80% of 200-token passages and about 95% of 400-token passages for content like psychology, and did substantially worse on mathematics, where word choice is constrained. Replacing 10% of words with synonyms dropped detection from about 92% to 66%. Replacing 25% dropped it to 17%.

We want to give this one real credit. Labs rarely publish the conditions under which their safety features stop working. OpenAI did, and it drew the right conclusion: these limits are why it is not handing the detector to the public. A tool that loses about a third of its hits after light synonym swaps, misses most text after heavier edits, and is tuned to accept some false alarms is a research instrument, not a lie detector.

The numbers also tell us who this will and won't catch. Long, unedited, free-flowing prose from a supported model is detectable. Short answers, math, translations, paraphrased drafts and anything run through another company's tool mostly are not. So the people most likely to be caught are the least sophisticated users, while anyone determined to hide AI use needs only a determined rewrite. That is a fairness problem if institutions ever treat detection as enforcement.

The company is clear that a watermark doesn't identify the user, measure human contribution, establish ownership, or verify accuracy, and that no detection doesn't prove a human wrote something. We'd add one more: it doesn't tell anyone whether using AI was appropriate in the first place. That judgment still belongs to people.

We also note OpenAI says it saw no meaningful benchmark differences on Astra, its latest frontier model, with and without watermarking. If that holds up, it removes the strongest practical objection to turning this on. And the pledge to release textGrain as open source is the piece we most want to see delivered, because provenance only works when it is shared across companies.

Where this leads: watermarks become a quiet background signal that platforms and researchers use to study synthetic text in aggregate. That is valuable. The danger is mission creep, where a statistical hint gets used as proof against an individual.

For images and audio, OpenAI already uses Content Credentials, is C2PA conformant, embeds SynthID watermarks, and offers public verification at openai.com/verify and through its Content Provenance API. It describes its approach as layered because no single technique is enough, and says it will revisit text provenance as technology, standards and regulation evolve.