Detectable as AI-Generated

TL;DR

• The marking obligation is live. Article 50(2) and California’s AI Transparency Act both attached on August 2, 2026 — but California excludes text, so a product can be clean in Sacramento and exposed in Brussels.
• The technique has one secret. A hidden key splits the vocabulary and nudges the model toward one half.
• Detectability is paid for in diction. The mark exists only because the model chose one word over its synonym. In a filing, that is an edit.
• Typos break it. One scheme fell from 0.993 to 0.222 detection under character-level typos; another attack stripped marks in 6.5 seconds.
• The key can be reconstructed. One 2024 paper recovers the hidden vocabulary split outright; another imitates an unknown scheme for about $908.
• Document the feasibility record. Both statutes hinge on what is technically feasible — a standard that moves with the attack literature.

Fifteen days ago a marking obligation attached to most general-purpose LLMs available to European users. Article 50(2) of the EU AI Act requires providers of systems generating synthetic audio, image, video or text to ensure those outputs are “marked in a machine-readable format and detectable as artificially generated or manipulated.” California’s AI Transparency Act took effect the same day.

Both point toward watermarking. Between 2024 and 2025, five papers — four peer-reviewed, one a preprint — took that answer apart.

What the Two Laws Actually Require

#

Article 50 is the AI Act’s transparency provision, and paragraph 2 governs synthetic content. It does not reach everything: the obligation lifts where a system “perform[s] an assistive function for standard editing” or does “not substantially alter the input data provided by the deployer or the semantics thereof,” and where a system is authorised by law for detecting or prosecuting criminal offences. A grammar tool sits outside it. A chatbot drafting prose does not. The qualifier is where the obligation actually gets set:

Providers shall ensure their technical solutions are effective, interoperable, robust and reliable as far as this is technically feasible, taking into account the specificities and limitations of various types of content, the costs of implementation and the generally acknowledged state of the art, as may be reflected in relevant technical standards.

Four adjectives — effective, interoperable, robust, reliable — each bounded by feasibility, cost, and the acknowledged state of the art. The standard is not “the mark survives attack” but “the mark is as robust as the field currently knows how to make it,” which sets the compliance floor by reference to the research literature and moves it whenever that literature moves.

The Code of Practice on marking and labelling AI-generated content was published on 10 June 2026, following transparency guidelines on Article 50. Adherence is voluntary: the Code offers practical steps rather than defining compliance, and providers may demonstrate it by other means. Under the Digital Omnibus, Regulation (EU) 2026/1744, systems already on the market before August 2 have until December 2, 2026 to meet the marking requirement.

California arrived at the same date differently. SB 942 was amended by AB 853 in October 2025 and applies to providers whose publicly accessible generative systems draw more than one million monthly users in California. They must offer users the option of a manifest disclosure — a visible label — and must embed a latent disclosure: provenance data carrying system identity, version, and creation date. They must also maintain a free detection tool. Penalties run $5,000 per violation, each day counted separately.

The two regimes do not overlap. California’s duties reach image, video, and audio; text is expressly excluded. Article 50(2) reaches text. A provider shipping both a chat product and an image generator can satisfy Sacramento on the images while carrying unresolved exposure in Brussels on the prose — and the prose is where the technique is weakest.

The Technique the Mandate Points At

#

Neither statute names watermarking; both are technology-neutral. For images and audio, provenance manifests like C2PA Content Credentials satisfy the machine-readable requirement. For unformatted text there is no other candidate: metadata is stripped on copy-paste, and a manifest travels with a file rather than with sentences pasted into an email. If the mark must survive leaving the file, it has to live in the word choices.

The dominant approach is a green-list scheme, introduced in Kirchenbauer et al.’s 2023 paper. At each generation step, a secret key seeds a pseudorandom split of the model’s vocabulary into a “green” half and a “red” half. During Inference, the sampler adds a small bonus to the Logits of green tokens, making them modestly more likely to be chosen. The bonus is small enough that fluency survives.

The bonus does not degrade text so much as redirect it: at each step the model reaches for the green candidate over the red one. The result reads perfectly well — it is simply not the sentence the unmarked model would have written. In general prose that distinction is invisible. In a filing it is not. “Tribunal” and “panel” are not interchangeable, the gap between “shall” and “will” has been litigated for a century, and an associate who quietly changed “reopen” to “revisit” in a brief would be asked why. A mark strong enough to detect is a mark that has already edited the document.

Detection is a counting exercise. Run the same key over a candidate passage, count how many of its tokens landed green, and compare against the 25% or 50% baseline expected from unmarked text. Enough excess green, and the passage is flagged. Zhao et al.’s Unigram variant uses one fixed split instead of re-seeding at every step, trading some detectability for resistance to editing.

The test’s power scales with length. The green share of a short passage is noisy: a thirty- Token excerpt can sit well above the baseline by chance, and a marked one can fall below it. Signal accumulates only across hundreds of tokens, which is why published results run on passages of roughly two hundred. A full brief is testable. A paragraph is not, and a quoted clause certainly is not — the detector returns a number either way, but at that length it is closer to a coin flip than a finding. Every attack figure below is measured on text long enough for the test to work.

A second family avoids nudging the distribution at all. Distortion-free schemes — Gumbel-style sampling, and Google DeepMind’s SynthID-Text tournament sampling — use the key to steer which sample gets drawn rather than reweighting what is likely. Different mechanism, same dependency: the detector needs a secret.

A green-list Watermark is not a signature that can be verified but not forged. It is a shared secret, embedded in text the adversary can generate on demand, in unlimited quantity, for the price of Inference.

What Robust Turned Out to Mean

#

WaterPark (Liang et al., Findings of EMNLP 2025) tests how the mark holds up under ordinary abuse: ten watermarking methods against twelve removal attacks, spanning linguistic variation, lexical editing, text mixing, and paraphrasing.

One leading token-bias scheme drops from a 0.993 True-Positive Rate on clean text to 0.222 under character-level typos. Copy-paste mixing — interleaving marked and human text — collapses most of the field, with the weakest scheme reaching exactly zero. The two designs that hold up do so at a measurable cost in MAUVE and semantic-similarity scores: their output stays detectable because it is worse. Those metrics measure distributional similarity, not whether a drafting attorney would have accepted the substitution.

The Smoothing Attack (Chang, Hassani & Shokri, Findings of EMNLP 2025) exploits that trade-off cheaply. Watermark signal concentrates where the model is uncertain — high-entropy positions where the green-list bonus actually changes the outcome. Estimate the model’s confidence at each position, leave the confident tokens alone, and resample the uncertain ones through a small reference model.

The attacker needs API access returning top-K probabilities, a few hundred calibration queries, and a 125M-parameter reference model. On OPT-1.3B at a 1% false-positive rate, KGW, SynthID, EWD and SWEET fall from 100% detection to 0%, Unigram to 5%, DIP to 6%. Only UPV and Unbiased hold anything, at 20% and 27%. Runtime is about 6.5 seconds per passage.

In the Unigram configuration, attacked text scored 9.44 Perplexity against the watermarked original’s 14.99 — and against 11.39 for text never marked at all. Removing the Watermark left the passage more fluent than leaving it in. Paraphrasing, translation, and recursive rewriting all paid for evasion in quality. Smoothing gets paid.

Reverse-Engineering the Key

#

Evasion leaves the scheme intact. Recovery does not.

Query and test. SCTS (Wu & Chandrasekaran, ACL 2024) determines Token colours by asking the watermarked model directly. Prompt it at temperature zero to produce paired outputs, count the frequencies, and run a chi-squared test: tokens appearing more often than chance are green. No open-weight copy, no detector access, no knowledge of the hyperparameters — just the API and the context size. It recovers colours only for the tokens it tests, which is enough to make substitution surgical: detection AUROC falls below 0.5 at a 35% edit budget, where colour-blind paraphrasing baselines still sit near 0.88. Theorem 1 extends the result to text of arbitrary length.

Solve it as an optimization. Zhang et al. formalize green-list recovery as a mixed integer programming problem, and this is the paper that takes the list whole. Green tokens must be dense enough in watermarked samples to clear the detection threshold, sparse enough in natural text to stay below it, and bounded by the green fraction. The attacker needs a corpus of each.

Precision depends on what else the attacker knows. Given detector access plus the green fraction and the z-score threshold, an idealized formulation exceeds 99% on Llama and the practical version lands in the 80s to mid-90s, cutting surviving green tokens to between 13% and 36%. Strip all of that and the attack still removes 77–82%. Against three-key schemes, precision holds between 64% and 91%, where a frequency-counting baseline manages 40–72%.

Distil a copy. B4 (Huang, Pu & Wan, NAACL 2025) drops the last assumption: the attacker does not know which scheme is running. It frames removal as constrained optimization between a watermark distribution and a fidelity distribution, then approximates the watermark side by distilling a small proxy model from samples queried out of the victim. What comes back is not a key but a working imitation of the mark’s statistical signature — enough for removal. The proxy takes about 200,000 queried samples, which the paper prices at $908. Across twelve settings, B4 improves attack success by up to 68% over paraphrasing baselines.

Recovery is worse than removal because it runs both directions. An attacker who strips the mark publishes machine text as human; an attacker holding the reconstructed split can bias a document toward green so a detector reports human prose as machine-generated. No published attack demonstrates the second, and it should be read as inference — but a recovered partition is symmetric by construction, and a counting detector cannot distinguish text biased toward green by a model from text biased toward green by an editor. [Medium confidence] Spoofing will surface as a live dispute before removal does, because removal only benefits whoever generated the text, while spoofing benefits anyone with a reason to discredit a document.

The Moving Floor

#

Three of these five papers were published after most deployed schemes were designed. Each arguably relocates the acknowledged state of the art — not by making watermarks better, but by establishing what a competent adversary can now do. A provider’s feasibility analysis is perishable in a way most compliance documentation is not.

[Medium confidence] The Code of Practice will become the working reference for adequate marking despite being voluntary, because a provider following published Commission guidance stands in a better position than one improvising — but it confers no presumption of conformity, so adequacy gets contested case by case. [Medium confidence] Whatever standard emerges will be framed in process and documentation rather than measured attack resistance, because any resistance threshold written down would be falsified by the next conference cycle.

None of this makes watermarking pointless. Against casual misuse — a student, an employee, someone pasting output without thinking about detection — it works, and the statutes are partly aimed at that population. [Low confidence] The durable version of provenance will be cryptographic attestation at the point of generation, with watermarking demoted to a hint, because signature schemes have the asymmetry that shared secrets lack.

What to Do About It

#

• Treat marking as the weakest layer of a provenance stack, not the load-bearing one. Content credentials, access logs, and generation records are all harder to forge than a token-bias Watermark — the same lesson as documents as an attack surface and connectors as an attack surface.
• For covered providers, document the feasibility reasoning. What was implemented, what alternatives were considered, which published state of the art the assessment relied on — with dates. Neither statute creates a safe harbour, but a qualifier keyed to technical feasibility is impossible to argue without that record.
• Read the Code of Practice as guidance, not as the standard. Voluntary, and no presumption of conformity: following it is a position to argue from, not a compliance answer.
• Check modality coverage against both regimes separately. California reaches image, video, and audio; Article 50(2) reaches text. Products shipping both need two analyses.
• Assume the key has a shelf life. Rotation, multi-key deployment, and monitoring for anomalous query volume answer a threat model where the secret is recoverable at four figures.

Further Reading

#

• Article 50 of the EU AI Act — the operative transparency text
• Code of Practice on marking and labelling AI-generated content — final version, 10 June 2026
• Commission guidelines on transparency obligations
• California SB 942, the AI Transparency Act and AB 853, the 2025 amendments
• A Watermark for Large Language Models — Kirchenbauer et al., the green-list scheme
• Provable Robust Watermarking for AI-Generated Text — Zhao et al., the Unigram variant
• Scalable watermarking for identifying large language model outputs — Dathathri et al. on SynthID-Text tournament sampling
• Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense — Krishna et al., the DIPPER baseline newer attacks are measured against
• C2PA Content Credentials — the provenance-manifest alternative
• Advancing content provenance — OpenAI on the industry direction, May 2026

This post is part of the Provenance and Proof series on LegalRealist AI, examining how AI-generated content is marked, verified, and disputed. It is intended for informational and educational purposes only and does not constitute legal advice. Regulatory requirements vary by jurisdiction and change frequently; consult qualified counsel in the relevant jurisdiction before acting on anything described here.