AI Watermark Evidence Fails Forensic Readiness: An Empirical Evaluation
A new study evaluates three LLM watermarking methods (KGW, Unigram, SynthID-Text) for forensic reliability, finding none meet courtroom evidence standards. In 846 paraphrase runs, KGW and Unigram watermarks were 100% removed, SynthID 98.3%. False-negative rates were 70-83%, and SynthID misclassified 5.4% of human-written controls as AI-generated. The authors propose a Forensic Readiness Score (FRS) framework but conclude the tested configurations fail to provide court-admissible evidence.
-->
[Submitted on 17 Jul 2026]
Title:AI Watermark Evidence Fails Forensic Readiness: An Empirical Evaluation
View a PDF of the paper titled AI Watermark Evidence Fails Forensic Readiness: An Empirical Evaluation, by Saifur Rahman Tamim and Amir Labib Khan
View PDF HTML (experimental)
Abstract:Governments are increasingly mandating that LLM-generated content carry watermarks. The EU AI Act calls for markings that are "sufficiently reliable and robust." California's SB 942 requires disclosure that is "permanent or extraordinarily difficult to remove." Both mandates rest on an untested assumption: that watermark detection yields evidence reliable enough for courts. This paper tests that assumption directly.
We evaluate three representative LLM watermarking methods -- KGW, Unigram, and the MarkLLM implementation of SynthID-Text -- against the Daubert admissibility criteria and the NIST SP 800-86 digital forensic process. To structure this evaluation, we propose a Forensic Readiness Score (FRS) framework with 12 criteria, three mandatory gates, and a 60-point scoring system. We focus on meaning-preserving paraphrase as the attack vector, since it is both legally realistic and difficult to dismiss as evidence tampering.
The results raise serious evidentiary concerns. Out of 846 valid paraphrase runs across 15 diverse prompts per method, every single initially-detected KGW and Unigram text lost its watermark after paraphrasing -- 100% conditional removal. SynthID fared only slightly better at 98.3%. Even before any attack, false-negative rates were already high: 70% for KGW, 83% for Unigram, 80% for SynthID. The SynthID configuration also flagged 5.4% of paraphrased human-written controls as AI-generated and showed an 18.6% paradox rate, with 80% of its own pristine watermarked output landing in the uncertainty deadband. None of the three methods satisfy more than two of five Daubert factors. We also find that the FRS point-based scoring system, despite working as designed, cannot fully capture forensic uselessness -- a limitation worth noting for future framework design.
These configurations, as tested, do not meet the evidentiary bar that courts require.
Comments: 9 pages, 4 figures. A version of this paper was submitted to the AAAI/ACM Conference on AI, Ethics, and Society (AIES) 2026
Subjects:
Cryptography and Security (cs.CR); Computation and Language (cs.CL); Computers and Society (cs.CY)
Cite as: arXiv:2607.16010 [cs.CR]
(or arXiv:2607.16010v1 [cs.CR] for this version)
https://doi.org/10.48550/arXiv.2607.16010
arXiv-issued DOI via DataCite (pending registration)
Submission history
From: Saifur Rahman Tamim [view email] [v1] Fri, 17 Jul 2026 14:45:53 UTC (71 KB)
Full-text links:
Access Paper:
View a PDF of the paper titled AI Watermark Evidence Fails Forensic Readiness: An Empirical Evaluation, by Saifur Rahman Tamim and Amir Labib Khan
View PDF
HTML (experimental)
TeX Source
view license
Current browse context:
cs.CR
new | recent | 2026-07
Change to browse by:
cs cs.CL cs.CY
References & Citations
NASA ADS
Google Scholar
Semantic Scholar
Loading...
Data provided by:
Bibliographic Tools
Bibliographic and Citation Tools
Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
Connected Papers Toggle
Connected Papers (What is Connected Papers?)
Litmaps Toggle
Litmaps (What is Litmaps?)
scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
Code, Data, Media
Code, Data and Media Associated with this Article
alphaXiv Toggle
alphaXiv (What is alphaXiv?)
Links to Code Toggle
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub Toggle
DagsHub (What is DagsHub?)
GotitPub Toggle
Gotit.pub (What is GotitPub?)
Huggingface Toggle
Hugging Face (What is Huggingface?)
ScienceCast Toggle
ScienceCast (What is ScienceCast?)
Demos
Demos
Replicate Toggle
Replicate (What is Replicate?)
Spaces Toggle
Hugging Face Spaces (What is Spaces?)
Spaces Toggle
TXYZ.AI (What is TXYZ.AI?)
Related Papers
Recommenders and Search Tools
Link to Influence Flower
Influence Flower (What are Influence Flowers?)
Core recommender toggle
CORE Recommender (What is CORE?)
Author
Venue
Institution
Topic
About arXivLabs
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)