The Calculator Discipline – AI-Assisted Disclosure Hallucinations
A new paper examines how AI-generated vulnerability reports are flooding open-source security projects, proposing a taxonomy, a pre-send tool, and real case studies to help reduce AI slop. The author, who has submitted such false reports himself, advocates for 'calculator discipline': AI makes a careful user faster and a careless user wrong faster.
Published May 26, 2026
| Version 1.0
Working paper
Open
The Calculator Discipline — AI-Assisted Disclosure Hallucinations
Authors/Creators
Thomas, Stuart Paul (Researcher)
Description
AI assistance has made source-code review cheap, and like every productivity multiplier in the history of engineering it has therefore made being wrong cheap. The open-source security community has spent the last eighteen months noticing the result: bug-bounty intake queues drowned in plausible-sounding but fabricated vulnerability reports, with the curl project's January 2026 closure of its HackerOne programme the headline example. The conversation so far has mostly been complaint. What is missing is a taxonomy of the failure modes, a pre-send filter that catches the most mechanical of them, and honest case studies from researchers who have themselves shipped the slop.
This paper supplies all three. We propose a four-class taxonomy (bug-shape fabrication, evidence fabrication, severity inflation, trivial-as-critical), present two real disclosure withdrawals and one near-miss caught before send, and describe a working pre-send tool (hallucination_check.py) whose four verifiers were derived from those cases. The author is one of the people who shipped the slop; the discipline described here exists because the failure happened to him.
The framing throughout is that AI is a calculator: a tool that makes a careful user faster and a careless user wrong faster. The fix is not to disown the calculator; the fix is to apply calculator discipline.
Other
The paper is released under CC BY 4.0. The accompanying tool described in section 6 (hallucination_check.py, approximately 35 KB) is released separately under the BSD 2-Clause Licence and is distributed via the project's public artefacts directory.
Case studies in sections 2 and 3 reference disclosures made to the OpenBSD project ([email protected] and [email protected]) during May 2026. Verbatim text of [email protected] correspondence is not reproduced in this paper out of respect for the list's private status; paraphrasing in section 3 preserves the substance.
This paper was drafted with LLM assistance (Claude, Anthropic) as a reasonable adjustment under Equality Act 2010 §20 (neurodivergent author). The author independently verified every cited file path, commit hash, person's name, and URL before publication.
Files
TheCalcDisc.pdf
Files (102.5 kB)
Name Size
Download all
TheCalcDisc.pdf
md5:ee8c526abee9ab32480f3e24743b010b
102.5 kB
Preview
Download
Additional details
Identifiers
URL
https://stuart-thomas.com/research/calculator-discipline/
URL
https://triageforge.co.uk/pages/case-study-calculator-discipline.html
Related works
Is documented by
Publication:
https://triageforge.co.uk/pages/case-study-calculator-discipline.html
(URL)
Is identical to
Publication:
https://stuart-thomas.com/research/calculator-discipline/
(URL)
Dates
Available
2026-05-26
References
[1] D. Stenberg, "Death by a thousand slops," daniel.haxx.se, 14 July 2025. https://daniel.haxx.se/blog/2025/07/14/death-by-a-thousand-slops/
[2] D. Stenberg, "The end of the curl bug-bounty," daniel.haxx.se, 26 January 2026. https://daniel.haxx.se/blog/2026/01/26/the-end-of-the-curl-bug-bounty/
[3] D. Stenberg, "AI slop attacks on the curl project," daniel.haxx.se, 18 August 2025. https://daniel.haxx.se/blog/2025/08/18/ai-slop-attacks-on-the-curl-project/
[4] B. Toulas, "Curl ending bug bounty program after flood of AI slop reports," BleepingComputer, 2026. https://www.bleepingcomputer.com/news/security/curl-ending-bug-bounty-program-after-flood-of-ai-slop-reports/
[5] "AI is drowning software maintainers in junk security reports," Help Net Security, 18 May 2026. https://www.helpnetsecurity.com/2026/05/18/problems-with-ai-assisted-vulnerability-research/
[6] T. Krazit, "cURL's Daniel Stenberg: AI slop is DDoSing open source," The New Stack, 2026. https://thenewstack.io/curls-daniel-stenberg-ai-is-ddosing-open-source-and-fixing-its-bugs/
[7] "AI slop got better, so now maintainers have more work," The Register, 6 April 2026. https://www.theregister.com/software/2026/04/06/ai-slop-got-better-so-now-maintainers-have-more-work/5223172