[Submitted on 12 Oct 2026]
Title:Lessons from Luck: Do LLM-Written Lessons Carry Chance into an Agent's Next Decision?
View a PDF of the paper titled Lessons from Luck: Do LLM-Written Lessons Carry Chance into an Agent's Next Decision?, by Kazuki Nakayashiki
View PDF HTML (experimental)
Abstract:Self-improving agents write lessons from the outcomes of their runs; when outcomes are noisy, a lesson can carry chance into later decisions. We measure one link of this chain in a registered study. In 256 synthetic worlds, each a recurring task with two interchangeable tools of recorded reliability, a scripted previous run used the more reliable tool (a good decision) or the less reliable one (a bad decision), with an assigned outcome: success or a transient failure. A model wrote a lesson from the run with ReasoningBank's memory-writing instruction; a fresh session of the same model, holding the authoritative log and the previous result, chose the next run's first tool with one memory item: that lesson, the lesson from the other outcome, an irrelevant item, or an unreasoned directive. The protocol was frozen, deposited and timestamped before any confirmatory call; 11 LLM endpoints gave 90,084 answers. After a chance failure of the better tool, the panel's lessons carried the chance (registered reading luck-carrying; interaction +0.112, 95% interval [+0.097, +0.126]): readers named the worse tool in 11.6% of next runs with the lesson written from that failure, against 0.4% with an irrelevant item and 0.4% with the lesson written from a success. The effect was +0.042 where the log stated that failures are independent and +0.181 where it did not (descriptive). After a lucky success of the worse tool, the interaction was bounded below 0.10 (+0.002), a bound, not evidence of no harm. In exploratory readings, 6 endpoints carried the chance (up to +0.352); 5 were bounded below 0.10. In secondary comparisons, a three-sentence decision-quality note in the writing instruction lowered the worse-tool rate after a chance failure to 7.5%, and one instruction for both outcomes with a no-change option to 3.1%; neither met the registered harm-reduction rule at every such endpoint.
Comments: 28 pages, 1 figure, 11 tables. A registered study: the protocol, code and stimuli were frozen, deposited (doi:https://doi.org/10.5281/zenodo.23253562) and timestamped in a Bitcoin block before the confirmatory run. The locked run records are public (doi:https://doi.org/10.5281/zenodo.23272809)
Subjects:
Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Information Retrieval (cs.IR)
Cite as: arXiv:2610.33029 [cs.LG]
(or arXiv:2610.33029v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2610.33029
arXiv-issued DOI via DataCite
Submission history
From: Kazuki Nakayashiki [view email] [v1] Mon, 12 Oct 2026 03:17:10 UTC (82 KB)
Full-text links:
Access Paper:
View a PDF of the paper titled Lessons from Luck: Do LLM-Written Lessons Carry Chance into an Agent's Next Decision?, by Kazuki Nakayashiki
View PDF
HTML (experimental)
TeX Source
view license
Ancillary-file links:
Ancillary files (details):
REIMPL_COMPARISON.json
RESULTS_registered.json
registered_supplement.md
Additional Features
Audio Summary
Current browse context:
cs.LG
new | recent | 2026-10
Change to browse by:
cs cs.AI cs.CL cs.IR
References & Citations
SciX
Google Scholar
Semantic Scholar
Loading...
Data provided by:
Bibliographic Tools
Bibliographic and Citation Tools
Bibliographic Explorer Toggle
Bibliographic Explorer (What is the Explorer?)
Connected Papers Toggle
Connected Papers (What is Connected Papers?)
Litmaps Toggle
Litmaps (What is Litmaps?)
scite.ai Toggle
scite Smart Citations (What are Smart Citations?)
Code, Data, Media
Code, Data and Media Associated with this Article
alphaXiv Toggle
alphaXiv (What is alphaXiv?)
Links to Code Toggle
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub Toggle
DagsHub (What is DagsHub?)
GotitPub Toggle
Gotit.pub (What is GotitPub?)
Huggingface Toggle
Hugging Face (What is Huggingface?)
ScienceCast Toggle
ScienceCast (What is ScienceCast?)
Demos
Demos
Replicate Toggle
Replicate (What is Replicate?)
Spaces Toggle
Hugging Face Spaces (What is Spaces?)
Spaces Toggle
TXYZ.AI (What is TXYZ.AI?)
Related Papers
Recommenders and Search Tools
Link to Influence Flower
Influence Flower (What are Influence Flowers?)
Core recommender toggle
CORE Recommender (What is CORE?)
IArxiv recommender toggle
IArxiv Recommender (What is IArxiv?)
Author
Venue
Institution
Topic
About arXivLabs
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Which authors of this paper are endorsers? | Disable MathJax (What is MathJax?)