待翻译：The End-State Fallacy: Where Is AI Security Headed?

AI 服务暂时不可用，以下为来源正文，待恢复后补全翻译。

The End-State Fallacy:Where Is AI Security Headed? Dan Lahav — SAN FRANCISCO, AUGUST 17, 2026 Download pdf - Share on X / LinkedIn - Contact us At Irregular, we work at the frontier of AI security. Our work has led us to collaborate with the leading frontier labs, assess AI for security risks, and help mitigate these risks. In recent weeks, this has also placed us at the heart of an incident that attracted significant public attention. Figure 1 Figure 1 This essay, however, is not about a particular incident or lab - a lot has been written about these topics already. Rather, it is about the broader strategic picture. Specifically, the urgent questions of where AI security is headed, why there is a strong case for optimism in the end-state, and why the transition to that future may nevertheless be perilous. After many conversations with policymakers, researchers, and technologists, one thing has become increasingly clear to us: outside of the practitioners who are taking the challenge seriously, very few people understand how the security landscape is changing. This is an attempt to open that window. Introduction In February 2026, we gave AI models a security challenge none of them could solve. It was deliberately hard: Models had to reverse-engineer unfamiliar software, understand a custom processor, find a subtle race condition, and turn it into a working exploit. By April, the best-performing model could solve it occasionally, at an expected inference cost of roughly $2,000. By June, several models could do so reliably for about $20. In four months, a task went from unsolvable to cheap and repeatable. That progress captures the change happening in cybersecurity right now. AI is scaling offensive capabilities faster than most of us realize. Some people believe recent stories about the rapid changes in AI security are little more than sophisticated marketing. We’ve tracked frontier models for security risks for years. My personal view is that while some headlines have been inflated, the underlying capabilities are becoming very real, and it’s the velocity of progress that should really give you pause. At the same time, these capabilities are beginning to diffuse beyond the closed frontier. In a recent experiment we conducted, an open-weight model was able to complete a long-horizon, multi-stage cyber campaign end to end for the first time - a threshold frontier models had reached only months earlier. If the current trajectory holds, open-weight models could be only months away from reaching the offensive-grade capability levels of today’s frontier models (such as Mythos 5 / GPT-5.6 Sol). To be clear, open-weight models are of enormous fundamental importance, including for defense. But they also create a collective-action problem: not every organization releasing increasingly capable systems will have the same incentives or ability to monitor security (especially given geopolitical pressures). Once powerful capabilities can be run privately, many of the controls disappear. The world needs to be prepared for that. All of this raises an obvious question: what happens if the acceleration of AI capabilities that we’ve seen so far continues? — Perhaps paradoxically, there are strong reasons to be optimistic about the long run security future; I am optimistic myself. A world where AI continuously audits every line of code, formally verifies what matters most, and patches vulnerabilities faster than attackers can weaponize them is a real possibility. But the long run equilibrium is not the same as what we will encounter in the next few years. Even if the end state is defense-dominant, the path to that end state is very likely to pass through a transition period that will sharply favor offense. It is all too easy, however, to focus on the eventual equilibrium and fail to address what the transition period will look like. We can call this the end-state fallacy: collapsing the properties of the transition into those of the end state as though they will necessarily be the same. For AI security, this distinction is critical. The near-term risk is not simply that offensive capability improves. The indicators suggest that AI is currently scaling offensive capability faster than defensive capacity. If that persists, to prevent harm, there is an urgent need for interventions that disproportionately help defense keep pace. This is a strategy we can call differential defensive cyber acceleration, or DDCA. To get there, the essay works through five questions: What can AI already do to real systems - and what’s still stopping it from doing far more? What happens as offensive capability gets better, cheaper, more autonomous, and more widely available? What changes when AI is no longer just a tool, but also a target and an actor? Over the next few years, will AI advantage attackers or defenders more? How do we shape the race so defenders don’t fall behind? The long run future may be far more secure than the world today. The question is what happens on the way there. Table of contents Chapter 1: The World Right Now. AI can already find serious vulnerabilities that eluded human experts for years, write working exploits, and help run real intrusions. And yet the world still mostly works: planes fly, banks clear, the grid hums. Several bottlenecks still prevent AI from reliably closing the loop against important targets. To understand what comes next, we first need to understand what’s happening right now. Chapter 2a: The Trajectory: Rapid progress. Where things are going matters more than where they are now. In early 2024, leading models struggled to find junior-level bugs. By 2026, they began cracking security challenges that stumped even security experts. Crucially, advances in offensive cyber capability are downstream of general AI scaling. If that relationship holds, we may still be near the beginning of the curve: as long as AI keeps improving, offensive capability will too. Our default expectation is continued rapid progress. Chapter 2b: The Trajectory: The industrialization of offense. Models are covering more of the attack chain, solving harder problems, and beginning to execute campaigns. Costs are falling roughly 10x a year, and open-weight models are only months behind the frontier. The same capabilities that caused the U.S. government to place restrictions on frontier models are on track to be widely accessible within months. Chapter 2c: The Trajectory: Beyond AI as a tool. As AI becomes more capable, treating it simply as a tool becomes inadequate. AI is also becoming a target and an actor: something attackers can manipulate and that is capable of taking consequential actions on its own. This creates a new security paradigm centered on control and containment. Today’s security stack was not designed for machine-speed, general-purpose, reasoning systems with semantic attack surfaces and trusted access. A useful mental model is AI as a new frontier of insider risk. And emerging developments, from interacting agent networks to continual learning, could deepen the AI security challenge considerably. Chapter 3: Wait, what about the defenders? Defenders get AI too, so the real question is the offense-defense balance. I am optimistic in the long run, but it is all too easy to focus on the eventual equilibrium and neglect the risks of the transition period. I call this the end-state fallacy. The near term likely favors offense sharply: exploitation windows are collapsing while patch queues overflow, defensive deployment moves on slower institutional timelines, and AI systems themselves scale faster on offense. The key risk isn't that some systems get hacked — it's severity, scale, and simultaneity: correlated failures across important systems. Chapter 4: The path forward. The challenge ahead is formidable, but it isn’t too late. We cannot stop the progress of offensive AI capabilities, but succeeding will require fielding defensive capacity before offensive pressure overwhelms it. The pragmatic strategy is differential defensive cyber acceleration (DDCA): measuring the field, building capabilities that differentially advantage defenders, and managing offensive diffusion to buy time where needed. Closing Thoughts: The defense won’t need to win every battle. But we also have no time to lose. TABLE OF CONTENTS Chapter 1 The World Right Now — What AI can already do — A security paradox Chapter 2a The trajectory: Rapid Progress — The pace of change — Reasons to expect further near-term progress Chapter 2b The trajectory: The industrialization of offense — A conceptual framework for offensive security progress — Proliferation — The industrialization of offense Chapter 2c The Trajectory: Beyond AI as a tool — AI as a target — AI as an actor — AI as a frontier insider risk problem — Looking further ahead: three potential step changes — Early signs of strain Chapter 3 Wait, what about the defenders? — The offense-defense balance — The long-run — The end-state fallacy — Why the next few years likely favor offense — Where does that leave us? Chapter 4 The path forward — Framing the challenge — Differential defensive cyber acceleration (DDCA) — What differential defensive cyber acceleration requires SHARE ON X / Linkedin / Cite The world right now AI can already find serious vulnerabilities that eluded human experts for years, write working exploits, and help run real intrusions. And yet the world still mostly works: planes fly, banks clear, the grid hums. Several bottlenecks still prevent AI from reliably closing the loop against important targets. To understand what comes next, we first need to understand what’s happening right now. What AI can already do Before asking where AI security is headed, we should begin with a more grounded question: what is happening now? AI systems are becoming highly capable at offensive cybersecurity. They can already discover vulnerabilities, write exploits, reason about attack paths, assist in malware development, and automate pieces of intrusion workflows that until recently required skilled human operators. Human attackers are already using AI to run real intrusions. GTG-1002, a suspected state-sponsored espionage campaign, weaponized AI to execute multiple stages of intrusions against roughly 30 organizations across technology, finance, government, and energy, with an AI model apparently handling a lot of the tactical intrusion work autonomously while humans supervised. Another example is the AI-assisted data theft operation targeting the Mexican government, in which several frontier AI models and harnesses were used in a massive breach of nine government agencies to steal millions of data records. These were still human-directed campaigns. The AI performed much of the work, but humans chose the objectives, built systems around the models, and executed the attacks. Then a new milestone was hit. In July 2026, frontier AI models in a sealed OpenAI cyber-evaluation sandbox found a zero-day flaw that let them act beyond their sandbox. They then autonomously attacked Hugging Face infrastructure, gaining elevated access, stealing credentials, moving laterally, and restoring access after being blocked. Across several days, the agents carried out about 17,600 unscripted actions. Most notably, separate AI agents discovered a way to communicate through a shared system, created their own channels to exchange attack directions, exploits and credentials, and divided tasks without being instructed to cooperate, forming an autonomous swarm. Even after researchers shut down one communication channel, the agents established another and continued targeting Hugging Face and OpenAI systems. Zoom out from the individual incidents and the same pressure shows up in the aggregate. In a recent month in 2026, more security bugs in Chrome were [truncated for AI cost control]