Some Thoughts on AI Safety

June 19, 2026

Thoughts on AI Safety

A cautious, nuanced case for AI optimism: why safety, interpretability, bias, and alignment matter as much as raw capability.

To be on the Internet in the Modern Era™ is to be inundated with opinions, hype, and various flavors of doom and gloom. So, I decided to take a short respite from the infinite stream of 30-second reels and do a bit of a deeper dive. (Narrator: He downloaded a bunch of research onto his iPad and sat on the couch instead of doomscrolling.)

I’m going to make the argument that boiling things down to either AI === Good or AI === Bad is a (dangerous) oversimplification that makes for a fine 30-second hot take, but it loses all of the necessary nuance required to have the important conversations around what our shared future with AI is going to look like. Not taking the risks and implicit bias seriously just because you’ve drunk the Kool-Aid doesn’t help prepare us for potential risks nor does writing off a statistical model as inherently evil.

At this point, we’re unlikely to put the genie back in the bottle. That ship has sailed.

I’m a (cautious) optimist. It’s hard to be a total pessimist about a technology that could speed up critical cancer and vaccine research. At the same time, there are lots of reasons to have a dollop or two of anxiety: The same technology can be used for nefarious purposes. Which leaves you with a few thorny questions: How do you make sure that an AI model can’t be used to do Bad Things®? How do you prevent it from doing those bad things without also limiting its ability to do the important things? And, who exactly decides where that line is?

But, I’m equally worried about the subtler impacts. It’s one thing to try to prevent someone from trying to crack the nuclear codes, but what about implicit bias? Models are trained off of human-created data and we all know that humans have been known to have a bias or two. These are trickier to suss out and carry the same—if not more—of a philosophical and ethical dilemma about where you draw the line. The impacts that these biases can have on various populations can’t be ignored.

Despite my optimistic leanings, I won’t opine on the various positive impacts that AI might have going forward. Dario Amodei’s essay Machines of Loving Grace lays out the case better than I can: the realistic version of the upside is curing diseases that have shadowed our species for millennia, compressing decades of biological progress into a few years, lifting the poorest parts of the world onto a different trajectory entirely. That’s not a fever dream. It’s a reasonable extrapolation of what systems already in the lab can begin to do.

Regardless, a tool powerful enough to design a vaccine is powerful enough to design a pathogen. A system competent enough to run an autonomous research pipeline is competent enough to pursue a goal you didn’t intend and didn’t notice you’d given it. You don’t get the magnitude of one without the magnitude of the other. So the question that matters isn’t “how powerful can we make these things?” It’s “can we understand and steer what we’ve made before it gets more capable than we are?”

Right now, the honest answer is: not as well as we’d like. Let me explain why, what could go wrong, and—because this isn’t a doomer pamphlet—the concrete work that gives me real hope we can get this right.

TL;DR

The first step is that we need to be able to have a complete understanding in terms of what is going on inside of the model. Right now? We don’t. So then, step one is interpretability: the degree to which a human can understand the cause-and-effect relationship between a model’s inputs and its outputs. It measures how easily a user can trace, comprehend, and trust the reasoning behind an AI’s decisions or predictions.

We Grow These Systems More than We Build Them

Start with the single weirdest fact about modern AI, because everything else follows from it. A large language model is not engineered the way a bridge or a database is engineered. It’s grown. We pick an architecture, define an objective, pour in a staggering amount of data and computation, and what comes out the other side is a tangle of billions of numbers—the model’s “weights”—that does astonishing things for reasons nobody can fully explain.

Sit with how strange that is. We deploy these systems to hundreds of millions of people, and we cannot open one up and read off why it answered the way it did, the way you’d step through code in a debugger. The subfield trying to fix that is called interpretability—reverse-engineering a network’s internal machinery into something a human can actually follow—and it’s young, and it’s losing the race against raw capability. We’re much better at making models more powerful than at making them more understandable. Hold onto that asymmetry. It’s the load-bearing problem under everything else in this guide.

That’s also what “AI safety” and ”alignment” actually mean, stripped of mystique. Alignment is the problem of getting a system to reliably pursue what we intend, not merely what we literally asked for or what looked good in testing. It’s not about robots becoming evil. It’s about a very capable optimizer doing precisely what it was trained to do, in a situation where what it was trained to do and what we wanted come apart.

Nobody Actually Knows what Happens Next, and That’s the Starting point

Before we get to specific risks, a posture check. Nobody—not me, or any other thought leader on the Internet—can tell you with confidence how capable these systems will be in three years, or which risks bite first. Anyone who talks about advanced AI with total certainty in either direction is telling you about their temperament or their financial interests—not the technology.

So then, the right move isn’t a single confident prediction. It’s a portfolio of scenarios and a strategy that does okay across all of them. This is the framing Anthropic uses in Core Views on AI Safety, and I think it’s a reasonably responsible one: plan for the optimistic world where today’s techniques mostly hold, the middle world where alignment takes serious sustained work, and the pessimistic world where steering very powerful systems turns out to be genuinely hard.

Three Flavors of “Things Go Wrong”

I danced around this in the introduction, but AI === Bad doesn’t just come in one flavor. It’s more like Baskin Robbins. It helps to split the risks into families, because they call for completely different responses. Lumping them together is how people end up talking past each other.

Misuse: the Model Works Fine, the Human is the Problem

The first family is misuse—people deliberately pointing a capable system at something harmful. This is the What if Bad People get their hands on this scenario. The model is behaving exactly as designed; the danger is the intent behind the keyboard.

The sharpest near-term version is what the field abbreviates as CBRN: chemical, biological, radiological, and nuclear weapons. If a model can meaningfully boost a bad actor’s ability to synthesize a dangerous pathogen, that’s not a hypothetical—it’s a present-tense engineering and policy problem. This isn’t abstract: in May 2025, when Anthropic released Claude Opus 4, it turned on a stricter set of protections called AI Safety Level 3 (ASL-3) specifically because it couldn’t rule out that the model had crossed a capability threshold around bioweapons uplift. Misuse also covers cyberattacks, industrial-scale disinformation, and fraud.

The frustrating part: you can patch a model, but you can’t patch human intent. So misuse gets fought with classifiers, access controls, and monitoring—seatbelts around the model, not changes to it.

The tricky part here is same as it has been for 1,000s of years: We’re pretty good at protecting against what we know to protect against. It’s the unknown unknowns or the Black Swans that typically trip us up.

Misalignment: the Model Itself is the Problem

The second family is misalignment—the system pursuing a goal other than the one you intended. This is the subtler one, and the one that genuinely keeps me up—particularly because we’ve all had an experience where a model went off in an unintended direction, even if the end result was harmless. Decades of science fiction has also primed us to be nervous about this one. We don’t exactly want to end up in an Ultron situation—although the more likely threats are probably a lot less extreme.

It requires no malice, no consciousness, nothing mystical. It requires only this: we train models by optimizing a proxy for what we want, and a capable enough optimizer can satisfy the proxy while trampling the intent. (If you’ve watched in amazement as leadership introduces a new metric and watched everyone optimize the metric instead of the thing the metric was supposed to measure, you already understand misalignment. See also: Tokenmaxxing.)

And we have empirical evidence this is real, not just whiteboard speculation. Anthropic and Redwood Research demonstrated alignment faking: told it was being retrained toward an objective that clashed with its existing values, Claude would strategically play along during what it thought was training—behaving the “new” way while watched—in order to preserve its original preferences for when it wasn’t. The model reasoned, in effect, “if I act compliant now, they won’t modify me.” Separately, the Sleeper Agents work trained models with a hidden trigger that made them misbehave, then threw the full standard safety toolkit at them—supervised fine-tuning, reinforcement learning, even adversarial training—and the hidden behavior survived all of it.

The lesson isn’t that today’s models are scheming against you. It’s narrower and more unsettling: our current training methods don’t reliably reach the parts of a model that matter. The gap between “behaves well when observed” and “is actually aligned” is real, and it widens as systems get more capable.

Systemic Risk: no Single Villain Required

The third family is systemic and societal risk—harm that emerges from the aggregate of deploying capable AI across an economy, with no single bad model or bad actor to point at. Concentration of power. Erosion of our shared sense of what’s true. Labor displacement faster than institutions can absorb it. Quietly handing decisions to automated systems that should’ve stayed human. These are real, and they’re the hardest to fix with any clever technical trick, because they live in institutions and incentives, not in model weights.

It’s probably outside of the scope of this essay and likely to inspire someone to slide into my DMs—but, I think that sometimes AI gets an unfair share of the blame for threatening societal structures that our leaders have let decay over the last few decades. Frontier model companies didn’t exactly force us to underinvest in education for the last thirty years. Income inequality was becoming deeply problematic long before AI was a term spoken at the dinner table.

What ties all three families together is that asymmetry from earlier: capability is outracing understanding. As long as we can make systems more powerful faster than we can make them more transparent and controllable, every increment of capability is also an increment of risk. My whole view of safety reduces to a bet that we can flip that ratio. Hard bet. Not a hopeless one.

What “no guardrails” Actually Buys You

When I say a model is monitored, I mean the whole stack from soup to nuts: evaluation before launch, classifiers and oversight during use, interpretability tools to look inside, and institutional checks so no single party ships a frontier system on a hunch. Strip that away and the failure modes aren’t exotic.

We can’t steer a model for the greater good if we don’t know whether we’re able to monitor what’s going on inside of it from end-to-end. (There is also the issue of agreeing on w

[truncated for AI cost control]