The Sequence Opinion #892: The Anatomy of a Good Environment: When Verifiability is Not Enough

Quick note: For over two years now, we’ve been running The Sequence without sponsors. Literally every week we receive tens of inquires to sponsor The Sequence. As a passion project that I have been running for over five years now, we are trying to keep The Sequence true to its ethos of technical depth, objectivity and unique content to help you navigate the world of AI. The best way to help us on that journey is to subscribe below. We understand is not for everyone but appreciate the support.

I was listening to the recent Dwarkesh Patel episode with Grant Sanderson, where he argues that “grindability is just as important as verifiability,” and it crystallized something I’ve been chewing on for months, so I decided to elaborate on that thesis here. The question is deceptively simple: what makes a domain a good domain for AI? Not good in the sense of commercially interesting, but good in the sense that if you point a modern training pipeline at it, capability actually compounds.

The standard answer is verifiability, and it’s not wrong. But I’ve come to believe verifiability is just one axis in a higher-dimensional space, and the domains where AI has been shockingly good (math, code, board games) are the ones that happen to score high on all the axes simultaneously. The domains where progress has been grinding and disappointing, like computer use, robotics, and open-ended knowledge work, are usually strong on one or two axes and quietly broken on the others. Once you see the full set of properties, a lot of otherwise confusing facts about the current moment snap into focus: why reasoning models got good at math before they got good at your inbox, why a cottage industry of RL environment startups suddenly commands billion-dollar budgets, and why I think some of those environments are going to disappoint their buyers.

Let me walk through the axes one at a time. For each one I’ll try to give you a domain that has the property and a domain that conspicuously lacks it, because the contrast is where the intuition lives.

Verifiability

Read more