AI サービスが一時的に利用できないため、復旧後に翻訳を補完します。
The following article originally appeared on Addy Osmani’s blog and is being republished here with the author’s permission. A software factory harnesses loops at scale. You can run the loop with humans in it (light factory), trading judgment and concentration against speed and breakage. Or you can ignore the humans (dark factory) and let those agents scope, build, and ship code without anyone reading the details. If people stop reading, though, they’ll stop understanding your software. Your hardest job now is knowing which checks to build and how much autonomy to delegate. This idea of the software factory is a term that dates back to Bob Bemer’s paper, “The economics of program production,” given in 1968. For half a century, many have dreamed of a world in which software is a repeatable and instrumentable production process (analogous to stamping out car parts in a factory) rather than the isolated craft of individuals. Historically, this dream has generally (although not universally) fallen flat, in part because of the difficulty of stamping out ideas. But in the last two years, things have changed dramatically enough that now it makes sense to take a fresh look at the old dream. And since some subtleties can easily be glossed over, it’s worthwhile to be somewhat precise about exactly what’s really new and different, and what may be recurring traps, dressed up as new opportunities. Dex Horthy, co-founder of HumanLayer recently gave a great talk at the AI Engineer World’s Fair called “Harness Engineering is not Enough: Why Software Factories Fail.” worth checking out on this topic. The loop is the atom. The factory is the loop at scale. Structure is everything, and it all starts with small units. The whole stack is really three concepts layered on top of each other: the loop, the harness, and the factory. A loop is one agent doing a single job on repeat: gather context, take an action, check the result, and go again until some condition is met. It is the smallest unit of agentic work, and everything above it is just loops stacked on loops. The point of loop engineering is that you stop prompting the agent turn by turn and instead design the small system that prompts it for you. A harness is the walls around a loop: the sandbox it runs in, the tools it can reach, the memory that survives between runs, and the gates that decide what “done” means. The loop is the behavior; the harness is the environment that behavior runs inside. Hand a raw model no harness and it will happily spin forever. The harness is everything around the model that makes it useful and safe to run. A software factory is many harnessed loops running at once, fed by a queue of work and drained through a review gate into production, with humans owning the whole thing from above. It isn’t a bigger agent; it’s an org chart made of loops. The final paradigm shift is moving from writing code to building and running the factory that writes it. The unit of work shifts up a level, to the loop, the harness, and the flow between them, rather than the individual code diff. Loop → harness → factory. A factory isn’t a smarter agent; it’s many harnessed loops feeding one review gate, with a human owning the outer loop. The factory, drawn The central slide Dex spent most time on was brilliant because it’s a clarifying wiring diagram that visualizes what otherwise is an obvious loop. Here’s my take on it: The factory is a closed loop: Intent and production signals feed a queue, the harness builds, automated checks and review gate it, deploy ships it, and monitoring turns production back into signals. Intent flows from the vision of engineering leadership and directly from engineers into a queue of work. Signals driven by incidents and user requests drive the same queue. The harness picks an item from the queue and builds a change for it. Beyond the harness, automated checks make changes safe enough to let into production. These automated checks run at once without any conscious involvement from engineers, thanks to CI, tests, static analysis, and scanning of all kinds. The review gate is the only decision point here. After approval, changes are deployed and monitored in production, with monitoring data feeding back into the signals that kicked the loop into motion to begin with. By and large, every box in this diagram is almost zero cost: generation, tests, scanning. They all run at scale for negligible cost. There’s only one expensive box that proves stubbornly resistant to scaling, and that’s the review gate. That shiny amber box is “judgment,” and where the crux of the argument about whether we can make development faster and more frequent resides. Why we call it “dark” A dark factory runs with the lights physically off because the only things on the floor are machines, which don’t need light to see. A dark software factory operates similarly, as code ships that no human has read and is verified only by other machines. The image is borrowed from manufacturing. Its origins are physical rather than digital, rooted in facilities where the lights are turned off and the work is carried out by robots. FANUC in Japan has been running lights-out factories of this sort since 2001. Xiaomi, in 2024, opened a heavily automated dark factory of its own. What these have in common is a product assembled and shipped without a single human having read any of it. The “dark” comes in when that act of reading is removed from the process. I’m not borrowing the concept for its vibe or as an insult. For all its creepy buzz, “dark” here is a simple physical claim: the original factory floor, but without light. In software, the floor is the diff. Whoever wrote the diff, whoever reviewed it, whoever shipped it, those humans are gone, and what remains is a diff verified only by the machines that built it. This is a surprisingly easy thing to do, at least at first. It’s easy because that missing review step gets in the way of everything. Its absence makes your perception of your team’s vertical throughput seem suddenly and radically higher. It feels as if you’ve broken the sound barrier. For all its apparent ease, it’s harder than it seems to survive those dark workflows, with all their buried costs. Harness engineering is not enough The harness of orchestration, sandboxed prototyping, and tool calling as models interact with the world and each other will become increasingly powerful and effective. However, there’s an inherent in-model failure in trying to keep up with codebase quality over the long game and through additive changes, and I think there’s good reason to believe that models alone will ultimately lose that battle against comprehension debt. Comprehension debt is the widening gap between how much code exists and how much any human still understands. A dark factory doesn’t pay it down; it takes it on as fast as it can, with the tests green the whole way. This is an important distinction because models do well at some tasks. But for anything that isn’t an immediate change to a small part of a codebase, especially in a complex brownfield system, model-only automated coding faces an insurmountable obstacle. Weekend toys and side projects are alike in that a few months of development cycles is usually enough to get things in working order, or at least close enough. But an enterprise system that has been under development for a decade or more is a different beast; it has to be maintained, in a professional environment at a professional pace. Three to six months into a project, you’re already drowning in unread code. That kind of environment, and especially the constraints enforced by production code, would make even a powerful agent do poorly, all of it in contrast to the vibe-coding enjoyed by developers working on weekend toys. Dex reports from experience that this is a major failure, so much so that it required painstaking manual debugging to pinpoint. This came from running a fully automated code factory for about four months, during which no human looked at the code that was written. Underlying the experience is a tradeoff between two conflicting metrics. One is maximizing token utilization, the number we currently treat as progress. The other, which it quietly minimizes, is the amount of the system any human participant still understands at any moment. Where the dark factory truly shines is in its ability to burn through pristine code while the tests stay green. The ultimate reckoning, when it comes, will not be a dramatic “it all goes sideways” moment. It will be quiet and late. A dark factory pipeline versus a lit factory pipeline: Dark and lit are the same pipeline with the lights in different places. The lit version moves human judgment upstream to design and architecture rather than just re-adding review at the end. The bottleneck was never generation The fundamental constraint in a software factory isn’t how much code we can churn out, it’s how quickly we can verify it. Back pressure is the rule that you can only hand a loop as much autonomy as you can cheaply and reliably verify, and not one inch more. Verification, not generation, is the real constraint on a factory. Because unbounded generation capacity is in perpetual tension with the finite, non-scaling resource of human attention, the core problem is the gap between cheap generation and bounded review. Look at the funnel: As long as the neck representing verification doesn’t widen, it’s going to back up. As Dex points out, volume alone isn’t the problem: What we’re really suffering from is a surplus of bad PRs. When you’ve got high volume without trustworthy gates, manufactured defects are unavoidable. This is just back pressure again: Autonomy can’t expand beyond what can be cheaply and reliably verified. The second-order problem is why improving the model shouldn’t automatically close the gap between what it can generate and what can be verified. Training on well-architected systems is an arguably more difficult proposition than passing simple tests: remember, the cost functions measuring architectural excellence aren’t measured in seconds or even minutes, but in months and years. Tidy gradients are functionally impossible to compute, so a system expecting crisp, instant evaluation of complex design decisions isn’t going to be trained on good examples. Generation is a wide mouth; verification is the narrow neck. Speeding up the mouth just deepens the pile at the neck. Turning the lights back on A lit factory is the same pipeline with the lights left on where judgment lives. Agents still do most of the building, but a human reads what comes out before it ships, keeping the lights on wherever a wrong call is expensive. The lit version doesn’t tack review onto the end but moves the point of human judgment upstream, to the product, the design, and the architecture before an agent starts a loop. One great thing about that upfront hour is that it leads to fewer implementation hours. It turns a long, frustrating code review into a quick read of a two-hundred-line plan. You get to review a decision before it’s built, so later you aren’t chasing through two thousand lines of generated code to find out what the decision even was. Some decisions are expensive and long-lived enough that you’d want a person in on them early, before the cost compounds. Of course, there are still times you look at diffs, even when you’ve spent time up front. You might be thinking that all sounds unglamorous. You’re right. The safety net is made up of perfectly ordinary architectural practices we’ve always known about and mostly ignored: good types and method signatures so that mistakes are caught by the compiler instead of in production; test seams where we can pin behavior and make change observable; laying out the code so the next reader, human or model, knows where to find the thing they care about; keeping call stacks shor [truncated for AI cost control]