The AI Verification Bottleneck: Why Writing Code Is No Longer the Hard Part

AI is making software cheaper to produce. The harder problem is establishing that the software is correct, secure, and safe to deploy. For years, improving developer productivity largely meant reducing the time required to turn a requirement into working code. AI coding assistants and agents are now very good at that part of the job. In a controlled experiment, developers using GitHub Copilot completed a programming task 55.8% faster than developers without it. (Microsoft Research) But faster implementation does not automatically mean faster delivery. A pull request still has to be reviewed. Tests still have to run. Security controls still have to pass. Integration behavior still has to be understood. Someone still needs to decide whether the change actually implements the business requirement. If implementation capacity increases much faster than those activities, the constraint moves downstream. For teams using AI heavily, I think that is becoming one of the more important engineering problems to solve. AI increases the amount of software that needs to be verified The important distinction is between code generation and trusted software delivery. Consider a typical enterprise change. A developer asks an AI agent to modify an API, refactor a service, or optimize a database query. The agent can inspect the repository, make changes across several files, generate tests, and produce a pull request in minutes. That is useful. The productivity gains are real. But the resulting pull request still has to pass through roughly the same system: automated tests, static analysis, security checks, dependency checks, integration testing, architecture review, business validation, deployment controls, and production monitoring. The first stage has accelerated much faster than many of the others. DORA’s 2025 research found that a 25% increase in AI adoption was associated with a 1.5% decrease in software delivery throughput and a 7.2% decrease in delivery stability. These are observational findings, not proof that AI itself causes worse delivery performance. DORA’s explanation is that AI can increase the size and volume of changes, making them harder to review and more likely to create instability. (DORA) That is the pattern I care about. The question is no longer only how quickly we can generate a change. It is how quickly we can establish enough evidence that the change is safe to put into production. Those are different capabilities. Verification debt I use the term verification debt for a simple situation: an organization increases the amount of software it can produce faster than it increases its ability to establish that the software is correct, secure, and appropriate for production. Imagine a team that historically handles 100 meaningful production changes per month. Its review process, test infrastructure, security controls, and operational practices have evolved around roughly that volume. Now AI increases implementation capacity to 300 changes per month. That does not mean the organization can safely deliver 300 changes. If the verification system can still process only 100 changes effectively, the other 200 changes have simply become a queue. The numbers are hypothetical, but the mechanism is straightforward. This is why I am increasingly skeptical of metrics such as lines of code generated, number of pull requests, or raw developer throughput when evaluating AI adoption. A more useful question is: How much verified production change can we safely deliver? That measures the entire system rather than its fastest component. Verification is not one thing “Verification” sounds like a single activity, but it is really a collection of different checks. A compiler verifies some properties of a program. Unit tests verify others. Static analysis catches another class of problems. Security scanners look for known vulnerability patterns. Integration tests examine interactions between components. None of these tells us whether we implemented the correct business behavior. I find it useful to think about verification in layers. Technical verification asks whether the implementation behaves according to its local expectations: does it compile, do the tests pass, and does static analysis find obvious problems? System verification asks what happens when the component interacts with databases, queues, APIs, caches, payment providers, and other services. Security verification asks whether the change introduces authorization problems, data exposure, dependency risks, or new attack paths. Then there is business verification: did we actually implement the intended business rule? This is where many automated checks become weak. Code can be internally consistent while implementing the wrong interpretation of a requirement. Finally, operational verification asks what happens under retries, timeouts, partial failures, traffic spikes, deployments, and rollbacks. Governance adds another question: can we explain what changed, why it changed, and what evidence supports releasing it? These checks overlap, but they are not interchangeable. A green test suite is evidence. It is not proof that the business process is correct. The problem with AI verifying its own work AI should be part of the verification system. An agent can generate tests, review a pull request, look for security problems, identify edge cases, inspect logs, and compare an implementation against documentation. But there is a problem with using essentially the same reasoning process to generate and verify a change. Suppose an agent interprets a requirement, generates the implementation, generates the tests, runs them, and reviews the resulting code. The workflow looks comprehensive. But the tests may encode the same misunderstanding that produced the implementation. If the agent misunderstood the business requirement, it can generate code consistent with that misunderstanding and tests that confirm it. Everything passes. The result is still wrong. This is why I think independence between verification layers matters more than simply increasing the number of checks. Static analysis can challenge implementation patterns. Security analysis can challenge security assumptions. Integration tests can challenge system-boundary assumptions. Property-based or adversarial tests can challenge behavioral assumptions. Observability can challenge assumptions about production behavior. Human review can challenge business and architectural assumptions. The goal is not to make every check AI-powered. The goal is to make the checks different enough that they do not all fail for the same reason. NIST’s current DevSecOps guidance makes a similar point: AI can improve efficiency, but AI-generated content should still be monitored and validated by humans, with verifiable processes for establishing its accuracy and trustworthiness. (NIST) OWASP’s guidance is even more direct. It recommends reviewing AI-assisted code for vulnerabilities and specifically warns against treating a passing, AI-generated test suite as independent assurance. (OWASP) A practical verification pipeline The answer is not putting another human in front of every pull request. That simply moves the bottleneck to humans. I would rather build a pipeline in which cheap, repeatable checks happen automatically and expensive human judgment is reserved for the cases where it matters: AI-generated change → static analysis → unit tests → security checks → integration tests → AI-assisted review → staged deployment → observability → targeted human approval The exact pipeline should depend on the system. For a low-risk internal service, automated verification may be sufficient. For a payment system, identity service, pricing engine, or system handling regulated data, the verification requirements should be much higher. There is no universal “AI verification pipeline.” Risk should determine verification depth. The practical consequence is that humans should spend less time checking things machines can reliably check and more time answering questions such as: Did we implement the intended business behavior? What assumptions does this change make? What happens when an external system behaves unexpectedly? What happens when an event is delivered twice? What happens if the database update succeeds but the message is not published? Those are harder questions than “does the test pass?” A simplified e-commerce scenario Consider a simplified checkout flow during a major sales campaign. Traffic is higher than normal and checkout latency has increased. The payment provider is healthy; the bottleneck is inside the checkout service. An engineer asks an AI coding agent to optimize the checkout path. Within minutes, the agent proposes a small change across several files. The code looks clean, follows existing patterns, and passes the fast unit tests. Under normal circumstances, the change would go through the full regression suite and a more extensive review. But the campaign is already running and checkout is slow. The change looks low-risk, so the team deploys it. The common checkout path works. There is, however, another path involving promotional discounts and gift cards. That path is not covered by the fast tests. The optimization changes how the final transaction amount is obtained. Under the less common path, the service can use a stale checkout value instead of the authoritative server-side amount. The payment succeeds. The problem is that it succeeds for the wrong amount. This is the kind of failure that is easy to miss in AI-assisted development. The AI did not produce obviously broken code. The code was plausible and the tests were plausible. The failure occurred at a system boundary that the verification process did not exercise adequately. The second failure is more interesting Now suppose the team finds the problem and asks the AI to generate a patch. The agent produces a fix that forces the checkout service to use the authoritative transaction amount. The team tests a normal card payment. The amount is correct. The patch looks good and is deployed. But the patch has changed another assumption in the payment flow. The authoritative amount is now established later in the transaction lifecycle, after some order state has already been created. Normal card payments continue to work. A less common payment method processes confirmation asynchronously. The confirmation can now arrive before the order reaches the state expected by the event-processing logic. The payment succeeds, but the order does not progress correctly. The engineering team now has to reconcile payment records against orders, identify affected transactions, repair order state, and determine whether customers or inventory records were affected. Again, the interesting part is not that the AI generated absurd code. It did not. The code was reasonable, the individual checks were reasonable, and the business outcome was still wrong. That is the verification problem I expect AI-assisted engineering to expose more often: failures at boundaries between otherwise reasonable pieces of software. What was actually missing? It is tempting to conclude that the team simply needed more tests. Sometimes that is true. But in this example, the deeper problem is that the verification questions were too similar. The team verified that the checkout service returned the expected amount. It did not independently verify that the amount sent to the payment provider was the authoritative transaction amount. It verified that a payment-success event could mark an order as paid. It did not sufficiently verify what happened when the same event was delivered twice or arrived at an unexpected point in the transaction lifecycle. The difference matters. The first question is: Does the code work? The more important question is: Can the business trust what the code does when real sys [truncated for AI cost control]