AI サービスが一時的に利用できないため、復旧後に翻訳を補完します。
In my previous article, I argued that agentic development needs enough specification to make correctness checkable. A quick prompt can get implementation started quickly, but someone still has to decide whether the result does what we wanted. I left a fairly practical question unanswered, though. Once the agent has written the code, how should we review it? You can give an agent a clear task and get back a serious implementation with tests. The build passes and the code looks reasonable. Now somebody has to approve the pull request, and that person has roughly the same amount of time they had before code generation became cheap. Reading every changed line is one way to approach this. But if each increase in generated code requires a similar increase in human inspection, review will keep limiting how many changes we can accept. We need a way to decide where that inspection is worth the effort. My approach is to make evidence a condition of accepting agent-generated code. Agree on the intended behavior and the expected blast radius before implementation starts. Then use the evidence to check for drift and decide which parts of the code need direct inspection. So far, that may sound like test-driven development. But what happens when every test passes and the agent has added behavior nobody asked for? A small diff can have a large blast radius Let’s use a small fictional example. Suppose we are adding an approval step to a refund service. Refunds above €100 require approval from an authorized employee before the service sends the payment request. Exactly €100 does not require this additional approval. The existing rules about who can request a refund continue to apply. An agent implements the approval step and adds tests. While updating the payment integration, it also adds retries after a timeout. That seems reasonable enough. Network calls fail, and retrying them is a familiar engineering practice. But a timeout leaves us with another question. The provider may have issued the refund before its response was lost. Sending the request again could issue a second refund. Whether that can happen depends on how the provider handles repeated requests and how our code uses that behavior. The approval feature may be correct while the added retry introduces a separate problem. I can see a human review focusing only on the requested feature and easily missing the new decision about payment behavior. By blast radius, I mean how far the effects of a mistake can spread. If the agent puts the retry in a shared payment client, it may affect every refund, including those below the approval threshold. The change now reaches beyond the feature we asked for. Counting the changed lines won’t tell us that. Treat blast radius as a review criterion on its own. Establish which operations a change is allowed to affect, then check whether the implementation stays within that boundary. Also consider how hard a mistake would be to undo. Deploying the previous version won’t recover money that has already been refunded twice. Look for drift in both directions This is the kind of drift I want a review process to expose. We need to check that the requested behavior exists and examine additional behavior that could affect the system. There is an established name for following these relationships in both directions: bidirectional traceability. NASA’s software engineering guidance describes tracing requirements through implementation and verification, including finding functionality that has no parent requirement. I think that the basic idea translates quite well to an ordinary pull request. For each important requirement, we should be able to find the implementation and the checks that support it. Looking back from the implementation, we should be able to explain why a change to behavior belongs in this pull request. An extra private method will usually need little discussion. A retry that changes how money moves needs a reason we can review. Of course, an unplanned addition isn’t automatically a mistake. The retry might be necessary. Implementation often reveals something the original specification missed. We can accept the new behavior once we understand its consequences and agree on how to check it. Until then, it remains an open decision. An agent can help identify these relationships. Give it the agreed specification and the actual repository changes, and ask it to find behavior that is missing or unexplained. For the refund service, that should include the changed payment client even if the agent’s own summary only mentions the approval feature. Start with the changed files so that the summary doesn’t determine what gets examined. Follow those changes into their callers, too. A shared client can change the behavior of code that has not been edited. The tests can share the same mistake While this is a general direction for our attention in a review, the links themselves tell us very little about correctness. A method can reference the right requirement and still implement it incorrectly. A test can have exactly the name we hoped to see and check the wrong result. Consider the threshold again. Suppose the agent reads “above €100” as “€100 and above.” It uses the same interpretation when writing the tests. The implementation requires approval for exactly €100, and the test expects that behavior. Everything passes because the code and the test agree. Running that test in a separate CI environment will reproduce an agreement but it won’t discover that the expected result was wrong. We still need someone to check the meaning of the assertion against the agreed behavior. Spend some of the review time on these expected outcomes. For a requirement like this, agreeing on the boundary examples before implementation is quite manageable. A refund of €100 proceeds without the additional approval. One cent more must wait. The reviewer can then inspect whether the tests actually observe that difference. Have the test observe what the service sends to the payment provider. Checking that an approval request was created is insufficient if the service can also send the payment immediately. For refunds above the threshold, the test should check that the payment request cannot leave the service before approval. That checks the rule at the point where the refund is requested. For an important rule, you can go a little further. Temporarily remove the approval check in a controlled test run and see whether the relevant test fails. If it stays green, it gives us very little protection. Google’s code review guidance asks reviewers to consider exactly this kind of question: Will the tests fail when the code is broken? Use another agent to propose cases we forgot and examine whether the assertions match the requirement. Keep the actual checks available to the human reviewer, though. A different model can still produce the same miss-interpretation. There is some research backing up that caution. A 2026 study of LLM requirement-conformance judgments found models rejecting correct implementations and accepting faulty ones. The study used programming benchmarks, so it does not tell us how every review agent will behave in a production repository. It does give us a reason to look beyond a model’s confident explanation when deciding whether code meets a requirement. Make the evidence easy to inspect Require a short review record with each pull request. It should connect the intended behavior to the checks and expose decisions that remain open. For our hypothetical refund change, it could begin like this: Agreed behavior: Refunds above €100 wait for authorized approval before payment. Exactly €100 does not need that additional approval. Evidence: Boundary tests check when the payment request is sent. The review links to their assertions and the CI results for this revision. Additional behavior: The payment client now retries after a timeout. Blast radius: If the retry applies to the shared client, all refund requests may be affected. Review its existing callers. Open question: The tests do not cover a provider completing the refund before the timeout. Safe retry behavior still needs to be established. Start the review with this record. Check that the approval assertions express the agreed rule and that the results support it. Then inspect the retry path, where both the evidence and the intended scope are incomplete. Require a test covering the uncertain provider response or remove the retry from this change. The open question has become a concrete review decision. Link each claim to the checks that support it. “Tests passed” is too broad to help much. The reviewer needs to see which tests ran against the revision being approved. If the agent changes the implementation after that run, refresh the evidence. Review changes to the checks Compare the checks with the agreed baseline as well. Suppose an earlier acceptance test expected a €100 refund to proceed, but the agent updates it to expect an approval request. The new implementation and the new test now agree. Comparing both with the agreed requirement exposes the change in meaning. An agent does not have to be trying to hide anything for this to happen. Updating a test to match new behavior is part of everyday development. The reviewer needs to decide whether that new behavior was intended. Changes to acceptance criteria deserve the same attention for the same reason. Sometimes the new behavior will be the right approach. Perhaps the business actually wants approval starting at €100. Fine, we can change the requirement. Record that decision and review its consequences before accepting the implementation. Quietly changing the spec to match the code leaves us with agreement and no explanation of how we got there. Set review depth by blast radius Match the depth of review to the potential impact and the strength of the evidence. A report can only expose the gaps it detects. Tests can pass even when the code relies on a wrong assumption. Require direct inspection of the payment path because a mistake can move money twice. Follow the affected calls even if some of them sit outside the diff. For a routine change with limited effects and established checks, the reviewer can spend less time reconstructing the implementation. The justification is specific: The affected behavior is understood, and the checks address its likely failures. Where the blast radius is unclear, investigate it before approving changes. Occasionally inspect changes that the process treated as routine, too. If those reviews keep finding missing assumptions, improve the checks before relying on them to reduce inspection further. This gives the team a way to test its review policy against the code it is actually accepting. Make evidence part of the development cycle Here is the process I recommend. Before generation starts, a developer agrees on the behavior and its expected blast radius with whoever owns the requirement. The agent implements within that scope and prepares a short review record. CI runs the checks against the proposed revision. The reviewer examines the evidence and investigates deviations, with direct code inspection where the consequences justify it. Release through a controlled rollout, then use what happens in production to improve the checks and the specification. Make the review record part of the definition of done. The developer submitting the change owns its accuracy, even when an agent writes it. The reviewer owns the acceptance decision. An important claim with inadequate evidence stays open. If the implementation affects more operations than agreed, resolve that change in scope before proceeding. For the refund example, that means the approval feature can proceed once its checks support the agreed behavior and the reviewer has examined the relevant payment path. The added retry needs its own justification. It can be removed [truncated for AI cost control]