MTTR is the wrong metric for Engineering Teams today

AI makes code generation and deployment faster, but our safeguards haven’t caught up.

For over a decade, mean time to recovery, or MTTR, was the industry’s primary protective measure against bad code. Organizations assumed failure was inevitable and used MTTR to measure how fast they could recover. The smaller your MTTR, the better.

MTTR, however, doesn’t map to AI-era failure modes. It doesn’t meaningfully capture the speed of AI-generated code production, nor the potential blast radius of autonomous agents.

Every engineering team is already rethinking their deployment process, so it’s time to rethink recovery, too.

Enter mean time to failure, or MTTF.

MTTF first emerged in hardware contexts, where it referred to the average operating time before a non-repairable component broke and required replacement. In software, we can use the same framework to focus on the component we can’t have broken: Your customer’s experience.

DORA, SRE, and the assumption of failure

MTTR might feel like it’s been around forever, but it’s only been around for about a decade. It has its roots in DORA and site reliability engineering (SRE), both of which were responses to the rise of distributed computing.

MTTR was a valid response to a real shift in how we produce software, but with the rise of AI, old ideas working from previously valid principles can cause damage. That’s why it’s worth rewinding to see why MTTR emerged, and why it no longer suits what we need today.

DORA: DevOps Research & Assistance

The first study in what would become DORA attempted to tie IT performance to organizational performance. The goal was ambitious: How could organizations link sometimes abstract-feeling technical goals to real business value?

The research turned up four values, split into two categories:

1 Throughput measures: Deployment frequency and deployment lead time.

2 Stability measures: MTTR and change fail rate.

Note that, in the years since, there have been updates. In 2023, DORA renamed MTTR to failed deployment recovery time to focus on failures caused by software changes, rather than events like data outages. And in 2024, DORA added a fifth metric: deployment rework rate. By then, however, MTTR had already become the de facto term.

In the years since, DORA has become the preeminent methodology for measuring performance, replacing (and in many cases upgrading) other methods. These five metrics subsequently became canonized, and even organizations that don’t use DORA know to prize these metrics.

(Source)

As DORA metrics were taking off, Google was formalizing a new role and philosophy: Site Reliability Engineering, or SRE. The SRE approach emphasizes automation, reliability, and hands-on incident management. This places SRE engineers in charge of building fault-tolerant systems and minimizing failure.

MTTR became a key metric for Google’s SREs, resulting in MTTR’s spread as other organizations adopted SRE roles.

As the metric spread, business leaders keyed in, too. Many organizations defaulted to these metrics, along with the frameworks they implied, to build consensus across teams. Over time, organizations could capture failure in one simple number, and trust everyone understood what they meant. Shared understandings can be dangerous, however, when new concepts unseat previously foundational assumptions.

The limitations of MTTR

MTTR is a widely accepted standard. The problem with standards is that the map can be mistaken for the territory. What happens then when the territory changes underneath this map?

Goodhart’s Law and its consequences

When Anthropic published a case study with Spotify, the online response wasn’t entirely positive, with developers pointing out that velocity doesn’t equal value, and fewer bugs don’t mean a better experience.

As Mitchell Hashimoto, cofounder of HashiCorp, writes in a representative post, “This is the problem with a lot of AI reporting. It reports completely meaningless metrics.”

Production deploys per day, proportion of AI-assisted PRs, MTTR: They’re all classic cases of Goodhart’s Law, which states, “When a measure becomes a target, it ceases to be a good measure.”

Goodhart’s Law is a common citation in these conversations, but it can be easy to dismiss.

As Hillel Wayne, a software development consultant, writes,

“The weak version of this is that people have perverse incentives to game the metrics.” Measure development productivity by lines of code, for example, and you’ll find yourself with verbose code, not greater productivity.

Wayne, instead, emphasizes a strong version:

“Even 100% honest pursuit of a metric, taken far enough, is harmful to your goals, and this is an inescapable consequence of the difference between metrics and values.” This version requires no bad faith or gaming. An honest pursuit of one metric among others has a way of twisting things in unintended ways.”

The philosophy underpinning MTTR (that you can't prevent every failure) isn’t wrong, but it licensed the treatment of customer pain as a cost of doing business. As development accelerates, MTTR, followed faithfully and innocently, can cause teams to rely on customers having negative experiences, especially as they scale up velocity.

The pain of MTTR in the age of AI

MTTR sanitizes what failure means. Ultimately, customers had to experience a failure, report it, file a ticket, and wait for it to be fixed. That’s true no matter how low your MTTR is.

Depending on your scale, that could mean tens, hundreds, thousands, or millions of people having bad experiences. And because there’s typically a delay between a bug happening and a user reporting it, users can live with bad experiences for long periods.

Worse, only a fraction of users ever report bugs, meaning many will suffer through the experience, feeling discouraged or becoming churn risks.

With AI, the risk and scale of this failure pattern is only increasing. According to research from DORA and Google, an increase in AI adoption has been accompanied by a 1.5% decrease in delivery throughput and a 7.2% decrease in delivery stability.

And they’re not alone. Research from Uplevel complements these results, showing a 41% increase in bug rate resulting from the use of generative AI tools.

If you measure success based on MTTR, you're allowing your customers to experience paper cuts frequently and big pains periodically. If you accept that pattern in the name of fixing it fast, you accept a bad experience in disguise.

This was just as true before AI as it is now, but with AI, velocity and bugs are rising.

You’re putting more work in front of your customer than ever, and it's buggier than ever. Bad experiences will occur more frequently. Even if AI helps you resolve them, you're still making your customers play quality assurance for you.

Sadly, the pain is also disproportionate in a way no company wants: The customers suffering the most will be your power users and early adopters.

Organizations end up in a situation where AI transforms everything except their ability to reliably provide quality user experiences. Fixing it fast still means you're on day zero, again and again and again.

That's no way to build a product, and it's certainly not a good way to measure quality.

That's why MTTR has to go.

MTTF: An old metric with a new life

You’ve likely heard of MTTF, or mean time to failure. MTTF has its origins in hardware, where organizations used it to measure the average time before components broke down and required replacement.

Reframed for software, MTTR has a new lease on life and the capacity to change how we think of failure in the wake of AI.

Here, failure isn’t an outage or a dip in reliability but a customer having some level of negative experience. This shift in metrics also requires a shift in emphasis away from post-deployment response to pre-deployment review and runtime analysis.

Code review is the current bottleneck

With AI, we can accelerate sheer code output. However, a 10x increase in code also requires a 10x increase in code review capacity. Unfortunately, as an industry, we’ve spent much more time increasing code production than improving our review processes.

Anyone familiar with systems building knows what happens next.

In The Phoenix Project, written almost 15 years ago, the authors write,

“Any improvement made after the bottleneck is useless because it will always remain starved, waiting for work from the bottleneck. And any improvements made before the bottleneck merely results in more inventory piling up at the bottleneck.”

Right now, we’re just putting more pressure on the code review bottleneck and hoping for increased throughput anyway. The results speak for themselves.

Faros AI’s 2025 telemetry study of 10,000+ developers found that “A 98% increase in merged pull requests coincides with a 91% increase in review time and flat organizational delivery metrics.”

The New York Times, providing an even more vivid example, spoke with a financial services company that went from producing 25,000 lines of code per month to 250,000, resulting in a backlog of 1 million lines of unreviewed code.

Some metrics moved (PRs and lines of code), but did the most important ones change for the better?

The blast radius of bad code is growing

Despite the bottleneck, the risk of rushing code review is enormous.

Research from METR shows that the length of tasks that AI agents can complete on their own is doubling about every 7 months. As true AI-generated code scales, the volume of potential bugs increases (recall the 41% increase in bug rate cited earlier).

With agents creating code autonomously, including bugs, the blast radius of failures can increase in tandem. More bugs across more code can result in domino effects that spread across systems, and because agents author most of the code, the failures can be inscrutable to human incident response teams.

Code review processes, as they stand today, are not up to the task.

You try to catch things before they ship, knowing that you likely won’t catch everything. That leaves you acknowledging that you’ll only catch the issue later, in production.

From there, you can roll back, fix, and keep compressing, but never eliminate, MTTR.

This is especially risky when MTTR fails to capture a complete view of poor customer experiences.

Do you know what you’re recovering from?

According to research from VOID, there is no correlation between incident duration and incident severity, meaning MTTR can be misleading.

According to the company’s research, organizations “can have long or short incidents that are very minor, existentially critical, and nearly every combination in between. Not only can duration not tell a team how reliable or effective they are, but it also doesn’t convey anything useful about the event's impact or the effort required to deal with the incident.”

As a result, MTTR captures skewed data when we need precise data more than ever.

The more code you generate, the more bad experiences MTTR will fail to capture and prevent. You’ll never know which issues were fine and only seen by one small customer, and which ones hit your core user base on a core workflow, or a key buyer on the day they’re making a purchasing decision.

New metrics, new workflows

MTTF implies a new way of thinking. Suppose there’s a bug in the code. Do we care?

The answer requires a follow-up question: Does the bug cause a bad customer experience?

If no, we don’t care; if yes, we do care, and we care enormously.

The throughline is not quality for quality’s sake, but confidence that customers will not have bad experiences.

Short-term, that means accelerating developers with AI, relieving the code review bottleneck without sacrificing or driving your engineers crazy with endless code review, and trusting that user experiences will remain good throughout. Long-term, this enables teams to ask difficult questions without worr

[truncated for AI cost control]