Measuring AI Productivity with gh, jq, and Git: Why Your Numbers Are Flat and Rework Is Up

Measuring AI productivity: Why your numbers are flat and your rework is up

All articles

Measuring AI productivity: Why your numbers are flat and your rework is up

Across engineering organizations AI usage is up ~65% but PR throughput rose only 7.76%. Here's why your productivity dashboard can't tell motion from delivery, and four metrics you can baseline next week with nothing but gh, jq, and git.

Brandon WaselnukJun 17, 2026Engineering InsightsAI Engineering

If you've had the misfortune of opening social media recently, you've probably seen it everywhere. Tokenomics has come and the Reapers are here.

Tokenomics refers to the concept that while many of us have started to use AI tools and are feeling productive by generating mountains of code, shipping refactors, and going to bed while agents work overnight, the mergeable output and shipped software to production has not kept pace. What it's describing is token yield: what is the outcome of all of this generation for the business, for your customers, and for your own sanity? Even the Linux Foundation has announced its Tokenomics Foundation as well as its upcoming conference Tokenomicon, a brilliant hat tip to the Necronomicon. They named the conference for AI budgets after the Book of the Dead, which feels about right.

These jokes exist because the bill is real and it hurts. AI is now one of the fastest-growing line items on an engineering budget and in the Linux Foundation's own words, "the discipline to measure and govern that spend has not kept pace."

Someone finally measured#

DX ran a 400 organization study from late 2024 through early 2026 and found that pull request throughput was only up 7.76% while AI usage rose ~65%. METR surveyed 349 technical workers in May, 87 software engineers among them, and the median self-report was two times more valuable work and three times faster. However, METR's own staff gave the lowest estimates of any subgroup in their survey. They believe it's because their team has read about the perception gap research and had an observation bias. When the people who are most familiar with the measurement data don't trust their own feelings, that tells you something.

In another 2025 METR study, 16 experienced open source developers worked on 246 real tasks on their own repos. Each task was randomly assigned to either allow or forbid AI use. Before starting, they predicted that AI would make them 24% faster. After finishing, they believed it had made them 20% faster.

They took 19% longer. The perception error survived contact with the real experience. Surveys can't save you.

METR followed up that study with another in February 2026 with 57 developers and over 800 tasks. They discovered that the slowdown shrank or in some cases even reversed, though the confidence intervals cross zero and METR itself claims only "some evidence for speedup." Three key things had changed in this time period. The models got better, Opus 4.5 had shipped mid-study. Developers got more practice with these tools. And the cohort itself changed, with 47 newly recruited developers working on a more diverse set of repos. This isn't a knock on the research, it's the finding. Every number in this field is a snapshot. Specific set of people with a specific set of repos using specific models to accomplish their outcomes. Borrowed conclusions don't transfer into your company. So the only productivity number worth anything is the one you measure yourself.

Why does my dashboard say we're productive?#

We all know that unconstrained AI produces more code and bigger diffs, so the cost moves towards review. Faros measured that review time was up 91% in teams with high AI adoption. That means your throughput chart counting when the PR opens isn't helpful. Like a tokenmaxxing leaderboard, while it may motivate people to start using AI it isn't tracking to useful outcomes. What's concerning is when this review queue backs up far enough. Teams do the obvious thing. They stop reviewing. A separate Faros report found that 31.3% more PRs are merged with no review at all.

CircleCI found another interesting fact. The median team pushed 15% more throughput to feature branches while their main-branch throughput dropped 7%. Even the top 10% of teams, with feature branch throughput up ~50%, saw main-branch activity stay roughly flat. And main branch merge success now sits at 70.8%, the lowest in over 5 years. Motion is up, yet delivery is flat. And all the tokens in between are on fire.

"It's the J-curve," they'll shout. "You're measuring in the dip!" This will remain unfalsifiable if we never define what climbing out of the J-curve looks like. A real J-curve claim requires a dated exit criterion. And if no one will write down when the curve turns and what number proves it, it's not a useful model. It's an excuse with a graph that's leading you to burn tokens.

What to do about it#

You can't prove a delta without a before. And most teams deleted their "before" the day they rolled out these new tools. It's critical that you first answer the question, "What was our baseline?"

Same-engineer baselining where you compare each engineer to their own pre-AI history, not to other teams, is a useful technique. This way it can kill confounds like tenure, team changes, seasonality, or even self-selection (early adopters were likely your strongest engineers). The proof that this works is from DX's financial services case where AI users had an increase of 30% PR throughput year over year against their own baseline and non-AI users had a 5% increase at the same company.

To get a baseline next week, here's what you do. For each engineer, track four items: Cycle time, review time, rework within 30 days of merge, and defect escape rate. Then segment AI-assisted work against the rest. Make sure you aggregate at the team level. Never use this for individual performance. The moment an engineer thinks that this is to track their personal productivity, it looks like a report card and your data will die.

None of this needs a vendor. If your code lives on GitHub or GitLab, the data already exists. The whole stack in the examples below uses GitHub CLI (gh), jq, git, and a cron job.

Cycle time#

Cycle time is the time from opening a pull request to merging it. Use the median, not the mean, otherwise one stale PR will ruin your analysis.

bashgh pr list --state merged --limit 100 \ --json createdAt,mergedAt \ | jq '[.[] | (.mergedAt|fromdateiso8601) - (.createdAt|fromdateiso8601)] | sort | {merged_prs: length, median_cycle_hours: ((.[length/2|floor]) / 3600 | floor)}'

Two things will mess with this number: Draft PRs and Bots

Draft PRs#

You only need to do this if your team leverages draft PRs. The clock above is designed to start when the PR is created. If your team opens drafts early to run CI, have AI agents review in remote, or share progress, the timer will run while that work is still half done and your cycle time will look worse than it is. GitHub records the moment a draft flips to ready for review so you can measure from that instead:

bashgh api graphql -f owner="$(gh repo view --json owner -q .owner.login)" \ -f repo="$(gh repo view --json name -q .name)" -f query=' query($owner: String!, $repo: String!) { repository(owner: $owner, name: $repo) { pullRequests(states: MERGED, last: 100) { nodes { createdAt mergedAt timelineItems(itemTypes: READY_FOR_REVIEW_EVENT, first: 1) { nodes { ... on ReadyForReviewEvent { createdAt } } } } } } }' \ | jq '[.data.repository.pullRequests.nodes[] | ((.timelineItems.nodes[0].createdAt // .createdAt) | fromdateiso8601) as $start | (.mergedAt | fromdateiso8601) - $start] | sort | {merged_prs: length, median_cycle_hours_from_ready: ((.[length/2|floor]) / 3600 | floor)}'

Bots#

Your team is likely running at least Dependabot. And its PRs either merge themselves in minutes or sometimes sit ignored for months. Either way, you don't want to measure a robot here, exclude them with: --search "-author:app/dependabot".

Review time#

One query that gets you two numbers. How long code waits for human eyes and how much of it merges with no eyes at all. Remember that 31.3% from Faros? This is how you measure your own:

bashgh pr list --state merged --limit 100 \ --json createdAt,reviews \ | jq '{ total: length, merged_with_zero_reviews: [.[] | select((.reviews|length)==0)] | length, median_hours_to_first_review: ([.[] | select((.reviews|length) > 0) | ((.reviews | map(.submittedAt) | sort | first | fromdateiso8601)

• (.createdAt|fromdateiso8601))]

| sort | (.[length/2|floor]) / 3600 | floor) }'

One caveat: first review doesn’t always mean a thorough review. A rubber-stamp approval counts the same as someone who read every single line. In this analysis, that's fine because you're trying to watch for a trend, not grading the actual reviews. But you should watch the two numbers together. If reviews are arriving faster while more PRs merge with no review at all, you may not be speeding up—you’ve just stopped reviewing.

Rework#

Rework can be measured in many different ways, here is an approach that's fast and requires nothing but git history, python, and a splash of cron. The goal is to answer the question “of all the code that we deleted this month, how much was brand new?” Old code getting deleted is normal and celebrated. Three week old code getting deleted means you paid to write it, paid to review it, and then paid a third time to replace it.

How the measurement works: Walk every commit from the last 30 days. For each line a commit deleted, ask git blame how old that line was. If it was less than 30 days old when it was deleted, count it as rework. Young deletions divided by all deletions, that's the rate. This is the metric that dashboards charge for, and under the hood, it's some version of these same three git commands.

Save this script as rework_rate.py at the root of your repo and run it with Python 3. One word of warning: it’s slow on big and busy repos because it runs git blame once for every block of deleted lines, so put it in a monthly cron.

Note: 30 days is the default. If your team is high velocity, run a 14 day version.

rework_rate.py#!/usr/bin/env python3 """Rework rate: share of deleted lines under AGE days old when deleted. Usage: python3 rework_rate.py [window_days] [age_days] (run at repo root)""" import re, subprocess, sys

WINDOW = sys.argv[1] if len(sys.argv) > 1 else "30" AGE = int(sys.argv[2]) if len(sys.argv) > 2 else 30

def git(*args): return subprocess.run(["git", *args], capture_output=True, text=True).stdout

young = total = 0 for entry in git("log", f"--since={WINDOW} days ago", "--no-merges", "--pretty=%H %ct").splitlines(): sha, ts = entry.split() deleted_at = int(ts) for f in git("diff-tree", "--no-commit-id", "--name-only", "-r", sha).splitlines(): diff = git("diff", "-U0", f"{sha}^", sha, "--", f) for m in re.finditer(r"^@@ -(\d+)(?:,(\d+))?", diff, re.M): start, count = int(m.group(1)), int(m.group(2) or 1) if count == 0: continue blame = git("blame", "--line-porcelain", f"-L{start},+{count}", f"{sha}^", "--", f) for a in re.finditer(r"^author-time (\d+)", blame, re.M): total += 1 if deleted_at - int(a.group(1)) = "2026-05-01" AND created = "2026-05-01" AND created = 2026-05-01 AND created < 2026-06-01"}'

Returns {"count": N} — run once per query, divide

Linear#

Two saved views filtered by the bug label and created date, one with the escaped label added. Each view shows its count. For the cron, one GraphQL call returns both numbers:

bashcurl -s https://api.linear.app/graphql \ -H "Authorization: $LINEAR_API_KEY" \ -H "Content-Type: application/json" \ -d '{"query": "{ all: issues(first: 250, filter: { labels: { name: { eq: \"bug\" } }, createdAt: { gte: \"2026-05-01\", lt: \"2026-06-01\" } }) { nodes { id } } escaped: issues(first: 250, filter: { and: [ { labels: { name: { eq: \"bug\" } } }, { labels: { name: { eq: \"escaped\" } } } ], creat

[truncated for AI cost control]