My Team Looks Six Times Bigger, and I Haven't Hired Anyone

How we delegated code review and verification to an AI agent, grew review capacity from 310 to 1,860 cycles a week, and kept a human on every merge.

#ai#ai-agents#code-review#productivity#engineering-culture
Cover image for the article: My Team Looks Six Times Bigger, and I Haven't Hired Anyone

Last quarter, someone in a budget meeting asked me how many engineers I'd added this year. I said none. They looked at our delivery numbers, then back at me, and asked the question again, slower this time, the way you talk to someone who might not have understood.

I understood fine. We're the same 45 people we were a year ago. But our review pipeline now moves like a team of 250, and I want to explain honestly how that happened, including the parts where I was wrong.

The bottleneck was never writing code

Here's what I believed for years: if my team ships slowly, we need more hands writing code. So you hire, and hiring is slow and expensive, and every new person needs six months to become truly useful.

Then I actually looked at where our time went. Our engineers wrote code fast. What was slow was everything after: waiting for a reviewer to free up, the reviewer context-switching out of their own work, the back-and-forth over issues that should have been caught in minutes, then someone manually confirming the change actually did what the ticket asked. A pull request opened on Tuesday morning routinely merged Thursday afternoon, and maybe 40 minutes of that 50-hour window was actual work.

We didn't have a coding shortage. We had a confirmation shortage.

What we actually delegated

So we gave the first pass to an AI agent. Not a linter with opinions, but an agent that does what a good reviewer does:

Diagram of the delegated review loop: a pull request goes to an AI agent that reads the diff and ticket, hunts bugs and security issues, runs tests and verifies results in about 7 minutes; issues go back to the author, and a clean verdict goes to an engineer who confirms and merges

The agent reads the diff and the ticket it claims to solve. It hunts for bugs and security issues. It runs the test suite and (this is the part that changed everything) it verifies the results: does the change actually produce the behavior the ticket describes? Then it writes up a verdict: what it checked, what it found, what it couldn't verify.

A human engineer reads that verdict and confirms. Not rubber-stamps, confirms. They check the judgment calls, they own the merge. Our rule is written on the (virtual) wall: the agent reviews, a human confirms. Nothing ships on the agent's word alone.

The part where I was skeptical

I'll be honest: I didn't trust it at first, and neither did the team. Our most senior reviewer (the person whose approval everyone quietly wanted) called it "autocomplete with confidence issues." Fair.

So we ran it in shadow mode for six weeks. The agent reviewed every PR, but its comments went to a channel nobody had to read. We measured it like we'd measure a new hire: how often was it right, how often was it noise, how often did it miss something a human caught?

The numbers surprised us in both directions. It caught real bugs humans had approved, including a race condition in our dispatch logic that two senior engineers had waved through. It also confidently flagged things that were fine, about one comment in five, early on. We tuned what it saw (giving it the ticket context cut the false positives almost in half) and set a bar: it earned the first-review job only when its precision beat our human baseline. It took a month and a half to get there.

What ×6 actually looks like

A "review cycle" for us means: code reviewed, tests run, results verified against the ticket, verdict delivered. Here's that number, same 45 people:

Bar chart of review cycles completed per week, rising from 310 with humans only to 1,860 with the agent reviewing and humans confirming, six times more

And the number engineers feel is how long you wait before anyone meaningful looks at your work:

Grouped bar chart of time to first review feedback: median dropped from 9 hours to 8 minutes, and 95th percentile from 26 hours to 40 minutes

The table I showed the budget-meeting person:

| Metric | Before | After | |---|---|---| | Review cycles / week | 310 | 1,860 (×6) | | Median time to first feedback | 9 h | 8 min | | Median time to merge | 26 h | 3.5 h | | Bugs caught before staging | baseline | +41% | | Senior-engineer hours on routine review / week | ~140 h | ~35 h | | Headcount | 45 | 45 |

That last row is the one I keep pointing at.

What the humans do now

This is the question I get from other CTOs, usually with a raised eyebrow: so what do your reviewers do all day?

They review the things that deserve a human. Architecture decisions. API contracts other teams will live with for years. The PR from the junior engineer who doesn't need a verdict, she needs a conversation. The judgment calls the agent explicitly marks as "couldn't verify, needs a human."

My senior engineers got roughly 100 hours a week back collectively, and none of them spend it doing nothing. They mentor more. They fix the things everyone had learned to live with. One of them finally rebuilt our flaky end-to-end test suite, which (irony noted) made the agent's verification pass even more reliable.

And the junior engineers may have gained the most: instant, patient, judgment-free first feedback. Nobody feels embarrassed pushing a rough draft at an agent. The polished version is what their colleagues see.

The honest fine print

  • The agent is confidently wrong sometimes. Less than it used to be, but it happens. That's exactly why the confirm step is non-negotiable: it converts "AI decides" into "AI drafts, human decides," which is a completely different risk profile.
  • It doesn't understand your politics. It will correctly flag that a change breaks another team's assumption, and have no idea that the fix requires a conversation, not a commit.
  • Capacity is not strategy. The agent made us six times faster at confirming work. It did not make us six times better at choosing what to build. That's still the hard part, and it's still ours.

The takeaway

I used to think the choice was: hire more people or ship less. It turns out there was a third option hiding in plain sight: stop making expensive humans do the part of the job a machine can draft, and let them own the part that actually needs them: the confirmation, the judgment, the call.

My team looks six times bigger. It's the same 45 people. They're just finally spending their hours on work worthy of them.

Comments

    No comments yet. Be the first to share your thoughts.