Back to Insights

The Review Ceiling

A growing PR queue can mean too few reviewers, or too much understanding left for them to supply.

7 min readBy The Bushido Collective
Engineering LeadershipAI ToolsCode ReviewTeam VelocityTechnical Strategy
Share:LinkedInX
Open your pull-request queue. Find the oldest change that’s ready for review. Is it waiting for someone to read it, or for someone to work out whether it’s safe?

That distinction matters if AI is helping your team produce changes faster. An untouched request may need someone to make time for it. A request stalled in a discussion about failure behavior may need a decision the author never made. Both look old on a dashboard. They call for different work.

When incoming changes consistently require more review effort than the team can supply, the backlog grows. That’s the review ceiling. Faster generation can push a team into it, but a queue alone doesn’t establish that AI caused the delay. Start with changes ready for review, excluding drafts and work waiting on the author.

Use the oldest request to find a cause worth investigating, then check whether it recurs in comparable changes. Separate time awaiting a reviewer from time spent investigating, and from time awaiting the author’s next revision. Comment timestamps can locate the waits; ask the people involved where the work went. A long gap between comments won’t tell you whether someone was reproducing a bug or working on something else.

The Confidence Problem

An author who flags an unresolved edge case gives the reviewer somewhere to slow down. A generated patch can arrive with tidy names, sensible-looking error handling, and an explanation that sounds settled. If the author passes that explanation along without identifying what remains untested, the reviewer loses a useful map of uncertainty. A subtle bug can look as assured as the routine logic around it.

Consider a handler that checks whether an event has already been processed, applies a credit, then records completion. Suppose duplicate deliveries can run concurrently and nothing prevents them from both passing the initial check. Each can see an unprocessed event and apply the credit. A test that sends the event once never exercises that failure. Clear names and helpful comments can describe every step perfectly while leaving the concurrency problem intact.

A human can write that code too. The extra review burden arises when generating a plausible implementation gets ahead of establishing why it works. The reviewer must reconstruct the event-delivery rules, inspect the storage guarantees, and decide what evidence would settle the question. Reading the patch is only part of the job.

In a 2022 study of 47 participants, Neil Perry and colleagues compared programmers with and without access to an OpenAI Codex assistant. The assisted group produced insecure answers more often on most of the security tasks, yet rated their answers as more secure on average. The sample was mostly students, and the experiment measured security exercises with an older model rather than production reviews. It supports caution about confidence; it doesn’t establish that generated code always takes longer to review or that polish caused the effect.

For the event handler, the handoff needs to show what prevents another delivery from applying another credit. A test should make duplicate attempts contend at that check and inspect the resulting balance in the actual store. Merely launching requests together can miss the race if one finishes before the other begins processing. The location of the protection matters too: if applying the credit commits separately, rejecting a duplicate completion record afterward is too late.

Review still has to find things the author missed. The author’s contribution is to make their assumptions and evidence inspectable, including what they haven’t tested. That gives the reviewer a claim to challenge instead of an assurance to take on trust.

Change What Reaches Review

If the delay is before anyone starts reading, review coverage is a reasonable first response. Give engineers who already understand the affected area time to review, and route work to them. If only one person has that context, pairing another engineer with them can develop an additional reviewer, though it costs the expert attention during the handover. More available reviewers can raise the ceiling. More names on an approval list, without the time or context, leave the work where it was.

When reviewers repeatedly have to untangle unrelated changes, reduce the amount they must hold in mind. Google’s review guidance defines a small change as one self-contained change, with its related tests and enough context to understand it. It also cautions against splitting so finely that the implications become difficult to see. A short patch can still require a large decision; cutting it into smaller files won’t make that decision disappear.

Automated review deserves the same attention to the kind of work it removes. In Google’s 2024 AutoCommenter study, researchers estimated from code changes that developers resolved about 40% of the tool’s posted comments about coding best practices. Its randomized rollout detected no statistically significant change in total review duration, active review time, or comment-response rounds. It did detect a slight improvement in coding speed. That leaves room for value without demonstrating a faster review queue.

That was a particular best-practice checker in Google’s workflow, so the result can’t settle the value of every AI reviewer. If your reviewers spend time catching issues that a linter or review assistant can catch reliably, automating that work is a sensible option. Judge it by the effort it removes, including time spent rejecting bad suggestions. A bot producing more comments gives you another activity count.

If an unresolved assumption could change the implementation, settle it before generating more code around it. For the event handler, that can mean the author and someone who knows the delivery system working through the duplicate-delivery case together. Put the evidence in the eventual review so the next reader can challenge the answer without having to recover the conversation.

That conversation uses the same scarce expert attention as review. It saves effort only if it avoids more work later, such as rebuilding a patch around a corrected assumption or explaining the same decision repeatedly. Making it mandatory for every patch can add a new queue.

Count the early work when comparing author-and-reviewer effort on similar changes, and follow elapsed time from starting the work to release. A shorter wait can be useful even when effort stays the same. But starting review work before the review clock begins can make the dashboard look faster without improving either. Routine changes with established behavior and useful checks may need nothing more elaborate than an available reviewer.

Follow the Wait Past Approval

Some delays survive a perfectly good review. If an approved change still needs another service to deploy, speeding up review moves it to the next queue. Trace a completed feature through those handoffs before adding approval capacity.

A cross-team check may protect a real dependency: changing a shared API can break a consumer. Removing the sign-off without handling that dependency leaves the risk intact. If a check merely repeats a decision already made by an accountable owner, there is a different conversation to have about keeping it. If the history doesn’t explain what the check protects, ask its owner before treating it as redundant.

Return to that oldest ready-for-review change. Name the fact or decision its reviewer still needs. If nothing is missing, give the review time. If something is missing, agree who will investigate it rather than sending another finished-looking patch into the queue.

Want this looked at in your business?

Start with the rough map: thirty minutes, owner to owner, and a written report on where AI pays off for you and what it's worth. It's free, and if you don't need us the report says so.

Get your rough map, free

Not ready to talk? Stay sharp anyway.

We send insights like this to technical leaders every week or two. The thinking we bring to our engagements, no fluff, no spam.

Keep reading

Share:LinkedInX