Code Is Cheap, Judgment Is Scarce: Engineering Roles in the AI Era

AI coding assistants have made the first draft of almost any change dramatically cheaper. An engineer can describe a feature and get back a working implementation, tests, and deployment configuration in minutes. The natural expectation is that teams should now ship proportionally faster.

Many teams find something different. Engineers feel more productive, and they open more pull requests than ever. But those pull requests pile up. Diffs get larger, approvals take longer, and senior engineers spend most of their week reading code they did not write. Delivery speed, measured end to end, improves far less than individual output suggests, and sometimes not at all.

Other teams see the opposite: delivery looks fast, approvals come quickly, and nothing seems stuck. Then incidents climb, rework grows, and fewer people can explain how parts of the system actually work.

These are two symptoms of the same cause. Writing code was never the only step in delivering software; someone also has to decide whether the code is right. AI made the first step cheap and left the second one roughly where it was. The bottleneck did not disappear. It moved from writing code to trusting code, and trusting code runs on human judgment. When that judgment cannot keep up, a team either waits for it or skips it.

The Constraint Moved

The Theory of Constraints offers a simple rule: a system moves only as fast as its slowest step. Speeding up any other step does not speed up the system; it only grows the pile in front of the bottleneck. Speeding up the bottleneck itself moves the constraint to whatever step is slowest next.

For most of software history, producing working code was a slow, expensive step. AI coding assistants made the first draft of that code dramatically cheaper. The next step in the line, deciding whether the code is correct, safe, and worth merging, did not get cheaper at the same rate. So the queue moved there.

The industry data is consistent with this picture, though it shows correlation rather than proof of cause:

• More output, longer reviews. Faros AI's 2025 analysis of telemetry from more than 10,000 developers across 1,255 teams found that teams with high AI adoption completed 21% more tasks and merged 98% more pull requests, while PR review time rose 91%. Average PR size rose 154%. The study found no significant correlation between AI adoption and company-level delivery improvement.
• Feeling faster is not the same as being faster. In a randomized study by METR, 16 experienced open-source developers working on 246 real issues in mature repositories took 19% longer when AI tools were allowed. Afterward, the same developers estimated that AI had made them about 20% faster.

None of these studies settles the question by itself, and tools have improved since the data was collected. But together they point in the same direction: individual output rose faster than the organization's ability to verify and absorb it.

Generating code is no longer the expensive part of software delivery. Establishing confidence that the code is correct is. Teams that keep measuring the first while the second piles up will feel busy and ship the same.

Two Ways the Bottleneck Shows Up: The Queue and the Rubber Stamp

When more changes arrive than reviewers can properly judge, something has to give. There are only two options, and most teams drift into one without choosing it.

The queue: the bar holds, delivery stalls

Reviewers keep their standard. Each change still gets real scrutiny, so pull requests wait. Review latency climbs, authors context-switch while they wait, and senior engineers spend their weeks reading diffs. This version is painful, but it is visible: anyone looking at lead time can see where the work is stuck, and the system's quality is protected while the team decides what to do.

The rubber stamp: delivery looks fast, quality quietly declines

Reviewers keep the pace instead of the standard. Nobody wants to be the person blocking ten teammates, so reviews get shorter and shallower. A 600-line change gets the same five-minute look that a 60-line change used to get. “LGTM” arrives quickly, the queue never forms, and the delivery dashboard looks healthy.

The cost does not disappear. It moves into the system: bugs that a careful reader would have caught, duplicated logic, inconsistent patterns, and code that nobody on the team fully understands. That debt shows up later as incidents, rework, and slower changes in the parts of the codebase that grew fastest.

The data that exists is consistent with this second pattern, though no study we know of measures rubber-stamping directly:

• Faster delivery, less stable delivery. The 2025 DORA report found that AI adoption is now associated with higher delivery throughput, but also with continued increases in delivery instability. DORA describes AI as an amplifier: it magnifies an organization's existing strengths and its dysfunctions.
• More review time, still more bugs. In the same Faros AI data, high-AI-adoption teams saw a 9% increase in bugs per developer, even though review time rose 91%. Review effort grew, but not enough to keep pace with change volume and size.

The rubber stamp is the more dangerous of the two outcomes precisely because it is the more comfortable one. A queue forces a conversation about review capacity. A rubber stamp hides the problem until it surfaces as an outage or a rewrite. A team that sees no review queue after adopting AI should not assume it has no review problem; it should check whether its reviews still mean anything.

Why AI-Generated Code Is Harder to Review

If review were just a matter of volume, the fix would be more reviewers. The harder problem is that each AI-assisted change can cost more to review than an equivalent human-written one. That makes both outcomes above more likely: careful review gets slower, and shallow review misses more. There are four reasons.

1. Plausible-wrong code removes the usual warning signs

Reviewers rely on cues. Inconsistent naming, awkward structure, or a missing test often signal that the author was unsure, and the reviewer slows down. AI-generated code tends to be fluent, well-formatted, and confidently commented even when its logic is wrong. A subtle off-by-one in a pagination query, an authorization check on the wrong object, or a retry loop that silently duplicates a payment can all arrive wrapped in clean, idiomatic code. The reviewer has to find the error without the signals that used to point to it.

2. The author may not be able to explain the change

The most efficient review question has always been “why did you do it this way?” When an engineer wrote every line, the answer was usually quick. When a large part of a diff was accepted from an assistant, the honest answer may be “it worked when I ran it.” The reviewer then has to reconstruct the intent that the author never fully formed, which turns a ten-minute review into an hour of reverse-engineering.

3. Volume invites rubber-stamping

When a reviewer faces twice as many PRs, each larger than before, the realistic response is not twice the effort. It is less attention per change, which is how teams slide into the rubber-stamp outcome described above. AI-generated code makes the slide easier, because its fluent surface gives a skimming reviewer nothing to snag on. The review gate stays in the process, but it catches less and less.

4. Cross-stack changes need cross-stack reviewers

AI makes it easy for one builder to change the UI, the API, the database schema, and the deployment configuration in a single pull request. That is efficient for the author. For the reviewer, it means one change now needs frontend, backend, data, and infrastructure judgment at the same time. Broader ownership on the authoring side concentrates expertise demand on the review side, and the few engineers who can judge all four layers become the constraint for the whole team.

Trust Is the New Unit of Work

It helps to name the scarce resource precisely. It is not code, and it is not reviewer hours in general. It is judgment: an experienced person deciding whether a change does what it should, fails safely when it doesn't, and fits the system it is entering.

Judgment does not get much cheaper. A senior engineer deciding whether a change to a billing flow is safe still needs the time that decision takes, and AI does not remove the responsibility for getting it right. What a team can make cheaper is trust: the total cost of reaching justified confidence in a change. That cost has three components, and each one can be reduced:

1. How often judgment is needed. Many changes can be decided by automated checks without a person examining them at all.
2. How much each judgment costs. A small, well-explained, well-tested change takes minutes to judge. A large, unexplained one takes hours.
3. How much a wrong judgment costs. If a mistake can be detected and reversed in minutes, the team does not need certainty before merging.

The goal is not to review less carefully. It is to spend human judgment only where it changes the outcome, and to make each of those decisions as cheap and as recoverable as possible. That is the only way out of the choice between the queue and the rubber stamp: if the total amount of judgment a team needs stays within what its reviewers can actually give, reviewers can keep both the pace and the standard.

Three Levers for Making Trust Cheap

Lever 1: Let machines decide the routine cases

Every repeatable rule a machine can check is one a reviewer does not have to verify by eye on every change. Many teams already run tests and policy checks automatically on each pull request in a CI pipeline. The shift is to make those checks required gates for the rules they cover, while people still judge intent, design, and high-risk changes.

Consider a feature that lets one user invite a teammate. A senior engineer identifies a rule that should survive future edits: expired invites must not grant access. The feature developer adds automated tests for that case, with a quality engineer helping find edge cases. The team makes those tests required in CI. Asking for a test on one PR is ordinary review; keeping the rule as a standing gate for later changes is the process shift. A passing test shows only that its covered cases passed. A change to who can grant access still needs a security specialist.

• Encode correctness where it can be encoded. Type systems, machine-checked API contracts, schema migration checks, and static analysis catch whole classes of mistakes before a person looks.
• Test properties, not only examples. Example-based tests check the cases someone thought of. Property-based tests and contract tests check rules that should always hold—“no user can read another tenant's records,” “a refund never exceeds the original charge”—across many generated inputs. They are especially valuable against plausible-wrong code, because they do not share the author's assumptions.
• Turn standards into policy-as-code. Rules a reviewer repeats in comments every week—no secrets in config, no public storage buckets, every endpoint behind an auth check—belong in automated policy checks.
• Tier changes by risk. Decide ahead of time which changes can merge on automated checks alone and which require a human. A copy change or a dependency patch inside an approved range is different from a change to authentication, payments, data deletion, or a database migration.

Lever 2: Make each human judgment cheaper

When a change does need a person, the author's job is to make that person's decision fast.

• Keep changes small. Review effectiveness falls quickly as diffs grow. Stacked changes, PR size limits, and feature flags that let incomplete work merge safely keep each review within what a person can actually hold in mind. AI makes large diffs effortless to produce, so the size limit has to be a deliberate rule.
• Require the author to explain the change. A PR description should state the intent, the approach, the alternatives considered, and the riskiest part of the change. If the author cannot write that, the change is not ready for review. This is the single most effective defense against the “it worked when I ran it” problem, and it puts the cost of understanding on the person who generated the code.
• Point the reviewer at the risk. “Please check the retry logic in charge.ts; everything else is generated scaffolding covered by tests” is worth more than any amount of reviewer diligence spread evenly across a diff.
• Use AI review as a first pass, not a verdict. AI reviewers are useful for triage: summarizing a change, flagging likely bugs, and catching style and consistency issues so the human can focus on design and intent. They are not an independent check. An AI reviewing AI-generated code can share the same blind spots, so an AI approval should never satisfy a gate that requires human judgment.

Lever 3: Make wrong judgments cheap to recover from

Review tries to establish trust before a change reaches users. Production tooling lets teams establish it afterward, with real evidence. The cheaper it is to detect and reverse a mistake, the less certainty a team needs before merging.

• Feature flags separate deploying code from exposing it, so a change can ship dark and be turned on gradually.
• Canary releases and staged rollouts expose a change to a small share of traffic first, with automatic rollback when error rates or latency move.
• Observability tied to the change—metrics, traces, and alerts that show the effect of a specific deploy—turns “did this break anything?” from a guess into a lookup.
• Fast, rehearsed rollback keeps a bad judgment a minor incident instead of an outage. Changes that cannot be rolled back, such as destructive migrations, are exactly the ones that belong in the high-risk tier.

The three levers reinforce each other. Automated checks shrink the number of decisions. Small, explained changes shrink the cost of each decision. Recoverable releases shrink the cost of a wrong decision. Together they let a team absorb much more AI-generated output without either drowning its reviewers or lowering its bar.

What Happens to Roles

If judgment is the scarce resource, roles reorganize around it. Titles may not change much, but what makes each role valuable does.

Senior engineers: from primary authors to judgment multipliers

The senior engineer's value used to show up largely in what they built. Now it shows up in two places: the quality of the decisions they make on high-risk changes, and the systems they help the team build so that fewer decisions need them at all—contracts, property tests, risk tiers, policy checks, and golden paths. A senior engineer who reviews every PR personally becomes the bottleneck. One who identifies recurring rules, has the team encode them as tests or policy, and makes them standing gates multiplies their judgment across the team.

Builders: accountable for explaining, not just producing

Any engineer can now produce a large amount of code. What separates a strong builder is whether they can stand behind it: explain why it works, name where it might fail, and size changes so they can be reviewed. “I own this change” has to mean “I understand this change.” Teams should treat a PR its author cannot explain the same way they treat a PR that fails CI.

QA and quality engineers: from test writers to verification designers

AI can draft example-based tests quickly, but generated tests tend to encode the same assumptions as the generated code. Quality specialists become more valuable, not less, when they focus on what generation does poorly: defining the invariants that must always hold, designing property-based and contract tests, maintaining test data that reflects real edge cases, and studying escaped defects to find which class of check was missing.

Platform and security engineers: building the trust infrastructure

The levers above are mostly platform work: CI gates, policy-as-code, preview environments, feature-flag systems, canary tooling, and change-level observability. Security engineers write the rules that decide which changes are routine and which need them. Their leverage is in making the safe path automatic, so that their own attention goes to the novel risks no rule anticipates.

Product managers: owning the definition of “correct”

Much of review is really about intent: does this change do what the customer needs? When product intent is vague, every reviewer has to infer it, and every review takes longer. PMs reduce the cost of trust by writing outcomes and acceptance rules precise enough to test, so that “does this do the right thing?” becomes a check rather than a debate.

In every role, the pattern is the same. The work that grows in value is either exercising judgment on the decisions that truly need it, or building the systems that make the other decisions automatic.

The Apprenticeship Problem

There is an uncomfortable consequence. Good judgment comes from experience: writing code, getting it wrong, debugging it, and learning why. If junior engineers mostly accept generated code that they never fully understand, they can ship plenty of features while building little of the judgment that the team will need from them in five years.

Teams that care about their future reviewers should design for this deliberately:

• Make juniors reviewers early. Pair them on reviews of high-risk changes, and have them review AI-generated code with a senior engineer who explains what they are checking and why.
• Hold the explanation standard for everyone. Requiring every author to explain their change is not only a review aid; it forces the understanding that builds judgment.
• Debug before you regenerate. When generated code fails, have the author find and explain the root cause before asking the assistant for another attempt.
• Rotate people through incidents. Post-incident reviews are where judgment is built fastest, because they show exactly which assumption failed.

An organization that optimizes only for short-term output can end up with plenty of code generators and a shrinking number of people able to judge what they produce. That is a slow-moving risk, and it is far cheaper to address now than after the senior engineers who hold today's judgment have moved on.

What to Measure

The metrics a team watches determine what it optimizes. If AI moved the constraint to trust, the metrics should follow it there. (For a deeper look at why output metrics break down, see Counting the Wrong Thing.)

• Review latency: time from PR opened to first meaningful review, and to merge. A rising number means the team is in the queue.
• Review depth relative to PR size: time-to-approval and review comments per PR, read against diff size. If changes grow while approvals get faster and comments get fewer, the team is likely in the rubber stamp. Review latency alone will not show this; a team that stopped reviewing has excellent latency.
• PR size distribution: rising median diff size is an early warning of rising review cost and falling review quality.
• Share of changes merged on automated checks alone: a rising share, without a rising failure rate, means Lever 1 is working.
• Change failure rate and recovery time: the DORA stability measures show whether faster merging is costing reliability, and whether Lever 3 keeps mistakes cheap. These are the lagging indicators of the rubber stamp; by the time they move, the debt is already in production.
• Escaped defects by category: each defect that reached production points to a check that could have caught it.

What not to celebrate on its own: lines of code generated, PRs opened, or tasks completed. Those measure the step that is no longer the constraint.

AI made producing code cheap. It did not make deciding whether code is right any cheaper, and that decision is now where software delivery strains. Teams that respect the constraint feel it as a queue. Teams that ignore it feel it later, as declining quality in a system that looked fast the whole time. The teams that benefit most from AI will not be the ones that generate the most code. They will be the ones that make trust cheap: automating the routine decisions, shrinking and explaining the rest, and making mistakes recoverable. The engineers who matter most will be the ones whose judgment the team relies on, and who build the systems that let the rest of the team rely on it less.