Understand, Design, Review: How We Handle AI-Generated Code

LinearB's 2026 Software Engineering Benchmarks Report (opens in new tab) looked at 8.1 million pull requests from 4,800 teams. Pull requests produced with AI were merged within 30 days 32.7% of the time. Unassisted pull requests: 84.4%.

That does not say AI writes bad code. It says that two thirds of AI-produced changes did not make it into the codebase within a month of being opened. Somewhere between opening and merging, those changes were judged not right, not finished, not understood or not wanted, or they never got the attention they needed to be judged at all. That judgement is the job now.

Agents made writing code cheap. They did not make knowing what to write cheap, and they did not make being sure it is right cheap. If you buy custom software, the second and third things are what you are paying for. This post is about how we do them.

The job didn't shrink. It moved.

The most cited experiment on this question is still METR's randomised controlled trial from 2025 (opens in new tab). Sixteen experienced open-source developers worked on 246 real issues in large repositories they knew well. With AI tools allowed, they took 19% longer. Before the study they expected a 24% speedup; afterwards they still believed they had been about 20% faster.

It is often waved around as "AI makes developers slower". That is the wrong lesson, and METR says so itself: the authors explicitly do not claim AI fails to speed up most developers, and their own February 2026 follow-up (opens in new tab) points to a speedup for returning participants (an estimated 18%, with a confidence interval from 38% faster to 9% slower). METR also calls that newer data unreliable, partly because so many developers no longer wanted to work without AI at all, and between 30 and 50% of participants said they had held back tasks because they did not want to do them without AI.

The lesson we take is narrower and more useful. One of the factors METR lists for the 2025 slowdown is that AI does comparatively worse in settings with very high quality standards and many implicit requirements: documentation, test coverage, conventions nobody wrote down. That is a precise description of business software that has to run for fifteen years. The bottleneck was never typing. It was the implicit stuff, and agents do not know it unless someone tells them.

So the developer's job did not get smaller. It concentrated in three places:

1. Understand the problem well enough that the implicit becomes explicit.
2. Design the architecture, and write it down, so the agent works inside agreed guard-rails.
3. Review the result critically enough that an engineer can own every line.

The production of code in between is where agents genuinely help. The rest of this post takes the three verbs in order.

Understand: agents amplify whatever you give them

Google's DORA team summarised their 2025 research on AI-assisted development (opens in new tab) in one sentence: AI's primary role is as an amplifier, magnifying an organisation's existing strengths and weaknesses. The finding we find most important sits in the DORA AI Capabilities Model (opens in new tab): a user-centric focus amplifies AI's positive effect on team performance, and without that focus, AI adoption can have a negative impact. In DORA's words, teams may just be moving quickly in the wrong direction.

That matches how agents fail in practice. They rarely fail by producing code that does not compile. They fail by producing confident, well-structured, well-tested code that solves a slightly different problem. The Stack Overflow Developer Survey 2025 (opens in new tab) found that the top frustration with AI tools, named by 66% of respondents, is "solutions that are almost right, but not quite". Almost right is the most expensive kind of wrong, because it passes a casual look.

Picture a wholesaler that asks for "automatic payment reminders". An agent given that sentence will produce a clean scheduler, an email template and a test suite, all in an afternoon. What it will not know: that key accounts on framework contracts must never receive an automated reminder, that early-payment discounts (Skonto) change the due date, that partial payments are common, and that the dunning stages need to match what accounting already does by hand. None of that is in the ticket. All of it is in the heads of the people who do the work today.

A developer writing that by hand would have hit these questions halfway through and walked over to ask. An agent fills the gap with a plausible default and keeps going. Fast generation removes the friction that used to surface misunderstandings.

This is why we think understanding the problem is now the highest-leverage engineering activity, not a pre-sales formality. Concretely, a good rule is: before any implementation is generated, the engineer should be able to state the edge cases, the exceptions and the "this must never happen" rules in the client's own vocabulary. If the engineer cannot, the agent certainly cannot, and the code that comes back will encode a guess.

Design: written decisions are the agent's guard-rails

Our first hard rule: architecture decisions are written down and agreed with the client before agents generate implementation code. Data model, module boundaries, integration points, where state lives, how errors surface, which dependencies we accept. Short documents, in plain language, signed off by the people who will live with the system.

There are two reasons, and only one of them is about the agent.

The obvious reason: a written decision is context the agent can use. DORA lists "AI-accessible internal data" among its seven capabilities. An agent that can read "invoices are immutable once issued; corrections are new documents" will make fewer wrong choices than one that has to infer it from the codebase.

The less obvious reason matters more: the written decision is the yardstick for review. Agents do not reliably follow instructions, however carefully written. Birgitta Böckeler's hands-on evaluation of spec-driven development tools (opens in new tab) on martinfowler.com reports exactly this: she frequently saw the agent ultimately not follow all the instructions, including one case where it ignored notes describing existing classes and duplicated them. So a spec is not a guarantee. It is the thing you hold the output against. Without a written decision, a reviewer can only ask "does this look reasonable?". With one, the reviewer can ask "does this do what we agreed?", which is a far sharper question.

How much to write

Böckeler's piece also names the failure on the other side: a small bug fix that a tool turned into four user stories with sixteen acceptance criteria, a sledgehammer for a nut, and piles of repetitive markdown that were harder to review than the code itself. We agree with her that over-specification is real.

Our view is that decisions should be written at the level where a wrong choice would be expensive to undo: data model, boundaries, integrations, security and data protection. We do not write a spec per button. Böckeler distinguishes "spec-first" (the spec is thrown away after the feature) from "spec-anchored" (the spec is kept and evolves with the code). For architecture, we think the second is the only defensible option. A Mittelstand system will outlive the engineers who built it, and the next person needs to know why the boundaries are where they are, not only where they are.

And because the client signs off on these decisions, the client is part of the guard-rail. That is deliberate. If an agent proposes a change that crosses an agreed boundary, that is not a code review comment. It is a conversation with the client.

Review: our rules before anything is merged

This is the part the how we work page points to, so we will be specific. Our core rule is simple to state:

An engineer reviews, understands and owns every line before it is merged. Not the agent, not "the team", not a bot. A named engineer, who can explain any line in the change if asked.

Addy Osmani made the same argument in Own the Outer Loop (opens in new tab): agents can run the inner loop of investigating, implementing and verifying, but humans own the boundaries, quality gates and the final verdict, because someone must be able to explain what changed, why it was safe and what happens if they are wrong. We agree completely. Accountability that cannot be delegated to a tool has to land on a person, and that person needs to understand what they are signing.

Four things are standard on every change, not optional extras:

• Tests. Generated code arrives without a track record. Tests are how we give it one, and how the next engineer finds out what the code is supposed to do.
• TypeScript with strict typing. Types are a specification the compiler checks on every build. With agents producing code at volume, a constraint the machine enforces for free is the cheapest review we will ever get.
• Quality gates. Type-checking, linting, the test suite and a production build run automatically on every change, and a change that fails any of them is not merged. The gates do not replace the reviewer. They make sure the reviewer's attention goes to the questions a machine cannot answer.
• Documentation that moves with the code. When a change alters behaviour, an interface, configuration or an architecture decision, the documentation that describes it is updated in the same change, not "later". Out-of-date documentation misleads the next engineer, and it misleads the next agent too, because agents read it as context.

What we reject, and why

These are our standing reasons to send a change back. None of them is about whether code was written by a person or an agent; they apply to both. Agents just trigger them more often.

• Code nobody on the team can explain. If the reviewer cannot explain it, nobody can maintain it. This is the ownership rule applied literally. Our post on comprehension debt covers why this bill arrives in year two rather than week one.
• Changes that drift from the agreed architecture. A new database table nobody discussed, logic in the wrong layer, a shortcut through a module boundary. The code may work. It still breaks an agreement with the client.
• Code without tests. No exceptions for "it's just glue code". Glue code is where integrations fail.
• Oversized diffs that cannot be reviewed. A two-thousand-line change does not get reviewed; it gets skimmed. DORA's capabilities model singles out working in small batches as especially powerful with AI. We split large changes before review, not after.
• Duplicated helpers instead of extending existing ones. Agents love writing a fresh formatDate rather than finding the one that exists. Each duplicate is a future inconsistency.
• Silently swallowed errors. An empty catch block or a fallback that hides a failure is how a system ends up wrong without anyone noticing. Errors are handled deliberately or surfaced.
• Changes that leave the documentation behind. If the code now does something the documentation does not describe, the documentation is wrong, and wrong documentation is worse than none. The change goes back until both agree.
• New dependencies nobody chose. Every package is a supply-chain, licence and maintenance decision. An agent adding one because it was convenient is not a decision.

For the incident data behind this discipline (the Cortex 2026 benchmark) and more review techniques, see our earlier post on review discipline.

The honest trade-offs

This way of working is slower per pull request than accepting what the agent produces. We are not going to pretend otherwise.

The industry data shows where the cost lands. In the LinearB dataset, pull requests produced by AI agents waited 17.6 hours for review at the 75th percentile, against 3.4 hours for unassisted work. Review is now the constraint, and a shop that reviews seriously will feel that constraint more, not less. The Stack Overflow survey (opens in new tab) adds the other side: 45.2% of developers said debugging AI-generated code is more time-consuming. The time saved in generation is partly paid back in verification. Teams that skip verification are not faster; they have deferred the payment.

At the same time, we think it would be dishonest to treat agents as a fad. METR's 2026 update is messy, but the reason it is messy is telling: developers increasingly refused to work without AI. Our own view is that agents genuinely help with:

• boilerplate and scaffolding inside a structure that already exists,
• generating test cases, especially the tedious edge cases a human skips,
• mechanical refactors and migrations with clear before/after rules,
• reading unfamiliar code and summarising it as a starting point for a human,
• quickly sketching two or three alternative approaches so the engineer can choose.

Notice the pattern. Agents are strongest where the decision has already been made and the work is execution. They are weakest exactly where the three verbs live.

What this costs a client is time up front: workshops to surface the implicit rules, decisions to read and sign off, and review time that shows up in the plan. What it buys is a codebase where every line has an owner who can explain it, the architecture is documented because it had to be agreed, the documentation still matches the code, and the next team, ours or anyone else's, can pick it up. For software that runs a business for a decade, we think that is the only reasonable trade.

What to ask any software vendor

If you are buying custom software in 2026, assume the vendor uses AI agents. That is not the question. These are:

1. Who owns each line? Ask whether a named engineer reviews and can explain every change before it is merged. Ask what happens when they can't.
2. Where are the architecture decisions written down? Ask to see them. Ask whether you sign them off before implementation starts.
3. How do you surface the rules nobody wrote down? Listen for workshops, edge-case lists and domain questions, not just "send us the requirements".
4. Are tests, strict typing and automated quality gates standard, or extras? If they're priced as an add-on, they'll be dropped under deadline pressure.
5. Is documentation updated with every change, or written once at handover? Documentation written at the end describes the system as someone remembers it, not as it is.
6. What do you reject? A vendor with no answer here has no review standard; they have a merge button.
7. How big is a typical change? Small, reviewable increments are a sign of discipline. Giant weekly drops are a sign nobody read them.
8. Could another team take over this code? The honest answer depends entirely on the previous seven.

None of these questions mention AI. That is the point. The tools changed. What makes software trustworthy did not: someone understood the problem, someone designed it on purpose, and someone is willing to put their name on the result.

Further reading

• METR, Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity (opens in new tab)
• METR, We are Changing our Developer Productivity Experiment Design (February 2026 update) (opens in new tab)
• DORA, 2025 State of AI-assisted Software Development (opens in new tab)
• Google Cloud, Introducing DORA's inaugural AI Capabilities Model (opens in new tab)
• Stack Overflow, 2025 Developer Survey: AI (opens in new tab)
• LinearB, AI in software development: what the 2026 data shows (opens in new tab)
• Birgitta Böckeler, Understanding Spec-Driven-Development: Kiro, spec-kit, and Tessl (opens in new tab)
• Addy Osmani, Own the Outer Loop (opens in new tab)
• Our posts: The 23.5% Problem and Comprehension Debt Is the Real AI Tax