Reading the diff is the review vibe coding lacks

The step vibe coding removes by definition

Vibe coding is not a set of tools. It is a relationship to code you did not write. The term arrived on 2 February 2025 in a post by Andrej Karpathy, recorded in the arXiv paper on vibe coding as programming through conversation, and the part people quote least is the part that matters most: “I ‘Accept All’ always, I don’t read the diffs anymore. When I get error messages I just copy paste them in with no comment, usually that fixes it.” Wikipedia’s definition keeps the same emphasis, describing development that may involve accepting AI-generated code without thorough review, relying on results and follow-up prompts to guide changes.

That is the whole distinction between vibe coding and AI-assisted programming. Same models, same editors, same prompts. In one, the diff is read and understood before it lands. In the other, the diff is never opened, and a working screen stands in for evidence. Karpathy’s own framing put the practice in scope for throwaway weekend projects, which is a sensible place for it. The term escaped that scope almost immediately, which is how a habit designed for disposable code ended up in repositories that carry payroll.

So when people ask whether reading the diff is really necessary, they are asking whether vibe coding, as originally described, is a method they should apply to work they intend to keep. Those are the same question.

What the numbers say happens when nobody looks

The empirical picture has been consistent for four years and has only sharpened. Pearce and colleagues ran one of the first security evaluations of Copilot output and found roughly 40% of generated programs contained vulnerabilities across 1,689 samples, with C worse than Python, as summarised in the systematic analysis of security degradation in iterative AI code generation. Perry and colleagues then compared developers working with and without an assistant and found the assisted group wrote significantly less secure code while rating their own insecure solutions as secure. That second finding is the more uncomfortable one. The defect rate is a tooling problem. The confidence gap is a review problem.

More recent counts point the same way. A January 2025 study put 38% of AI-generated code as containing at least one security flaw, spanning injection, hardcoded secrets and broken authentication. In December 2025, CodeRabbit’s analysis of 470 open-source pull requests found AI co-authored code carried 2.74 times more security vulnerabilities than human-written code, a figure IBM cites alongside a 45% rate of vulnerabilities introduced across development tasks. Georgia Tech’s tracking of CVEs traceable to AI coding tools grew from a handful in early 2025 to 74 by March 2026, catalogued in the Cloud Security Alliance research note on the AI-generated CVE surge.

Two failure classes deserve naming because a diff read catches them in seconds. Secrets: AI-assisted commits expose credentials at 3.2% against 1.5% for human-only commits, roughly double. Dependencies: a USENIX Security 2025 study of 2.23 million code samples found 19.7% referenced at least one package that does not exist, and 43% of those invented names came back on every rerun, which is why attackers now register them and wait. A new line in package.json is one of the cheapest things in a diff to check and one of the most expensive to miss. We cover both patterns in more depth under the risks and limits of vibe coding.

The generation-review asymmetry

Here is the finding that changes the argument. A 2026 formal-verification study, Broken by Default, took code that had been proven vulnerable using the Z3 solver and fed it back to the same model that produced it, asking only for a security review. Of 89 artefacts, 70 were correctly flagged as vulnerable. That is 78.7%, with a 21.3% false-negative rate across the full set.

Read that carefully, because it rules out the comfortable explanation. The models were not failing because they lack security knowledge. They had the knowledge and stated it plainly the moment they were asked to look rather than to build. The authors call it a generation-review asymmetry: a failure of spontaneous application, not of capability. Writing code and checking code are different modes, and the same weights behave measurably differently in each.

Everything practical follows from that. If review were merely a redundant second look at the same reasoning, skipping it would cost little. It is not. It is a distinct pass with a different output distribution, which means adding it recovers most of a class of defects that generation alone will not catch no matter how carefully you word the prompt. Asking for secure code up front helps less than asking, afterwards, what is wrong with this.

Using the model as a reviewer without trusting it as one

The cheapest version of the review step is to make the second pass explicit. When an agent finishes a change, ask it to review that specific diff for security problems, with the diff in front of it rather than as a vague instruction attached to the original prompt. On the evidence above, that catches a good share of what the first pass introduced, and it costs a fraction of the time you spent describing the feature.

It does not replace you, for two reasons. The first is arithmetic: a 21.3% false-negative rate is not a clean bill of health, and the misses cluster in exactly the categories where a missing check looks like nothing at all. The second is more fundamental. A model reviewing its own diff can tell you whether the code is dangerous. It cannot tell you whether the code is what you asked for. Intent lives outside the repository. When an agent renames an endpoint you never mentioned, relaxes a permission check to make a test pass, or quietly widens a query’s scope, no automated reviewer has the specification to notice, because the specification was a sentence you typed and then forgot. That gap is the subject of why ‘Accept All’ is the weakest habit in vibe coding.

So the division of labour is reasonably clear. Automated checks own the mechanical classes: static analysis, secret scanning, dependency verification, tests. Model review owns the vulnerability classes it can recognise on sight. You own intent, authorisation, blast radius and anything where the harm is an absence rather than a behaviour. That last category is where a line-by-line diff read earns its time and nowhere else really does.

Karpathy’s own revision

By early 2026 the person who named the practice was proposing to retire the name. Speaking to The New Stack, Karpathy offered “agentic engineering” instead: agentic because the default is that you are not writing the code directly most of the time, you are orchestrating agents who do and acting as oversight, and engineering because there is a craft and an expertise to it that people can learn and get better at.

The substance of that revision is the reintroduction of the oversight step that vibe coding, by its own definition, left out. Nothing about the speed claim changes. Describing a feature in plain language and getting working code back remains genuinely fast, and the barrier to getting from an idea to something that runs is genuinely lower than it was. The trade-off is that the review you skipped does not disappear; it is deferred, and it comes back with interest in the form of a bug report, an exposed key, or a due diligence questionnaire. If you want the mechanics of the loop this sits inside, how vibe coding works sets out prompting, context and the review pass in order.

Making the review step small enough to actually happen

Most people who abandon diff reading do not abandon it on principle. They abandon it because the diff was 900 lines across 30 files and reading it properly would have taken longer than describing the feature did. The fix is upstream of the review, in how the work is shaped.

Ask for one change at a time, and stop the agent when it starts volunteering adjacent improvements. Commit after each accepted change, so the diff you are reading has one purpose and version control can undo exactly that purpose. Keep tests running, because a suite that passes is evidence in a way that a rendered page is not. Turn on secret scanning and dependency checks in CI, so the two highest-frequency mechanical failures never depend on your attention. Then spend your own reading where the machine cannot help: the permission check, the migration, the destructive operation, the new package nobody has heard of.

Done that way, the review step is minutes rather than an afternoon, and it is the difference between code you have adopted and code you are merely hosting. The practices this site recommends are collected in vibe coding best practices, and the vocabulary, including diff, agent and context window, is defined in the glossary.

---

Written with AI assistance and reviewed under our editorial policy. Vibe coding changes quickly; details reflect sources current on 2026-09-06. Some passages are the author's opinion. Spotted an error? tell us.