Research

The check that never runs

AI coding agents do not produce random security bugs. They produce the same one, over and over: a protection that exists but does not propagate. The page is guarded; the action behind it is not. Auth gets built in the first prompt; the export route added in the fortieth forgets it. The code works, which is exactly why it ships.

The caveat, first

We did not run a controlled study. This is a synthesis of what practitioners keep reporting and what our own scans keep flagging, with primary sources attached at the bottom. The claim is not "here is a new vulnerability." It is that AI-built apps share a signature, a recognizable shape to how they fail, and that the shape is a consequence of how the code is written, prompt by prompt.

Working code is the disguise

The most quoted line in this space is worth repeating: a server that returns every user's data and trusts the browser to filter it "has not implemented authorization, but rather a very polite suggestion." The app looks finished. It demos perfectly. The list loads, the page is behind a login, the reset email arrives. Nothing errors. Nothing crashes. There is no bad pattern for a linter to match, because each individual line is idiomatic and correct.

What is missing is an invariant, a rule that has to hold across more than one place at once. And invariants are exactly what a model writing one prompt at a time does not carry.

Four instances of the same failure

The page is protected. The action is not. In a Next.js app the page calls verifySession() at render, but the server action it posts to has none. As one write-up puts it: "The page check runs at render time. The action check never runs." The guard the developer sees is real; the guard that matters, on the mutation, reachable by a direct POST, was never written.

Auth is built in prompt 1. Prompt 47 exports the database. A documented vibe-coding session builds a proper auth flow early, then later is asked to "add a feature to export user data as CSV." The export handler ships without inheriting any of the auth context established forty prompts earlier, an unauthenticated bulk dump. The model was locally correct twice and globally wrong once.

The reset token matches. It never expires. AI-generated password-reset flows routinely store plain tokens, skip expiry, and stay valid after use. The token comparison works, so the feature is "done", but account takeover is one leaked database row away. One practitioner audit found reset tokens with no time bound in four of ten vibe-coded codebases.

The list returns everything. The UI hides the rest. A normal-looking list endpoint runs findMany() with no session-scoped filter and returns every row; the frontend renders a filtered subset. The full dataset, names, emails, private messages, is already in the browser before the user clicks anything. One reviewer found this shape in three of four real apps they looked at.

Different surfaces, one root: a protection that stops at the boundary of the prompt that created it.

Why it happens, and why review misses it

A model generating code optimizes for the thing it can see succeed: does this feature work. It does not hold the whole application's security invariants in view while writing the next route, and it has no reason to re-derive "everything that touches user data must be scoped to the user" every time it adds an endpoint. Each prompt is a fresh, locally-correct answer. The invariant lives between the answers, where nothing is looking.

That is also why the usual safety nets slide off. A linter matches bad patterns; this is the absence of a good one. A checklist asks "is there auth?" and the answer is truthfully yes, somewhere. A test suite covers the happy path the feature was built for. And the builder, who sees a working app, has no signal that anything is wrong. The failure is invisible precisely to everyone positioned to catch it.

What actually catches it

The only thing that reliably finds a missing invariant is a check that reads across surfaces at once and asks whether the rule holds everywhere, not "does this line look bad" but "is every route that returns user data scoped, is every mutation guarded, does every token expire." And it has to run at the moment the code would reach someone else: the ship. That is the entire job Seatbelt does, a Ship Read across six surfaces before you share the link, looking for the guard that stops one prompt short.

What we do not claim

Some of these shapes we read statically with high confidence; others we surface as soft prompts or assist questions, because proving server-side scope from source alone is not always possible. We flag the pattern and tell you where to look; we do not claim to prove exploitability. The point of this note is narrower and, we think, load-bearing: the way AI writes code, locally correct, one prompt at a time, is the same reason its output leaks, and it is why an independent read across the whole change, at the ship moment, is not optional.

Connor · Seatbelt · July 21