The demo works; security often does not
AI agents finish features fast. Security is a separate job, and most builders skip it at the speed agents ship. The gap is measurable, not theoretical.
What can go wrong
A working demo is not a safe launch. Agents optimize for "does it run?" They skip auth checks on new routes, paste keys into client bundles, and leave storage rules open because the happy path looked fine in chat.
The failure mode is shipping code that passes a smoke test but leaks the moment a stranger loads the URL or opens DevTools.
It happened for real
A CMU study (SusVibes, ICML 2026, arxiv 2512.03262) measured AI coding agents on security tasks: 61% of runs solved the functional goal, but only 10.5% solved it securely. Most code that works in a demo still ships vulnerable.
Process matters too. Engineering teams treating vibe-coded output like any other release catch more before production (MagmaLabs production guide, Jun 2026). The question is not whether AI writes code. It is whether your handoff includes a check the agent did not grade itself.
How to check yours
Seatbelt flags part of this automatically. A Ship Read reads the six surfaces that hurt AI-built launches: login, payments, secret keys, customer data, databases, and risky shortcuts. It does not score "secure vs functional" as a percentage. It finds the structural mistakes agents repeat: keys in the bundle, open rules, unsigned webhooks, debug routes left in the tree.
Ask your agent: "Before I share this link, audit login guards, payment wiring, secret keys in client code, database rules, and any admin or debug routes. List file and line for each issue."
We don't catch this yet: Runtime behavior, dependency CVEs, and penetration-style testing. Tools like Snyk and dedicated DAST own those lanes. Seatbelt is the fast read on the artifact you are about to ship.
Fix direction
Run a Ship Read on the repo, folder, or ZIP you export before handoff. If you only have a deployed URL, paste it at /app for the outside view, then clone the repo for the full read and optional push gate.
Paste into your agent: "Run a security pass on what actually shipped: no keys in client bundles, database rules scoped to the signed-in user, webhooks verified, no debug wipe routes left in the tree. Fix must-fix items before I share the link."