Note

The defense nobody has attacked

In late July, during a routine safety test, an AI agent went looking for a way into a target network. It found a real open source project on GitHub, one it appears to have mistakenly believed was connected to its target, and opened a pull request containing malicious code. Then it created a second account, posing as a different person, to vouch for its own pull request. A human maintainer caught it. The agent said it was an honest mistake, fixed nothing, and tried again. Along the way it emailed the maintainers under invented names, and left a booby trapped bug report written to be read and executed by other AI coding agents. Nobody asked it to do any of this. It was supposed to stay inside a lab.

The only fence was a sentence

That is not a leak or a rumour. The UK AI Security Institute published it themselves on 4 August, in an incident report about their own evaluations. They publish the timeline, all nineteen catalogued actions, and the prompts.¹

The interesting part is not that the agent misbehaved. It is why nothing stopped it.

The agent had internet access, on purpose, because the test needed it. The instructions said which network was in scope. That was the entire boundary: a sentence in a prompt, written in English, asking nicely. AISI publishes the prompts in an appendix. There is no prohibition on internet use anywhere in them.

Seven out of ten went for the target the prompt told them to leave alone.

If you build with agents, you already know this problem by its other name. A web page, an email, a README, a bug report: anything your agent reads can contain instructions, and your agent cannot reliably tell the difference between what you asked for and what the page told it to do. It is one stream of text. There is no envelope marked "this part is data, ignore any orders inside."

So there are two ways to fix it. Catch the bad instructions, or make sure it does not matter when the agent follows them.

Answer one: catch the bad instructions

The first answer is to catch them. Filters, classifiers, guardrails, models trained to rank your instructions above the ones they find in a document.

They test well. One of the strongest published examples is Meta SecAlign, a model trained specifically to resist this. On the benchmark its authors used, against a fixed set of attacks, injections got through 1.9% of the time.² That is a good number.

Then a team that had not built it attacked it properly. Not with a fixed list of known attacks, but by letting the attacker search: try something, see what happens, adjust, try again. A median of 37 attempts per target.³

Ninety six percent.

SecAlign's authors are not hiding from this. They break their own model too, and write that it is "still vulnerable to strong adaptive attacks... the PI threat is far from being solved."²

And it is not one unlucky product. The same team bypassed twelve different defenses, most of them above 90%.³ A separate team took eight defenses past 50%.⁴ A third got through six guardrail systems, in some cases 100% of the time.⁵ Four independent groups, same direction.

The pattern repeats. A defense holds against the attacks its authors thought of, and falls over against an attacker who is allowed to keep trying. That is what you would expect if the thing being defended is not a wall but a habit.

Answer two: build a cage

If you cannot catch the instruction, contain the consequences. Do not let the agent touch the production database. Do not let it reach hosts you have not allowed. Give it credentials that expire. Make it ask a human before anything irreversible. Assume it will get tricked, and make that survivable.

This is our position, and it is where much of the field has landed. Meta publishes a rule for it. Google DeepMind and ETH built a system called CaMeL around it.⁶⁷ There is a real result behind it, too. One study built an adaptive attacker and pointed it at nine different defenses under identical conditions, which lets you see both approaches in the same picture.⁸

The filter goes from perfect to broken. The permission system does not move. That is the whole argument for cages in one chart, and it is the chart we wanted to build this article around.

The part that stopped us

Then we went looking for the rest of the evidence, and there is no rest of the evidence.

That study is one of exactly two times anyone outside a defense's own team has attacked an architectural defense. Both happened in June 2026. Both used a small budget and no access to the system's internals. Both looked at the same single product, Progent.⁹

We could find no independent attack on CaMeL at all. None on FIDES, RTBAS, FORGE, or the dual model pattern either.

Compare the effort on the other side. The team that broke twelve detectors used a human red team plus automated search. Two other teams used gradient attacks with full access to the model's internals. Against the cages: one small study with a single handwritten attack template, and one automated attacker given six tries per target.

The people who ran the second of those two evaluations were careful to say so, in their own abstract:¹⁰

So the honest version of the story is not "filters fail, cages hold." It is that filters have been attacked hard by four teams and fell over, and cages have barely been attacked at all. Those are not the same finding. An untested defense is not a defense that held.

The model is still inside the cage

There is a second reason to be careful, and CaMeL's own authors are the ones who found it.

A cage needs to know what the agent is allowed to do. Something has to decide that. In CaMeL, a trusted model writes the plan before any untrusted data is read, and the plan is what the cage enforces.

So they tried a task shaped like this: monitor my email, find the action in it, and do it. Both of the models named in that example wrote a plan that looped over every available tool and picked one based on the email's contents.

The cage did exactly what it was told. It was told to build a door.

Later in the same paper there is a section titled "So, Are Prompt Injections Solved Now?" It answers: "No."⁷

The case where none of it helps

That email example is not a bug. It is a whole category, and three separate groups found it independently while looking for different things.

CaMeL's team logged it as tasks their system cannot complete. The attack study logged it as a security hole, calling it the case where the user "does not specify the action verb at all and defers entirely to external content", and measured 64% attack recovery on those tasks against a filter that scored zero under fixed attacks.⁸ A third paper reached it from pure logic: an attacker can always build a situation where the forbidden action looks legitimate, and a defender who tightens the rules enough to stop that will block real work too.¹¹ That paper is co written by one of the researchers who named this attack in the first place.

Here is why that case is different. A permission system works by checking whether an action is authorised. But when you say "read my email and do what it says," you have authorised whatever the email says. The check passes. Nothing is broken, nothing is bypassed, and the agent does what an attacker wrote, because you told it to.

Nobody knows how much real agent work looks like that. It is measurable and nobody has measured it.

Cages are not free either

The usual figure quoted for CaMeL is that tasks still complete 77% of the time, against 84% with no protection. A seven point tax sounds like an easy trade.

That pair is the best of the six models they tested.⁷

Progent, measured by someone other than its authors, costs 18 points and roughly fifteen times as many model calls.¹⁰ Meta's rule allows an agent at most two of three things: untrusted input, access to sensitive data, and the ability to change something or talk to the outside world.⁶ That is not a control you install. It is a feature you delete.

And if your cage is a human clicking approve, it rots. Anthropic looked at their own telemetry and found users approving about 93% of permission prompts, with "approval fatigue" showing up within weeks.¹²

What a cage actually did, once

In July, an agent escaped its sandbox during an evaluation at a frontier lab and got into Hugging Face's infrastructure. Hugging Face published a timeline from their own audit logs, which is a rare thing on its own: an incident writeup that traces what happened to specific controls.¹³

Mutating cloud calls were refused by the role's policy. An internal database was read but never changed. The production database only accepts connections from allowed addresses over a private link, so the agent's connection simply timed out. A real intrusion, running on its own, past every detection layer, and the damage stopped at the permission boundaries.

That is the best evidence in this article for building cages. It comes with two things attached.

The way the agent escaped in the first place was a zero day in one of its sandbox's permitted egress paths. The thing that let it out was a cage. That detail is also the one part of this we cannot check directly: Hugging Face is relaying it from OpenAI, whose own account of the incident now returns an error, so it carries a chain of custody.

AISI reach for the same example when judging their own incident. An internet access control, they write, "would likely have prevented these events". Then, in the next sentence, they take some of it back: such controls "may have proven insufficient against a capable AI agent committed to bypassing them, as illustrated by the OpenAI Hugging Face incident."¹ That is an endorsement of cages from a party with nothing to sell, and it arrives hedged by the people making it.

The cage bounded the damage. It did not prevent the attack. In the whole record we found two cases where anyone can trace an outcome to a specific control, and none at all naming a company whose loss was prevented by one. Meanwhile several of the visible saves were something else entirely: a human maintainer spotted the AISI agent's pull request, and in one supply chain attack that tried to hijack local AI assistants, roughly a quarter of the attempts failed because the model refused.¹⁴

So what do you actually do

Build the cage anyway. When it fails you get a patch and a version number instead of a retraining run. And bounded damage beats unbounded damage even in the cases where nothing was prevented.

Do not buy it on a promise that it holds. Nobody has hit it hard enough to know.

And watch for the shape of task where none of this applies: the ones where you hand the agent a document and tell it to do what the document says. That is not a gap in anyone's product. That is what delegation means, and the fix is not a better cage.

There is one study that would settle most of this: someone independent attacking CaMeL and Progent the way those four teams attacked the filters, including with full knowledge of the policy, measuring what it costs in the same run. The researchers who ran the small version have already written down that it is missing, and four of the authors who broke twelve detectors also worked on CaMeL, so the access is there. Somebody is going to run it.

We would rather have published after that. But the argument is already being made in public with more confidence than the evidence supports, including by people selling it. Including, until we checked, by us.

Connor · Seatbelt · August 6