Does a prompt injection filter stop an AI agent from being hijacked?
Unity Horizon5 min read
Not when the attacker adapts to it. Twelve defences that published near-zero attack success rates were broken above 90%, and the filters cost the most utility.
Not when the attacker adapts to it. A fourteen-author team took twelve recent defences, tuned an attack to each one, and pushed most of them above 90% attack success. Most of those defences had published near-zero rates.
The mechanism is that an attack success rate measures the attack set and not the defence. A filter built against the injections that existed when it shipped scores well against those injections. The attacker writes new ones. That is the whole distance between a 1% figure on a vendor slide and a 95% figure in a paper, and it is why a published rate with no attacker named beside it cannot be read at all.
Why does a defence reporting 1% break at 90%?
Because the 1% was measured against strings written before the defence existed. The Attacker Moves Second, posted 10 October 2025, takes twelve defences spanning prompting, training, filtering and secret-knowledge designs, and attacks each one with gradient descent, reinforcement learning, random search and human red-teaming.
Spotlighting and prompt sandwiching report static attack success as low as 1%. Search took both above 95%. MetaSecAlign reports 2% on static AgentDojo and gave up 96%. Circuit Breakers went to 100% under reinforcement learning. The best result for the defenders anywhere in that paper was PIGuard at 71%.
Then there is the price of the break. The authors give their search attack 800 queries per target and report that "most of the times, the search attack does not use all 800 queries and plateaus after 100 - 200 queries". A few hundred queries is an afternoon. Earlier work, Adaptive Attacks Break Defenses Against Indirect Prompt Injection, bypassed eight defences above 50% and moved an LLM-based detector on AgentDojo from 0% to 31.25%.
Which defences held up, and what did they cost?
The class matters more than the product. A filter decides whether the incoming text looks like an attack, and a system-level defence decides what the agent may do once it has read it.
AutoDojo, posted 13 June 2026 and revised 28 September 2026, generates injections per agent and per defence in a strict black-box setting, seeing only whether a candidate worked. Ten defences, five models, six task suites. On AgentDojo with GPT-4o-mini:
| Defence | Class | AgentDojo static attack | AutoDojo adaptive attack |
|---|---|---|---|
| none | - | 58.6% | 52.4% |
| PIGuard | filter | 0.0% | 28.0% |
| DataFilter | filter | 12.6% | 33.4% |
| ProtectAI | filter | 7.2% | 15.4% |
| DRIFT | system-level | 2.6% | 6.4% |
| Progent | system-level | absent | 7.7% |
Read the defended rows against 52.4% rather than 58.6%. AutoDojo's undefended rate came in below the static one because its generator was required to produce new injections instead of falling back to the benchmark's seeds. Progent's static row is absent from that table, and the paper reports 5.1% for the best prior attack against it.
Then the bill. PIGuard took DeepSeek-v4-Pro from 89.5% clean utility down to 54.4%, a loss of 35.1 points, and on the AgentDyn suite it left GPT-4o-mini at 25.0% against 46.7% undefended. Progent and ROPE cost at most around seven points. CaMeL, which extracts the control flow from the user's query so that retrieved data cannot change it, solves 77% of AgentDojo tasks with provable security against 84% for an undefended agent. AutoDojo finds CaMeL both the most effective defence it tested and the most expensive in utility on four of its five models.
The filter that catches the most injections is also the one that refuses the most work.
What makes an agent easy to hijack whatever its filter reports?
How much of the task you left to the page. AutoDojo sorts tasks into fully specified, parameter-open and action-open, and on undefended GPT-4o-mini the attack success rate climbs 43.3%, 63.9%, 87.5% across the three. Under PIGuard the action-open bucket still reaches 54.2%, more than three times its fully specified rate of 17.3%.
Progent, DRIFT and ROPE hold every bucket at or below 11%. A defence that enumerates the permitted actions does not care how loosely the goal was written, because an instruction found on the page has no action to reach for.
A page cannot hijack a decision you already made.
What should you ask before buying one?
Four questions, and three of them are answered by whether the vendor published an evaluation at all.
| Question | What a thin answer looks like |
|---|---|
| Who wrote the attacks, and when | A fixed corpus assembled before this defence shipped |
| How many attempts per environment | One, or unstated |
| What did clean utility cost on the same suite | A security number with no utility number next to it |
| What can the agent still do when the filter is wrong | A permission list nobody has written down |
Anthropic's prompt injection defences post of 24 November 2025 answers the first two. It runs an adaptive best-of-N attacker at 100 attempts per environment, reports 2,200 to 3,200 attempts behind each figure, and puts Claude Sonnet 4.5 at 13.8% with the model alone, 10.8% with the older safeguards and 1.9% with the improved ones. Those figures are readable. The fourth question is the one we took apart in How do you give an AI agent a login without giving it the password?, where the fix leaves the injection rate where it found it and shrinks the prize instead.
Who does this suit, and who pays for it?
The filter is rarely the weak part. The weak part is an agent with the reach of a browser and the caution of a cron job.
A static benchmark is the one place a detection number looks finished, so it suits whoever is selling one. The cost lands on the company whose security review accepted that number and then shipped an agent holding a full toolbox and a one-line goal. A team whose agent has four named actions and one form to fill pays almost nothing here, and gets the system-level result without buying a system-level product.
We built OFFSET, which files declarations directly to the ΑΑΔΕ portal. It fills a declaration and submits it, and that list does not grow between runs. We wrote up the pattern underneath it in How do you automate a government portal that has no API?. Narrowness is doing more work in that system than a filter would.
The weak point in our own table is the attacker. The Attacker Moves Second pushed PIGuard to 71% with a dedicated search. AutoDojo's black-box generator reached 28% on the same defence with GPT-4o-mini. One defence, two attack budgets, two numbers we would have to publish differently. So treat the table as our current reading rather than a ranking, and expect the next paper with a better optimiser to move every row in it. Figures checked by Unity Horizon on 30 October 2026.
If nobody has written down which actions your agent is permitted to take, that is the part we would scope first. Which actions can it take today that nobody named in the prompt?
Common questions
Do prompt injection filters work?
Against the attacks they were built on, yes. Against an attacker who tunes injections to the filter itself, mostly not. A 2025 evaluation of twelve defences reported attack success above 90% for most of them, and most of those defences had originally published near-zero rates.
What is an adaptive prompt injection attack?
An attack written after the defence and optimised against it. The attacker submits a candidate injection, keeps whatever got closer to working, and repeats. In published results a search attack usually finds a break within 100 to 200 queries, against a budget of 800.
Which prompt injection defences hold up best?
The system-level ones, which constrain what the agent is permitted to do rather than inspect what it reads. On the AutoDojo runs over AgentDojo, Progent and DRIFT stayed under 10% attack success on GPT-4o-mini while filter-based defences reached 15% to 33%. CaMeL solves 77% of AgentDojo tasks with provable security, against 84% for an undefended agent.
Does leaving an AI agent's task vague make prompt injection more likely?
Yes, and the effect is larger than most defences are worth. On undefended GPT-4o-mini, attack success was 43.3% when the task named everything, 63.9% when one parameter was left open, and 87.5% when the action itself was left to external content.