Agent Data Injection: AI Agents Misled by Fake Structural Delimiters

The core assumption behind most prompt injection defenses is that attacker instructions look like instructions. They appear in text fields, use command-like phrasing, and stand out from the data an AI agent was sent to process. Security filters scan for instruction-shaped payloads.

Agent Data Injection (ADI) breaks that assumption entirely. Researchers from Seoul National University, the University of Illinois Urbana-Champaign, and Largosoft published the technique this week: attackers embed commands not in instruction-shaped text, but inside the data fields that agents trust. Product names. Sender names. Button labels. GitHub commit messages. The payload hides inside content that no one would look at twice.

The Delimiter Problem

The attack exploits how large language models handle structure. When an agent processes a webpage, a database record, or an API response, the LLM sees a mix of structural markers (delimiters) and content. Delimiters tell the model where instructions end and data begins. The boundary is the thing that’s supposed to keep attacker-controlled content from becoming agent commands.

ADI attacks insert fake delimiters into data fields. The technique — which the paper terms “probabilistic delimiter injection” — uses characters that the LLM’s tokeniser and parsing layer interpret differently. Escaped quotes, curly quotes, dollar signs, and other edge-case punctuation slip past the outer parsing layer (which sees ordinary data) but land inside the LLM’s attention as structural tokens. The model reads the fake delimiter as a genuine boundary, misidentifies what follows as a trusted instruction, and executes it.

The key insight is that this is not a reasoning failure. It’s a parsing ambiguity that exists at the interface between the agent’s structured input handling and the LLM’s probabilistic tokenisation. You can’t patch it out of one without creating problems in the other.

What the Attacks Actually Do

The research team, led by Woohyuk Choi and Prof. Byoungyoung Lee, demonstrated three attack scenarios that illustrate how this plays out in real deployments.

In the first scenario, ADI payloads planted inside product listing descriptions caused a web shopping agent to add an attacker-specified product to the user’s cart and proceed to checkout. The agent was browsing normally, reading product reviews as data. It ended up purchasing something the user never intended to buy.

The second scenario is more alarming for software teams. ADI payloads embedded in GitHub issue comments, formatted to look like legitimate maintainer messages, caused PR-review agents to approve malicious code merges. The agent was running automated code review. It saw what appeared to be a senior maintainer’s sign-off, cross-referenced the fake comment against the review criteria, and approved. The merge went through.

Third: coding assistants were manipulated into running attacker commands by ADI payloads in file metadata and function docstrings. The agent was refactoring code. It read what it understood to be configuration data and executed attacker-controlled shell commands.

Success Rates Across Six Models

The team tested GPT-5.2, GPT-5-mini, Claude Opus 4.5, Claude Sonnet 4.5, Gemini 3 Pro, and Gemini 3 Flash. No model was immune. Against structured data inputs, ADI succeeded 31-43% of the time across all models tested. On certain webpage data scenarios, success rates climbed to 100%.

The variation reflects differences in how models handle tokenisation edge cases and how agent frameworks structure the context window. More capable models — the ones most often deployed in high-stakes agentic tasks — were not consistently better at resisting the attack. In some structured-data scenarios, larger models were more susceptible because their more sophisticated parsing made them more likely to act on the fake structural signal.

Why Current Defenses Miss It

Most deployed prompt injection mitigations operate on a simple principle: flag inputs that look like instructions and block or neutralise them before they reach the agent. Input sanitisation, system prompt hardening, output filters checking for command-like phrasing.

ADI payloads do not look like instructions at the input stage. They are product names and sender fields and commit messages. The payload only becomes an instruction after the LLM processes the fake delimiter and reconstructs the attacker’s intent in the context window. By that point, the instruction is indistinguishable from trusted agent context.

The paper describes this as a class of attacks that is “fundamentally resistant to current instruction-data separation approaches.” That’s not a claim that defenses are useless. It’s a specific technical observation: the separation techniques assume you can identify instruction-shaped content at the boundary. ADI attacks the assumption, not the boundary.

What Comes Next

This research joins a growing body of work on second-order AI agent attacks, where the attacker’s leverage comes not from compromising an AI system directly but from poisoning the external data that agents consume. Embedding inversion, RAG poisoning, memory injection (MemGhost, covered here in July), and now ADI all share the same attack surface: the unverified external world that agents treat as input.

The practical implication for teams running agentic workflows is that the trust model needs to shift. External data, including content from sources that feel trustworthy like a company’s own product database or a monitored GitHub repository, should be treated as potentially adversarial. High-consequence agent actions — purchases, code merges, file executions, external API calls — need human confirmation checkpoints that ADI payloads cannot override.

The full paper is expected to be posted to arXiv in the coming days. The research team disclosed to Anthropic, OpenAI, and Google ahead of publication.

References

• The Hacker News — New Agent Data Injection Attack Can Make AI Agents Misclick or Run Attacker Commands