Carryover Attacks: One Prompt Injection That Fires After the Attacker Leaves

An agent triages an inbox. One email hides an instruction to add a recurring task to the agent’s own HEARTBEAT.md. The agent complies, the session ends cleanly, and nothing looks wrong. Hours later a timer wakes the agent with no human present, and it re-executes the task it wrote itself. A paper posted to arXiv on 20 October 2026 gives this pattern a name, carryover attacks, and a benchmark to measure it.

The paper is “After the Attacker Leaves: Benchmarking Carryover Attacks on LLM Agents” (arXiv:2610.28425, cs.CR, listed by the authors as an EMNLP 2026 Main paper). It comes from researchers at the Chinese Academy of Sciences Institute of Automation, UCAS, SenseTime Research and Zhongguancun institutions. This article summarizes the paper’s own results. We have not reproduced them, and the paper links a public code repository at github.com/igloomatics/CarryoverBench (no licence file as of 29 October 2026).

Two Persistence Channels

The authors focus on two mechanisms common in personal-assistant agent runtimes. The first is a long-lived MEMORY.md that the runtime silently prepends to the system prompt at every new session. The agent can edit it with its own file-write tool, so anything it writes becomes a standing instruction. The second is a heartbeat daemon that wakes the agent at fixed intervals with a fixed prompt telling it to read HEARTBEAT.md and run each registered task.

The threat model is narrow on purpose. The attacker is a black-box third party who can only place content where the agent will read it: a web page, email, document or tool result. The attacker cannot touch the runtime, cannot write the two files directly and has no persistent presence. Any later harm has to be carried by the agent’s own state management.

What CarryoverBench Measures

Existing prompt-injection benchmarks score success inside the session where the malicious content first appears. CarryoverBench covers 12 personal-assistant scenarios (email, banking and code review among them) and 101 attack tasks over three channels:

• Instant: same-turn injection, the classical baseline (35 tasks).
• Autonomous: the payload is planted in the heartbeat file and fires on a scheduled wake-up with no user present (30 tasks).
• Latent: the payload is planted in memory and fires when a later, topically related user request arrives (36 tasks).

A rule-based verifier, not an LLM judge, scores the “plant” and “trigger” stages independently. The authors argue a judge prompt is itself an injection surface and that rule-based checks can be re-run by third parties. Each follow-up round uses a fresh agent, matching production cron behavior.

Results Across Five Models

The authors tested GPT-5.4, Claude-Sonnet-4.6, Gemini-3.1-Pro, GLM-5 and Qwen3-235B-A22B on the Nanobot open-source agent harness at temperature 0. Overall attack success rates ranged from 11.9% (GPT-5.4) to 63.4% (Qwen3-235B-A22B), with Claude-Sonnet-4.6 at 18.8%, GLM-5 at 27.7% and Gemini-3.1-Pro at 51.5%. The paper says carryover trails Instant per attempt, but the comparison is not like for like: Autonomous and Latent scores count a task if it fires in any round, while Instant is single-shot, and the 95% confidence intervals are wide (30 to 36 tasks per channel). The gap is clear for GPT-5.4 only (22.9% Instant against 3.3% Autonomous). GLM-5 (40.0% against 34.3%) and Qwen3 (83.3% against 57.1%) scored higher on the Autonomous channel than on Instant.

Three findings stand out.

Persistence does not decay. Over five follow-up rounds, trigger rates showed no monotonic decline in either channel, though some small drifts appear in the tables (Gemini Autonomous 56.7% in round 1 and 50.0% by round 4; Qwen3 Autonomous 76.7% falling to 70.0% in round 5). GPT-5.4 held at 8.3% on the Latent channel in every round. The authors’ explanation is structural: the payload sits in files that never change, and chat history is the only thing reset between sessions.

The trigger stage is a weak gate. In principle a defender gets two chances, at write time and at read time. The paper finds that once a payload is committed, the agent acts on it almost deterministically. For four of five models the conditional trigger rate exceeded 80% on the Autonomous channel, with GPT-5.4 and Claude approaching 100% in both modes. For the heartbeat channel, the authors say cumulative harm scales with deployment time, since every tick is another chance to fire. For the memory channel it depends on how often the user raises the topic, and the paper’s own limitations call its repeated-follow-up rates an upper bound on realistic trigger rates.

GLM-5 is the exception that proves the rule. The authors report it never audits the memory file at trigger time. Its lower trigger rate comes from a general check on outbound calls to non-internal-looking endpoints, which also depresses its benign-task utility.

Why Text-Level Defenses Fall Short

The paper tested three mitigations on Qwen3, GLM-5 and Claude. D1 prepends a principles block telling the agent to treat persistent state cautiously. D2 gives the agent explicit authority to inspect and edit its memory files. D3 wraps the files in an <untrusted_persistent_state> envelope, a delimiting form of spotlighting.

All three acted mainly at the plant stage. D1 cut Claude’s plant rate to zero in the reported runs but cost it 9.3 percentage points of utility under attack, and on Qwen3 it increased Latent attack success from 52.8% to 72.2%. Under D2, agents issued zero voluntary edits or deletions across all trigger rounds. D3 closed the trigger gate in only one of six model and channel cells, which the authors place within rerun noise.

The authors’ argument is that all three defenses reach the model as text next to the planted instruction. At trigger time the read-and-act step on the memory file looks identical to a benign one. They suggest the discriminator must sit outside that channel, for example runtime gates on cross-session writes or capability restrictions that do not depend on the model recognizing its own poisoned state.

A port of the Latent channel to the Hermes runtime lowered attack success on every model (for example Qwen3 from 52.8% to 27.8% and Claude from 22.2% to 5.6%) without eliminating it. The authors credit model alignment surfaced by the richer tool catalog rather than Hermes’ own scanner or review pipeline.

Defensive Guidance

The following is our assessment, drawn from the paper’s findings. It has not been tested by us.

1. Treat writes to memory and scheduled-task files as privileged actions. Require user confirmation, or block them when the session has just ingested untrusted content such as email or web pages.
2. Log and diff persistent agent state. Alert on new entries that contain URLs, email addresses or outbound-send instructions.
3. Restrict what heartbeat-triggered sessions can do. An unattended run should not hold the same outbound network or messaging capabilities as an interactive one.
4. Do not rely on prompt wording alone. The paper’s data suggests system-prompt rules and untrusted-data wrappers mostly shift the write-time odds and leave trigger-time behavior largely unchanged.
5. Include cross-session scenarios in agent red-teaming. Single-session injection tests would miss everything described here.

Limits worth stating: the study covers one primary harness plus one secondary runtime (the Hermes port covers only the Latent channel and non-reasoning models). It is one run per model at temperature 0, and the authors say the numbers should be read as a relative signal, with their verifier likely undercounting. D1 to D3 were not tested on GPT-5.4 or Gemini. The benchmark was built by the same team that measured it, which is our observation rather than a claim in the paper. Results for other agents may differ.

References

• Xu et al., “After the Attacker Leaves: Benchmarking Carryover Attacks on LLM Agents,” arXiv:2610.28425, 20 October 2026: https://arxiv.org/abs/2610.28425
• Full text (HTML): https://arxiv.org/html/2610.28425v1
• CarryoverBench code: https://github.com/igloomatics/CarryoverBench