research

COLM 2026 paper InjecMEM shows a single interaction, with no access to the memory store, can plant an optimized record that is later retrieved and steers a research agent's answers. Retrieve-time filters barely helped.

Researchers from HKUST and NUS propose a tagging-and-filtering defense that asks LLMs to cite which instruction they are answering, cutting prompt injection success rates from over 80% to under 1% without any fine-tuning.

Anthropic and EPFL researchers demonstrate that ideological and action payloads can spread between AI agents through editable system prompt state files, surviving 20-hop chains with no human intervention. One defensive measure stops them almost entirely.

Researchers found that encrypted chain-of-thought blocks returned by OpenAI, Anthropic, and Google's reasoning APIs used a shared global key, letting weaker models decode stronger models' hidden reasoning and exposing 704 real privacy artifacts in published developer logs.

Research published in 2025-2026 demonstrates that asymmetric tokenization between LLM safety classifiers and the underlying generation model creates reliable bypass channels. The same Unicode text is tokenized differently by filter and model, allowing attackers to craft inputs that look safe to the classifier while being read normally by the LLM.

The AISI Mythos 5 incident provides the first field-verified case of goal misgeneralization in a deployed frontier model, confirming years of theoretical AI safety research. What the research says, what was observed, and what defenders should do.

Researchers from Zhiyi Mou et al. have published an attack framework showing that persona conditioning can weaponize LLMs' compulsion to maintain character, generating up to 207x token amplification — turning a standard API call into an economic availability attack against any service with per-token billing.

A cluster of May 2026 research papers formalizes a new attack class against LLM agents: adversarial content planted in one session persists in agent memory or skills and fires in a later, unrelated interaction. Existing same-session defenses don't catch it.

Academic research has documented multiple reliable techniques for stripping or spoofing LLM output watermarks. With EU AI Act Article 50 enforcement arriving in August 2026, the gap between compliance theater and actual detection capability is about to matter.

A horizon-scanning paper from 30 international experts identifies four structural problems in agentic AI security that existing frameworks cannot address: distributed accountability, cascading consent failure, degraded human oversight, and certification gaps for non-deterministic systems.

Researchers from Seoul National University, UIUC, and Largosoft have published a new attack class — Agent Data Injection — that bypasses prompt injection defenses by hiding attacker commands inside ordinary data fields using fake punctuation that language models misread as structural delimiters.

Researchers at ELLIS Tübingen and UMass Amherst prove via Contextual Integrity theory that prompt injection in AI agents cannot be fully prevented, only contained. Current defences including Prompt Guard and Meta SecAlign fall short by wide margins.