KV-Cache Side Channels: LLM Inference Optimizations Leak Tenant Data

The performance optimization that makes commercial LLM deployments economically viable has a quiet privacy problem. Global Key-Value cache sharing, the mechanism that lets inference frameworks avoid recomputing attention over repeated prefixes, creates a timing side channel that lets one tenant’s API calls learn what another tenant wrote. Three research papers published in 2026 have now documented the attack in detail, demonstrated it against production frameworks, and named affected software. Multiple CVEs have been assigned.

The vulnerability class is not new to computer security. Timing side channels have compromised cryptographic implementations for decades, typically by revealing secret-dependent branching in algorithms through observable latency differences. What’s different here is the context: a cloud-hosted AI service marketed for enterprise use, processing confidential business data and user PII, leaking that content to co-tenants through a performance feature no one thought to treat as a security boundary.

How KV-Cache Sharing Works

Transformer inference is computationally expensive, and the attention computation over long contexts is a significant fraction of that cost. The Key-Value cache stores intermediate results of attention computations so that when a new request shares a prefix with a previous one, the framework can skip recomputing those layers entirely and begin generating from the divergence point.

For a system prompt shared across all user sessions, this optimization delivers substantial throughput improvements. For shared prefixes like document contexts in RAG deployments or common instruction blocks, the cache hit rate can be high enough to make the economics of commercial inference workable. vLLM, SGLang, AIBrix, LMDeploy, rtp-llm, and GPTCache all implement global KV-cache sharing across tenants in their default or high-performance configurations.

The security assumption that was not made explicit: that a timing difference observable through the API constitutes a security boundary violation.

The Timing Side Channel

When a cached prefix is matched, TTFT drops because the compute step is skipped entirely. When a cache miss occurs, TTFT reflects full recomputation. The difference is measurable through normal API calls. No privileged access to the inference backend is required. No modifications to the serving stack. The timing signal is the API response time.

The attack proceeds in three phases. First, the attacker constructs a candidate prefix based on what they suspect another tenant may have submitted, such as “My email address is” or “Authorization header: Bearer”. Second, they append a candidate next token to the prefix and measure TTFT. A low TTFT indicates a cache hit, confirming that sequence was previously processed. Third, they iterate across possible next tokens, identifying the hit and moving forward, reconstructing the sensitive sequence token by token.

Research published at NDSS 2026 (“Unveiling and Mitigating Privacy Risks of KV-cache in LLM Inference”) documented this in more granular form, identifying three distinct attack variants: a direct inversion attack that extracts content from cached data, a collision attack with broader applicability, and a semantic-based injection attack that uses semantic properties to improve inference. Evaluated against Llama 3.3 70B, Qwen variants, and DeepSeek R1, the attacks were practical under realistic deployment conditions.

The RAG Extension

A separate paper from June 2026 (arXiv 2606.21842, “Agent-Assisted Side-Channel Attacks on Non-Prefix KV Cache in RAG”) identified that the attack extends beyond prefix-matched caching to RAG deployments that use non-prefix KV-cache fusion, a technique for accelerating retrieval-augmented generation by fusing cached document representations with live user context.

The SpliceLeak variant exploits the deterministic micro-architectural mechanisms used to align and fuse disjoint memory chunks in these deployments. Rather than requiring exact prefix alignment, it works against real-world RAG queries with unique user-specific prefixes. The attack proceeds in two phases: structural fingerprinting (determining the exact length of hidden private prompts via timing) followed by semantic extraction (recovering exact token content through boundary collision manipulation).

The reported figures are striking: up to 100% extraction success in bounded-entropy scenarios, approximately 63 requests per token, functioning under continuous batching noise. vLLM integrated with LMCache was the confirmed affected configuration. The result is that the document context retrieved for one user, potentially containing confidential business data, is recoverable by another tenant’s API calls.

Affected Systems and CVE Status

Confirmed affected frameworks: vLLM, SGLang, GPTCache, AIBrix, rtp-llm, LMDeploy. Multiple CVE identifiers have been assigned, though the specific identifiers were withheld in the research papers pending coordinated disclosure. The research tracked in the PromptFoo LLM Security Database confirms CVE assignment without publishing the IDs.

vLLM has shipped mitigations in its v1 engine, replacing the default prefix cache key derivation with SHA256 hashing rather than raw token sequences, which prevents predictable cache construction by attackers. The initial workaround added a random number component to cache keys. AIBrix and GPTCache have also completed fixes. SGLang, rtp-llm, and LMDeploy were at varying stages as of publication.

Cloud-hosted deployments running vLLM pre-v1 or unpatched versions of the other frameworks should be treated as vulnerable if they serve multiple customers or users with different trust levels on shared infrastructure.

What Gets Recovered

The research documented practical recovery of multiple data categories:

System prompt content. Proprietary system prompts containing application logic, persona definitions, access control instructions, and business rules can be reconstructed character by character through TTFT probing if those prompts are cached and reused across user sessions (the common pattern for hosted LLM applications).

User PII. Email addresses, SSNs, credit card numbers, and other structured PII that appears in user inputs are reconstructable if a tenant can form a plausible starting prefix and iterate. The attacker does not need to know what data is present; they need a reasonable hypothesis about structure (“My credit card is ”) and can test possible continuations.

RAG-retrieved documents. In the SpliceLeak configuration, document content injected via retrieval is recoverable through the non-prefix fusion timing signal. Documents that tenants assumed were accessed only by their own sessions are exposed to attacker-controlled measurement.

Mitigations

The primary fix is removing cross-tenant KV-cache sharing. vLLM v1 with SHA256 prefix hashing makes it computationally infeasible to construct prompts that deliberately collide with another tenant’s cache entries. This is the correct mitigation and the direction the ecosystem is moving.

Operators who cannot immediately upgrade have a harder path. Per-tenant cache isolation prevents cross-tenant leakage but sacrifices the throughput benefits that make the optimization worthwhile. For many providers, completely disabling shared caching is economically significant, which is why the SafeKV research (arXiv 2508.08438) is worth watching: it proposes selective isolation based on runtime sensitivity classification, recovering 40-50% of the performance benefit while blocking identified sensitive sequences from cross-tenant sharing.

For teams deploying their own inference infrastructure, the immediate action is to check which version of vLLM or SGLang is running and whether global prefix caching is enabled. The vLLM v1 engine with SHA256 prefix keys is the current recommended configuration. Deployments using vLLM with LMCache for RAG should specifically test whether SpliceLeak variant patches are included, as the attack surface is distinct from prefix caching alone.

The broader lesson is about optimization and trust boundaries. A performance feature designed to reduce compute costs assumes that what was previously computed is safely shareable. In a multi-tenant context where tenants are mutually distrustful, that assumption was not examined. It rarely is, until a paper names it.

References

• Selective KV-Cache Sharing to Mitigate Timing Side-Channels in LLM Inference
• Shadow in the Cache: Unveiling and Mitigating Privacy Risks of KV-cache in LLM Inference
• Agent-Assisted Side-Channel Attacks on Non-Prefix KV Cache in RAG
• KV-Cache Sharing Timing Side-Channel, PromptFoo LLM Security Database
• Cache Me, Catch You: Cache Related Security Threats in LLM Serving Frameworks, NDSS 2026