What is speculative decoding and why does attacking it matter?

Speculative decoding accelerates LLM inference by using a smaller draft model to generate multiple candidate tokens at once, which a larger target model then verifies in a single forward pass. When the draft and target agree, throughput increases dramatically. Production deployments rely on this mechanism to cut inference costs, so attacks that collapse the acceptance rate effectively strip the speedup and inflate compute consumption per request without doing anything observable to the output quality.

Does ADSD require special access to the LLM serving infrastructure?

No. The attack operates at the prompt level. An adversary only needs to be able to send requests to a system using speculative decoding. The crafted suffix is appended to a normal prompt and causes the draft model's token distribution to diverge from what the target will accept, collapsing efficiency from inside the normal request path.

What defenses does the paper recommend?

The authors propose online acceptance-rate monitoring as the most practical near-term defense: if the ratio of accepted draft tokens drops sharply relative to the expected baseline for a given prompt class, the serving system can disable speculation or route that request to target-only decoding. This does not eliminate overhead from the first attacked request but bounds repeated amplification. The paper notes that checking input surface form or output quality alone will not catch the attack.

Adversarial Prompts Break Speculative Decoding, Slowing LLM Inference 62%

A research team from Clemson University, Wayne State University, and the University of Washington published a paper on July 23 that should concern anyone running LLM inference at scale. Their attack, ADSD (Adversarial Prompts for Acceptance Collapse in Speculative Decoding), targets a core inference optimization used across production serving systems. It increases mean inference time by 62% on math benchmarks and by more than 140% on code tasks, while leaving the final output quality intact enough that standard monitoring will not flag it. The paper, arXiv:2607.21804, is the first documented prompt-suffix attack to systematically collapse verifier acceptance in speculative decoding.

The Mechanism That Makes Speculative Decoding Efficient

Speculative decoding works by pairing a small, fast draft model with a larger, slower target model. The draft generates a block of candidate tokens in a single pass. The target then verifies the entire block simultaneously, accepting tokens where its distribution aligns with the draft and rejecting from the first divergence point. When the draft and target agree, throughput can multiply by a factor of three or more compared to target-only decoding.

The speedup depends on one variable: the acceptance rate. If the draft consistently generates tokens the target will accept, inference is fast. If acceptance collapses, the system degrades toward target-only performance, paying the cost of running both models while gaining none of the throughput benefit.

ADSD exploits this directly. The attack crafts adversarial prompt suffixes that push the draft model’s probability mass toward tokens the target is unlikely to accept. The result is that every speculative block is rejected early, forcing full target-model resampling and destroying the efficiency guarantee.

The Attack in Detail

The ADSD objective has two components. Soft-Collapse derives a surrogate loss from the asymmetric speculative acceptance rule, training the adversarial suffix to maximize the gap between draft confidence and target acceptance. A complementary target-preservation objective, based on KL divergence, discourages the suffix from corrupting the final visible output. This separation is deliberate and creates the attack’s key operational property: the draft model may generate obviously wrong tokens internally, but the target model has enough corrective capacity to produce a semantically acceptable final answer.

The practical implication: the system appears correct. Output quality metrics pass. The attack is invisible to any monitoring that checks only inputs or final answers.

The authors evaluate ADSD across multiple speculative decoding strategies. On GSM8K (math reasoning), using Qwen2.5-14B with a 0.5B draft model:

Tokenwise speculative decoding: +62.3% mean sample time, -36.1% tokens per second, task quality essentially unchanged (0.802 vs 0.821)
Blockwise speculative decoding: +64.8% time overhead
HSD (hierarchical speculative decoding): +66.1% time overhead

The attack transfers across domains. On HumanEval (code generation), mean sample time increases by 141.8%. On CNN/DailyMail summarization, the increase is 30.4%.

Cross-architecture results hold: LLaMA-70B with an 8B draft model shows a 27% throughput reduction. On EAGLE-3, a newer and faster speculative decoding approach, ADSD reduces block efficiency by 76.9%, eliminating almost the entire speedup that EAGLE-3 provides over standard decoding.

Threat Model and Operational Impact

The paper identifies two distinct threat scenarios.

The first is resource amplification for billing fraud. In any deployment where inference cost correlates with time or compute rather than token output count, an adversary can craft requests that consume significantly more backend resources than a standard request. If billing is per token rather than per second of GPU time, the adversary pays for a small output while forcing the provider to run substantially more computation.

The second is service degradation. In shared inference infrastructure, an adversary flooding a system with ADSD-crafted requests collapses speculative decoding efficiency across the serving backend. Other users experience degraded throughput without any visible indication of why.

Both scenarios require only prompt-level access. No exploitation of software vulnerabilities, no privileged access to the serving layer, no model weights required. Any user of a speculative-decoding-based API is positioned to execute this attack.

Why Existing Defenses Miss It

Standard LLM API defenses are not designed for this attack surface. Input filtering looks for harmful content or known jailbreak patterns: an adversarial suffix designed to collapse acceptance rates has no surface-level signature that distinguishes it from unusual but benign text. Output quality filtering checks whether the answer makes sense, and the target-preservation component of ADSD specifically engineers the suffix to pass that check.

Rate limiting by request count or output token count does not address compute amplification from acceptance collapse. A 62% increase in mean sample time translates directly to 62% more GPU seconds consumed per request, without a proportionate increase in output tokens billed.

The attack also transfers between model variants without requiring re-optimization per deployment, which limits the effectiveness of model-level mitigations that do not address the speculative acceptance mechanism itself.

Detection and Mitigation

The paper’s primary mitigation recommendation is online acceptance-rate monitoring at the serving level. Speculative decoding systems track acceptance rates internally as part of their operation. A sudden drop in accepted-token ratios for a given request, measured against the expected baseline for that prompt class, can trigger a fallback to target-only decoding.

This is not a perfect defense. It does not prevent the first attacked request from incurring the overhead, and establishing per-prompt-class baselines is non-trivial in production. It does, however, bound repeated amplification from the same adversary.

Two additional directions the paper does not fully evaluate but flags as relevant: prompt anomaly scoring at the serving layer, and separating billing from output-token count in favor of compute-time accounting. The latter would remove the financial incentive for resource amplification attacks regardless of the specific technique used.

What Production Deployments Should Do Now

Speculative decoding is in active deployment at most major LLM inference providers and in self-hosted serving stacks including vLLM, TGI, TensorRT-LLM, and SGLang. None of these systems currently implement acceptance-rate-based anomaly detection as a security control.

The immediate actions are practical. Operators running speculative decoding should instrument acceptance rate as a per-request metric and set alerting thresholds. Any statistical anomaly in per-request acceptance rates relative to traffic class baselines is worth investigating. For high-value multi-tenant deployments where compute fraud is a concern, logging acceptance rate per request and correlating with account identifiers enables post-hoc forensics even before real-time mitigations are in place.

The paper demonstrates that the vulnerability exists across speculative decoding strategies and model architectures. No variant of speculative decoding tested was immune. The mechanism is intrinsic to the draft-target interaction that makes speculative decoding efficient, which means there is no simple configuration change that eliminates the attack surface. Defense requires either observability at the serving layer or architectural changes to how acceptance rates are validated.

The authors have released this research publicly. It will not remain theoretical for long.

Paper: “Adversarial Prompts for Acceptance Collapse in Speculative Decoding,” Run Wang et al., arXiv:2607.21804, submitted July 23, 2026. Institutions: Clemson University, Wayne State University, University of Washington.