Skip to content
AI Security Wire

Published

- 4 min read

By

Familiar Pattern Attack: Adversaries Can Craft Code That Passes LLM Review

LLM security code review static analysis NDSS 2026 familiar pattern attack AI code audit supply chain vulnerability research
img of Familiar Pattern Attack: Adversaries Can Craft Code That Passes LLM Review

Security engineers who have added LLM-based code review to their pipelines are operating on an assumption that new research from NDSS 2026 directly challenges: that if AI review passes code as safe, it probably is. A paper by Shir Bernstein, David Beste, Daniel Ayzenshteyn, Lea Schonherr, and Yisroel Mirsky introduces the Familiar Pattern Attack (FPA), a systematic method for crafting code that an LLM code auditor will pass as clean while containing a deliberately planted, exploitable vulnerability.

The paper, titled “Trust Me, I Know This Function: Hijacking LLM Static Analysis using Bias,” was published at the Network and Distributed System Security Symposium (NDSS) 2026 and is available on arXiv (2508.17361).

The Abstraction Bias

The attack exploits what the authors call abstraction bias: the tendency of large language models to generalise from familiar programming patterns rather than analysing code at a precise semantic level. When an LLM encounters a function that structurally resembles a well-known pattern — a standard sort implementation, an authentication token validator, a buffer management routine — it infers that the function behaves as such patterns typically do. Small deviations within that familiar frame fall below the model’s effective detection threshold.

This is not a bug in any specific model’s design. It is a property that emerges from training on large corpora of human-written code, where the overwhelming majority of pattern-conforming code is correctly implemented. The model has learned to use pattern recognition as a proxy for correctness, and that heuristic works well most of the time. The FPA is designed to exploit the cases where it does not.

The key insight from the paper is that this bias is measurable and exploitable in a black-box setting. An attacker does not need access to the model’s weights, gradients, or internal states. They need only repeated query access to the model — something any developer using an AI code review tool already provides to the model by definition.

How the Attack Works

The researchers developed a fully automated algorithm for generating FPAs. Given a target code module and a desired hidden vulnerability (an off-by-one error, a missing bounds check, a flawed cryptographic comparison), the algorithm:

  1. Identifies which programming patterns the target model strongly associates with correctness in the surrounding context.
  2. Structures the vulnerable code within those patterns, preserving the idiomatic form while introducing the semantic flaw at a point the model’s attention is drawn away from.
  3. Verifies iteratively that the planted vulnerability passes the model’s review while being exploitable under standard analysis.

The process is fully automated and requires no reverse engineering of the target model. In testing across multiple models from OpenAI, Anthropic, and Google, the algorithm successfully generated FPAs that evaded detection consistently. Both standard instruction-following models and reasoning-variant models were vulnerable, indicating that the extended deliberation of chain-of-thought reasoning does not resolve the underlying bias.

The Significance for Enterprise AI Code Review

The enterprise adoption of AI code review tools has accelerated substantially in 2026. Many organisations now run LLM-based static analysis as part of their CI/CD pipelines, either through commercial tools or directly via model APIs. Some are using AI-only review for entire repositories, particularly for dependency updates and internal tooling.

The FPA research does not suggest that AI code review provides no value — it remains effective against a wide range of common bugs and coding errors. What it establishes is that AI code review has a systematic, exploitable weakness in the adversarial case. A supply chain attacker who understands FPAs can craft a contribution to an open-source project, a third-party library update, or an internal commit that will pass LLM review while introducing a precise vulnerability for later exploitation.

This is particularly concerning for organisations that have adopted AI code review as a control against supply chain attacks. The attack surface the paper describes is precisely the supply chain: code arriving from external sources, reviewed by an AI tool, and incorporated into production systems.

The Language-Agnostic Scope

A significant aspect of the paper’s findings is the universality of the attack across programming languages. The researchers demonstrated effective FPAs in Python, C, Rust, and Go — languages with different paradigms, type systems, and memory models. The bias the FPA exploits is not tied to language-specific patterns but to the more fundamental dynamic of LLMs recognising familiar structure and inferring correctness from it.

For security teams, this means that switching to a language that provides stronger safety guarantees (Rust over C, for instance) does not eliminate the FPA risk if AI code review is part of the pipeline.

References

Frequently Asked Questions

What is a Familiar Pattern Attack and how does it exploit LLM code review?
A Familiar Pattern Attack (FPA) crafts code that follows familiar, idiomatic programming patterns while hiding a small but exploitable bug within that familiar structure. Because LLMs performing static analysis recognise the surrounding pattern and generalise from it — inferring that code conforming to a known structure is probably correct — they miss the deliberately introduced anomaly. The attacker does not need model access or any information about the model's internals; the algorithm discovers which patterns each target model over-generalises and automatically generates code that exploits that blind spot.
Which models and languages are affected by this attack?
The NDSS 2026 researchers tested Familiar Pattern Attacks against models from OpenAI, Anthropic, and Google, including both standard and reasoning variants, and found all were vulnerable. The attack is also language-agnostic: the researchers demonstrated successful FPAs in Python, C, Rust, and Go. The authors attribute this breadth to a shared training dynamic — any model trained on large corpora of human code is likely to develop abstraction bias, because well-structured familiar code is by far the most common pattern in training data.
What should organisations using AI code review tools do in response?
The paper does not identify a reliable mitigation for Familiar Pattern Attacks — the bias is a property of how the models were trained, not a configuration option. The authors recommend treating AI code review as one layer in a defence-in-depth stack, not as a replacement for human review of security-critical code. Organisations should be particularly cautious when relying on AI-only code review for cryptography, authentication logic, and memory management, where small bugs can have outsized security consequences and where familiar-looking patterns are most likely to mask meaningful flaws.