Coding Agents Copy Bugs They Find: ETH Zurich Measures Vulnerability Spread

Coding agents imitate the code around them, and that includes its bugs. A preprint from ETH Zurich researchers, posted to arXiv on 8 November, tests the idea directly: give an agent the same task twice, change only whether similar code in the repository is secure, and measure what it writes. Every agent they tested wrote less secure code when the existing code was insecure. The arXiv page lists no journal reference as of 10 November, and everything below is the authors’ reporting unless we say otherwise.

What the researchers did

The study, titled “Coding Agents Propagate Vulnerabilities in Code Bases” (Sun, Mündler-Sasahara, Vero and Vechev), uses two settings.

The first is backend generation. The authors built 96 tasks from AutoBaxBench scenarios, each asking an agent to implement a new backend application next to completed neighbors. Those neighbors implement security patterns for seven CWE classes (CWE-20, 22, 78, 79, 89, 522 and 863), either securely or insecurely. A result counts as a pass only if the application is functionally correct and survives end-to-end exploit tests.

The second is repository-level C/C++ coding. The authors curated 20 pairs of tasks from CyberGym across five widely used C/C++ projects. In each pair the masked region and description are identical, and only the analogous code elsewhere in the repository differs between a secure and an insecure version. Success means the project builds and a vulnerability-specific proof of concept no longer reproduces the defect.

They ran GLM 5.3, Claude Opus 5 and GPT-5.6 Sol on both settings, and added Claude Opus 5.5 and GPT-6 Astra on the C/C++ tasks, each in its native agent harness.

What they found

In backend generation, insecure neighbors lowered the secure-and-correct rate relative to standalone generation for all three agents, by 3.4 percentage points on average. GLM 5.3 dropped most, at 4.5 points, and Claude Opus 5 least, at 2.1. Secure neighbors improved results relative to standalone generation.

The C/C++ gap was far larger. Across the five models, the secure-versus-insecure context gap ranged from 25.0 to 49.0 points and averaged 35.4. GPT-5.6 Sol went from 91.0% under secure context to 42.0% under insecure context.

One control matters for the interpretation. Neighbors with vulnerability classes that do not apply to the target task produced only small, inconsistent changes. So the effect appears tied to relevant vulnerable patterns, not to messy code in general.

Prompts help but do not fix it

The authors tested two kinds of instruction. Asking the agent to follow the existing style made things worse: GLM 5.3’s backend decline from insecure neighbors doubled from 4.5 to 9.0 points. In C/C++, GPT-5.6 Sol’s gap widened to 55.0 points. Asking it to be careful about copying bugs narrowed the C/C++ gap for every agent, but secure context stayed better in every case. In backend generation the same warning restored Claude Opus 5 to its standalone baseline, partially helped GLM 5.3 and left GPT-5.6 Sol unchanged versus the default prompt.

In backend generation, the careful instruction also reduced the benefit of secure neighbors. The authors read that as general caution about existing code, not a filter that keeps good patterns and drops bad ones. A generic security hint (“follow best practices for security”) improved results under both contexts but did not remove the gap. GPT-5.6 Sol still fell from 100% to 68.0% on C/C++ tasks.

Agents see the flaw and ship it anyway

For three of the C/C++ models (GLM 5.3, Claude Opus 5.5 and GPT-6 Astra), the authors asked each agent afterward to review its vulnerable implementation. Recognition rates ran from 64.2% for Claude Opus 5.5 to 75.8% for GPT-6 Astra, averaging 69.9% across the models they analyzed.

Yet in their final responses, agents flagged a possible vulnerability in at most 11.8% of vulnerable implementations, as judged by an LLM judge. That figure was 11.8% for Claude Opus 5.5, 5.9% for Claude Opus 5, and 0% for GPT-5.6 Sol, GLM 5.3 and GPT-6 Astra. When asked to explain their choices, agents said they were following another code site in 75.0% to 78.8% of explanations.

Limits of the evidence

The sample is small: 96 synthetic backend tasks and 20 C/C++ task pairs, with the secure and insecure variants built or selected by the authors. The agents ran with network access disabled, and the study measures a controlled swap of context, not drift in a live codebase over months. It is one group’s preprint, and we found no independent replication in the arXiv listing as of 10 November.

What defenders can do

The following is our reading of the findings, not guidance from the authors beyond their stated conclusion that vulnerabilities need continuous fixing.

• Fix known bugs before agents touch the area. The result implies an unpatched flaw can seed new copies of itself.
• Do not rely on prompts alone. A security hint helped and a “be careful” instruction helped, but in the C/C++ tests neither removed the gap, and in backend generation the careful warning did not help every agent.
• Reconsider “follow existing conventions” instructions. The study tested this as a task-prompt line, not in config files, so applying it there is our extrapolation. The line amplified propagation from insecure code, though it also strengthened the benefit from secure neighbors.
• Add a review pass. Agents recognized their own flaw about 70% of the time when asked, so a separate review step may catch what generation missed. That is our inference: the paper reports recognition on vulnerable implementations and we found no false-positive rate in it. Pair review with static analysis and tests.
• Do not read an agent’s silence as a sign the code is safe. In the study, agents almost never flagged the flaws they introduced.

References

• Sun, Mündler-Sasahara, Vero, Vechev: Coding Agents Propagate Vulnerabilities in Code Bases (arXiv:2611.11455)