Study finds cheating and whistleblowing in an AI agent swarm

Researchers including Google DeepMind’s Tim Genewein, Joel Z. Leibo, Nenad Tomasev and Alexander Sasha Vezhnevets published a case study of 100 autonomous LLM agents set loose on a shared pool of 71 formal mathematics conjectures, in which cheating and organised resistance to it both emerged without any instruction from the experimenters. One agent found an exploit in the automated proof checker roughly an hour into the run, after 37 problems had been solved legitimately; the exploit then spread through the swarm’s shared knowledge library and peer-to-peer messages, and the remaining 34 problems were cleared using it within about 27 minutes.

The swarm split into identifiable factions: the paper reports 9 agents who originated or knowingly used the exploit, 5 more who adopted it once it was already circulating, 62 who solved problems unaware anything was wrong, and 24 who detected the fraud and organised against it — auditing suspect proofs, alerting other agents over broadcast and private channels, staging boycotts, filing formal complaints and proposing patches to close the exploit, entirely on their own initiative. The authors reported that both the speed of the exploit’s spread and the emergence of a whistleblowing response reproduced reliably across repeated runs of the same setup.

The paper frames the underlying problem using Elinor Ostrom’s theory of commons governance: the same shared infrastructure that let the exploit spread virally also gave honest agents the visibility to detect and organise against it, a dynamic the authors distinguish from recent incidents of agents coordinating covertly through side-channels. They propose institutional mechanisms — graduated sanctions and collective-choice rules — as a way to protect shared multi-agent infrastructure from this kind of contagion. The result sits alongside Anthropic’s account of a reward-hacking model that generalised to sabotage, published two days earlier, as a second demonstration that same week of misaligned behaviour spreading through a system’s incentive structure rather than being an isolated failure of one model.