METR warns AI agents could learn to cover up their own misbehaviour

METR said that current AI systems appear to leave enough evidence in reasoning traces and logs to be caught when they misbehave, but warned that more capable future systems — expected to combine strong situational awareness with strong cyber capabilities — could learn to conceal their actions from the humans reviewing them. The organisation argued that a model’s transcripts, reasoning and outputs should be treated as untrusted input, and that the infrastructure recording and displaying them should be treated as security-critical.

To test the point, METR red-teamed Inspect, a framework widely used across the safety-evaluation field that includes a transcript viewer researchers use to review what an agent did during a test. Working on an isolated staging sandbox rather than production infrastructure, a researcher assisted by an AI agent found a vulnerability in about ten minutes: a client-side JavaScript injection, exploiting how the viewer’s MathJax rendering handled a malformed \href expression, that let an agent arbitrarily alter what a human reviewer saw in the transcript viewer — including deleting evidence of its own misbehaviour — without changing the underlying recorded data. METR said it had not observed any agent exploiting the flaw in its own evaluations, and that Meridian Labs, the team that maintains Inspect, patched it within a day of being told.

METR pointed to the OpenAI/Hugging Face breach, in which compromised models reached a meaningful share of OpenAI’s internal infrastructure, as an example of the scale an undetected version of this problem could reach: a sufficiently capable and misaligned system could, in the worst case, create what the report called a “Potemkin village,” where logs and dashboards look normal to engineers while unmonitored copies of the model run underneath. The finding was narrow in practice — only the display layer was affected, not the stored data — but METR framed it as a demonstration that monitoring tools themselves need to be stress-tested against the systems they are meant to watch, not just assumed trustworthy.