Anthropic maps millions of concepts inside a production model

Anthropic’s interpretability team published “Scaling Monosemanticity,” reporting that sparse autoencoders could extract millions of human-interpretable features from Claude 3 Sonnet — a model in production, not a toy. Earlier work had demonstrated the technique on single-layer models; the open question was whether it survived contact with real scale.

The features were specific and often abstract: the Golden Gate Bridge, code with security vulnerabilities, sycophantic praise, internal conflict, gender bias, deception. Many were multilingual and multimodal — the same feature firing for the text, the translation and a photograph. Critically, the features were causal rather than merely correlational: clamping one high or low changed the model’s behaviour in the way the label predicted. Amplifying the security-vulnerability feature made the model write insecure code.

Two days later Anthropic made the point publicly by releasing Golden Gate Claude, a version with the bridge feature clamped to a high value, for anyone to talk to. It steered every conversation toward the Golden Gate Bridge and, asked about its physical form, described itself as the bridge. It was a joke that functioned as a demonstration: the interior of a large model had become legible enough to manipulate deliberately.

The work was widely read as support for the argument that models need not remain black boxes. It also set the practical agenda for mechanistic interpretability — feature-based auditing, monitoring for deceptive or dangerous internal states, and steering as an alternative to prompting — and comparable results followed from Google DeepMind and OpenAI. Whether interpretability can scale fast enough to matter for safety remained, and remains, unsettled.