Lab leaders sign a one-sentence statement on extinction risk
"Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war."
- Ideas & essays
- Safety & alignment
- Major
The Center for AI Safety published a statement of exactly 22 words: “Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.”
The brevity was deliberate. The March pause letter had been criticised for bundling a contested policy proposal with a claim about risk, allowing disagreement about the former to be read as disagreement about the latter. The CAIS statement asserted only the claim, which made it signable by people who disagreed about everything else.
They signed. The list included the chief executives of OpenAI, Google DeepMind and Anthropic — the three organisations then building the most capable systems — alongside Geoffrey Hinton and Yoshua Bengio, two of the three Turing Award winners recognised for deep learning. Hinton had resigned from Google earlier that month specifically so he could speak about the risks without the statement reflecting on his employer. Yann LeCun, the third Turing laureate, declined to sign and became the most visible critic of the position.
The objections were structural. Critics noted that executives warning of catastrophe from their own products were simultaneously arguing for regulation that would raise barriers to entry, and that a focus on extinction crowded out attention to bias, labour and concentration of power. Others held that the claim was simply unsupported by evidence.
Whatever its merits, the statement functioned as a credential. For the following two years, legislators and journalists cited it as evidence that concern about catastrophic risk was a mainstream position inside the field rather than a fringe one, and it appears in the background of the US executive order, the UK summit, and the EU’s treatment of systemic-risk models.