EnigmaEval

EnigmaEval is built out of puzzle hunts — the elaborate, multi-layered puzzles that teams of enthusiasts spend hours or days unpicking at events like the MIT Mystery Hunt. Scale AI and the Center for AI Safety transcribed 1,184 of them into a mixed image-and-text form, each with a single verifiable answer, precisely because such puzzles resist the shortcuts that saturate ordinary benchmarks: there is no template to pattern-match, the relevant knowledge is implicit, and the path from clue to answer runs through several leaps of lateral reasoning.

That design makes EnigmaEval one of the hardest tests in circulation. A puzzle like “Switching Channels” hands the solver two columns of television shows and asks them not merely to pair the shows but to discover the hidden theme those pairings encode — the kind of step that is obvious in hindsight and very hard to reach. The benchmark keeps a Normal split and a Hard split drawn from the most demanding hunts, so a single number still separates strong reasoners from the rest.

EnigmaEval is tracked on the Center for AI Safety’s AI Dashboard as an independent third-party evaluation, and there it remains among the least saturated: GPT-4o solved well under 1% of puzzles, reasoning models such as o3 lifted that into the teens, and by mid-2026 the frontier — Claude Opus 5 at 43.9% — had reached the low 40s while still failing the majority. Its slow climb is the reason it has become a standard citation for reasoning that unfolds over many steps rather than one.