TextQuests

TextQuests turns a genre of 1980s computer games into a test of long-horizon reasoning. The Center for AI Safety, with collaborators at Carnegie Mellon and Gray Swan AI, has models play 25 classic Infocom interactive-fiction games — Zork among them — where the only interface is typed commands and prose responses, and success depends on mapping an unfamiliar world, tracking an inventory, and following a plan across hundreds of turns. There are no tools and no external scaffolding: the model must hold the whole growing game state in its context.

That makes the benchmark a clean test of two things generalist chat models are not optimised for: sustained planning and reliable memory. The paper’s illustrative failure is telling — a model confidently misremembering where it had left an object dozens of turns earlier — because it is exactly the kind of error that compounds in a long game and rarely shows up in a single-turn benchmark. Runs are capped at 500 steps and scored on how many of a game’s labelled milestones the model reaches, with a companion metric tracking harmful in-game actions.

TextQuests is one of the agentic evaluations on the Center for AI Safety’s AI Dashboard, run independently across frontier models. Progress there has been steady rather than sudden — from around 13% for GPT-4o, to roughly 31% once reasoning models arrived, to Claude Fable 5 at 56.1% by mid-2026 — and the recurring bottleneck is memory over a long horizon rather than any single hard puzzle, which is precisely the capability the benchmark was built to isolate.