Terminal-Bench

Most coding benchmarks hand a model a finished snippet of context and ask for one answer back. Terminal-Bench, released in 2025 by Stanford researchers and the Laude Institute, asks something more demanding: put an agent in front of a real, disposable command-line environment and see whether it can actually drive it — running commands, reading their output, and adjusting — until a task like compiling a repository, training a small model, or fixing a broken server configuration is genuinely done. Success is judged by an automated check on the final state of the sandbox, not by how convincing the agent’s commands look.

Early results exposed a real gap between talking about terminal work and doing it: general commercial coding agents beat the benchmark’s own minimal reference scaffold, but even the strongest systems finished well under half the tasks at launch. That gap closed fast as agentic coding models matured through 2025 and 2026 — OpenAI’s Codex-tuned models climbed from the low 50s to the high 70s on Terminal-Bench 2.0 within a few months, and by mid-2026, on the harder 2.1 revision, Claude Fable 5 topped Terminal-Bench’s own live leaderboard at 83.8% via the Claude Code scaffold. Other labs reported comparable figures around the same time — Zhipu’s GLM-5.2 at 81.0% and xAI’s Grok 4.5 at a self-reported 83.3% — but these were vendors’ own numbers, and Grok’s result under the benchmark’s own scaffold was lower, at 79.3%.

That last detail points to a recurring pattern in this record: a vendor’s self-reported launch score and the same model’s result under Terminal-Bench’s own leaderboard scaffold don’t always match, since the agent harness wrapped around a model can matter almost as much as the model itself. The benchmark has also kept pace with progress by getting harder, moving from an 80–100 task 1.0 release through a tougher 2.0 and 2.1, and by mid-2026 a much harder v3.0 that reset the frontier from the low 80s back to the mid-30s — GPT-5.6 Sol and Claude Fable 5 both around 34%. A headline Terminal-Bench number, in other words, means little without the version and the scaffold it was measured on.