Riemann-bench
also: RiemannBench, Riemann-Bench
Can a model solve research-level mathematics problems of the kind working mathematicians meet in their own research — problems that took their authors weeks?
Surge AIReleased 24 March 2026Live
Surge AI says it helped OpenAI build GSM8K, the grade-school maths test that defined an earlier generation of models. Riemann-bench, released in March 2026, sits at the other end of the scale. Its 25 problems come from the authors’ own research — professors, graduate students and former IMO medallists — and the sample problem in Surge’s launch post is rated at 40 to 50 hours’ work for an expert in the field. The problems are kept private so they cannot leak into training data. Each has a single closed-form answer that a program can check, and each was solved independently by two experts before it was accepted.
Surge positions it alongside FrontierMath, and criticised that benchmark’s harness for capping models at a million tokens. Riemann-bench imposes no limit, and on Surge’s board models can search the web and run code. That choice has moved the numbers. The arXiv paper reports every frontier model below 10%. Surge’s launch post reported higher figures, and it was revised after the pipeline dropped a one-hour timeout. By late October 2026 the board’s top scores were above 70%, led by GPT-5.6 Sol at 74.4%.
Anthropic reports the benchmark in its own model cards with a corrected version of the private set, both with and without tools, and its figures differ from Surge’s. Google’s Gemini 4 Carbon launch table gave Carbon 80.0%, which would lead the board. Carbon had not been added when the board was read the next day.
The set
25 private problems written by mathematics professors, graduate students and PhD-holding International Mathematical Olympiad medallists, each drawn from the author's own research and verified double-blind by two experts who solved it independently from scratch. Every problem has a unique closed-form answer checked by a programmatic verifier. On Surge's board models may search the web and run code, with no time or token limit per problem.
Where it stands
Surge's evaluation pipeline changed during 2026: its original blog post reported lower scores under a one-hour timeout and was later revised for unlimited time, and the arXiv paper reports all frontier models below 10% under its own setup. Anthropic reports its own runs with and without tools, which differ from the board (Claude Opus 5: 79% with tools in Anthropic's card, 68% on the board). Only like-for-like settings should be compared.
How the top score changed hands
Current best: GPT-5.6 Sol (Max) — 74.4% Surge's board, read 29 October 2026, with tools: GPT-5.6 Sol 74.4%, GPT-6 Astra 72.0%, Claude Opus 5.5 69.6%, Claude Opus 5 68.0%. Google reported 80.0% for Gemini 4 Carbon and 76.0% for Gemini 4 Argon, citing this board, but neither was listed.