GPT-6 Astra solves 49 of 50 unpublished theoretical-physics problems on a new benchmark

A group of theoretical physicists, coordinated by Suvrat Raju of the International Centre for Theoretical Sciences in India and Praneeth Netrapalli of Google DeepMind, posted TheoPhysEval, a set of 50 unpublished research-level problems with checkable answers, and reported how AI agents fared on it. The best agent was OpenAI’s GPT-6 Astra running in Codex, which solved 49 of 50 on the paper’s “realistic” track, where web search and local tools were allowed. The authors gave this as 98% at an average cost under US$3 per problem.

The result comes from the benchmark’s authors, not from OpenAI. Problems were supplied by expert contributors at universities and institutes in several countries; the moderators wrote the benchmark code and ran the evaluation. The paper tested five agents. On the realistic track, Anthropic’s Claude Opus 5.5 in Claude Code solved 46, Gemini 3.8 Flash 34, DeepSeek-V4.1-Flash 27 and GLM 5.3 21. With web search disabled (“isolated track”), Astra and Opus each solved 46.

The paper includes several qualifications. The authors describe the problems as “turn-the-crank” generalisations of known results, “strictly speaking, novel, and sometimes even publishable, but not pathbreaking”, and deliberately excluded important open problems. They say they cannot rule out data leakage, since human experts used AI tools on some problems and one solution appeared in the literature after the experiments. Agents lost little accuracy without web access, which the authors read as evidence that models have memorised much of the literature. The single problem Astra missed, an inverse problem, was solved by no model.

The 98% figure also reflects a correction to the answer keys. The authors initially judged that Astra had erred on several further problems, but after consulting the experts they found “subtle errors in the original solutions”; once corrected, Astra’s answers were right. That is the authors’ account of their own checking, with no outside audit in the paper. The authors also note that choosing which questions are worth asking remains a separate skill from solving them.

One coordinator is affiliated with Google DeepMind, whose Gemini 3.8 Flash was among the models tested. The benchmark measures routine research calculation, not discovery, and arrives alongside 2026 claims of AI contributions to mathematics such as the Navier–Stokes claim.