MathVista

Word problems are easy to make harder by taking away the words. MathVista, built by researchers at UCLA, the University of Washington and Microsoft Research, tests mathematical reasoning that depends on reading a picture first — a plotted function, a geometry diagram, a bar chart, a logic-puzzle figure — and only then doing the arithmetic or algebra the picture implies. The benchmark pools 6,141 examples from 28 existing maths-and-vision datasets plus three tasks the authors built themselves to cover gaps: puzzle figures, function plots and academic-paper diagrams.

At launch in October 2023, the gap between models and people was stark. GPT-4V, the best of twelve systems tested, scored 49.9%, comfortably ahead of Google’s Bard but still 10.4 points short of the 60.3% human baseline the authors measured. That gap closed fast, and then reasoning models blew past it: Claude 3.5 Sonnet reported 67.7% in June 2024, already above the original human baseline, and by the end of that year OpenAI’s o3 reported 86.8% on the “testmini” subset with high reasoning effort — more than 25 points clear of the human baseline. Alibaba’s Qwen2.5-VL-72B reported 74.8% in February 2025, the strongest open-weight figure but still short of o3.

Unlike some contemporaries, MathVista has not been superseded by a discrete “Pro” successor, but it has faded from frontier labs’ own headline evaluation reports; Google’s Gemini 3 Pro evaluation document from November 2025, for instance, reports a newer slate of multimodal and reasoning benchmarks and does not include a MathVista figure. With the top scores now well above the human baseline, the benchmark is treated as effectively saturated.