Chartography
Can a model read the specialised charts professionals actually use — Kaplan-Meier survival curves, candlestick charts, contour maps, Sankey diagrams, Bode plots — accurately enough to give a precise numerical answer?
Surge AIReleased 16 July 2026Live
By 2026 frontier models scored around 90% on the standard chart-reading benchmarks, which mostly ask about bar charts and line graphs. Surge AI, a data-labelling company that also builds evaluations, released Chartography in July 2026 to test the charts specialists actually work with: wind roses, atmospheric back-trajectories, riprap-apron design curves, pediatric growth charts. Each of its 100 questions was written by a domain expert, needs a precise numerical answer, and comes with an accepted range — reading a growth chart to the nearest half-centimetre, for instance.
At launch the best model, GPT-5.6 Sol, answered 45% correctly. The paper, accepted at a computer-vision workshop, found the rest of 30 model configurations between 9% and 40%. Scores rose quickly after that: by late October Gemini 4 Argon and GPT-6 Astra were both near 72% on Surge’s no-tools board. Google reported 72.5% for Gemini 4 Carbon at its launch.
Anthropic adopted the benchmark in its own model cards from Opus 5.5 onward, reporting both the no-tools setting and a with-tools setting, where models can crop and zoom the image and run code. With tools, its models scored 80–90% — Claude Opus 5.5 rose from 64.4% to 89.0% — against 61–66% without.
The set
100 tasks, each a real chart image with an expert-written question and answer, drawn from engineering, finance, manufacturing and supply chain, and healthcare. Each answer has an expert-set acceptable range and was reviewed by three further experts; every task defeated at least one of two frontier models during curation. On Surge's board models answer with no tools, ten runs per task, and an LLM judge that never sees the chart checks whether the final answer falls within the range.
Example
What height corresponds to the 50th percentile for a four-year-old girl? Round to the nearest 0.5 centimeters.surgehq.ai
Where it stands
Built because older chart benchmarks such as ChartQA and CharXiv were near saturation. Scores rose from 45% to about 72% in three months. Anthropic also reports a with-tools setting (an image-cropping tool and code execution) where its models score 80–90%; those figures are not comparable with the no-tools board.
How the top score changed hands
- July 2026GPT-5.6 Sol (Max)45%Best model at launch; Surge's paper reported the other configurations at 9–40%.
- October 2026Gemini 4 Carbon72.5% (Google-reported)Google's launch table; not yet on Surge's board when read on 29 October.
Current best: Gemini 4 Argon (High) — 71.6% Surge's board, read 29 October 2026, no tools: Gemini 4 Argon 71.6%, GPT-6 Astra 71.0%, Claude Opus 5.5 66.3%, GPT-6 Sol 53.6%. Google reported 72.5% for Gemini 4 Carbon, citing this board, but Carbon was not yet listed.