Blueprint-Bench
also: Blueprint-Bench 2
Can a model work out the layout of a home from photographs alone — which rooms there are, how they connect and roughly how big they are — and draw it as a floor plan? A test of spatial reasoning rather than image recognition.
Andon LabsReleased 24 September 2025Live
Andon Labs, the company behind Vending-Bench, built Blueprint-Bench to measure spatial intelligence — working out the shape of a space that can only be seen in pieces — and lists it among its robotics evaluations. A model is shown a set of photographs taken inside an apartment and has to draw its floor plan — which rooms exist, which doors connect them and roughly how large each is. The plan is scored on how closely its graph of room connections matches the real one, not on how it looks.
The original paper, in September 2025, found most models little better than chance. GPT-5, the best single-pass language model, scored 0.431 against a random baseline of 0.279 and a human baseline of 0.547. Blueprint-Bench 2, released in May 2026, tests only agents. It normalises scores so that random guessing is 0, and lets an agent keep notes across the 50 apartments so it can learn as it goes.
By October 2026 the top of the board had caught up with people. Gemini 4 Carbon scored 0.586 at its launch, tying the human baseline, ahead of Gemini 4 Argon at 0.544. All the published scores come from Andon Labs’ own runs.
The set
Version 2: an agent receives about 20 interior photographs of an apartment and must produce a 2D floor plan, working through 50 apartments in sequence with a persistent notepad it can use to carry lessons from one apartment to the next. Plans are scored on their room-connection graph against the real layout: overlap of room-to-room connections 50%, degree similarity 20%, density 10%, room count 10%, door count 5%, orientation 5%, normalised so a random plan scores 0 and a perfect one 1. A human baseline, measured on 12 of the apartments, scores 0.586.
Where it stands
Run only by Andon Labs, which lists it among its robotics evaluations; no other lab or independent evaluator publishes scores. The v1 paper scored on an unnormalised scale (random baseline 0.279), so v1 and v2 numbers are not comparable.
Editions, and how each was led
Blueprint-Bench 2current
Current best: Gemini 4 Carbon — 58.6 (0.586 on the board's 0–1 scale) Andon Labs' board, read 29 October 2026, which says Carbon now ties the human baseline. Gemini 4 Argon 0.544, Claude Opus 5.5 0.512, GPT-6 Astra 0.497, Claude Fable 5.1 0.419.
Blueprint-Bench (original)Retired
Current best: GPT-5 — 43.1 (0.431 on the paper's 0–1 scale) Best single-pass language model in the paper; most models scored near or below the random baseline.