BLINK
also: BLINK: Multimodal Large Language Models Can See but Not Perceive
Whether a multimodal model can do core visual perception — judging relative depth, matching visual correspondences, spotting image tampering, reasoning across multiple viewpoints — that people solve almost instantly but that resists being reduced to language description.
University of Pennsylvania, University of Washington, Allen Institute for AI, UC Davis and ColumbiaReleased 18 April 2024Live
BLINK’s title makes its argument in one line: multimodal large language models can see but not perceive. Researchers from the University of Pennsylvania, University of Washington, Allen Institute for AI, UC Davis and Columbia took 14 classic computer-vision tasks — relative depth ordering, visual correspondence between images, forensic detection of tampering, reasoning across multiple camera views — and turned them into 3,807 multiple-choice questions. The tasks were chosen because a person solves most of them almost instantly, “within a blink,” yet they resist the trick that helps language models on other visual benchmarks: describing the image in words and reasoning over the description.
That gap showed up immediately. At the benchmark’s release in April 2024, human annotators averaged 95.70% accuracy on the same questions, while the best model tested, GPT-4V, managed 51.26% — barely ahead of Gemini Pro’s 45.72% and only modestly above random guessing on some tasks. The result argued that strong scores on knowledge- or reasoning-heavy multimodal benchmarks did not imply strong low-level visual perception, and that the two needed to be measured separately.
Models have since closed part of the gap without erasing it: a comparison table in Alibaba’s Qwen2.5-VL technical report, published in February 2025, put GPT-4o at 68.0%, ahead of Qwen2.5-VL-72B (64.4%) and InternVL2.5-78B (63.8%); Claude 3.5 Sonnet was not included in the same comparison. Human performance was not re-measured in that report, but the original 95.70% ceiling suggests the perception gap BLINK was built to expose had narrowed rather than closed. A follow-up benchmark, BLINK-Twice, arrived in 2025 to push past perception into perceptual reasoning.
The set
3,807 multiple-choice questions built by reformatting 14 classic computer-vision tasks, each paired with one or more images and, for several tasks, visual prompting such as a circled region. Scored on accuracy against a human baseline collected on the same questions.
Example
A relative-depth item, verbatim: 'Which point is closer to the camera?' with two labelled points marked on an image and answer choices '(A) A is closer' / '(B) B is closer' — trivial for a person, but it isolates raw depth perception from semantic reasoning.huggingface.co
Where it stands
Frontier models still trail the roughly 96% human baseline recorded at launch by a wide margin; a 2025 follow-up, BLINK-Twice, was built to push further into perceptual reasoning rather than perception alone.
How the top score changed hands
- April 2024GPT-4V51.26%Best-performing model at launch, against a 95.70% human average and roughly 38% random-guessing baseline; Gemini Pro scored 45.72% and Claude 3 Opus 44.11%.
- February 2025GPT-4o68.0%Reported in a comparison table alongside Qwen2.5-VL-72B (64.4%) and InternVL2.5-78B (63.8%); Claude 3.5 Sonnet was not reported in the same table.
Current best: GPT-4o — 68.0% Highest score in a third-party comparison table that also lists Qwen2.5-VL-72B (64.4%) and InternVL2.5-78B (63.8%); Claude 3.5 Sonnet was not reported in the same table. Not from an official BLINK leaderboard, and later frontier models may score higher without a comparable figure confirmed here.