WinoGrande

WinoGrande scaled up an older idea. The original Winograd Schema Challenge offered just 273 expert-crafted sentence pairs testing whether a model could resolve an ambiguous pronoun using commonsense knowledge rather than surface pattern-matching — famous but too small to train or robustly evaluate large models on. AI2 researchers rebuilt it at scale in 2019: 44,000 similar problems collected through crowdsourcing, then run through a debiasing algorithm called AfLite that stripped out the word-association shortcuts a model could exploit without genuinely reasoning about the sentence.

The bias reduction did its job. At launch, the best available models scored between roughly 59% and 79% depending on how much training data they had, against a 94% human ceiling — a real but not enormous gap by the standards of commonsense benchmarks at the time. Within a year, GPT-3’s 175-billion-parameter model reached 77.7% in a few-shot setting, still short of a fine-tuned T5 model’s 84.6% but well ahead of earlier fine-tuned baselines, evidence that scale alone was closing ground that hand-designed bias reduction had opened.

WinoGrande remained a standard secondary citation alongside HellaSwag and ARC through the early 2020s as a quick commonsense-reasoning check, typically reported in a results table rather than a headline. As frontier labs shifted to explicitly graduate-level and professional benchmarks such as MMLU and GPQA, pronoun-resolution tasks stopped being hard enough to distinguish leading models, and WinoGrande faded from release announcements even as it stayed part of standardised academic evaluation suites.