AIME

AIME is a 15-question, integer-answer exam that the Mathematical Association of America has run each February since 1983 to identify the strongest scorers on the AMC for US Mathematical Olympiad qualification. It has no connection to AI research by design — but from late 2024 its problems became the standard way frontier labs summarised a model’s mathematical reasoning in a single number, because a 30-problem set with no partial credit and an integer answer between 0 and 999 is trivial to grade automatically at scale.

The trigger was OpenAI’s o1-preview, which reported solving 83% of AIME 2024 problems against GPT-4o’s 13% — a gap large enough that AIME scores became a fixture of release announcements. The climb that followed was fast: o3 missed only one of 30 problems three months later, DeepSeek’s open-weight R1 reported a comparable 79.8% alongside a published method, and a single point release of R1 later that year lifted its AIME 2025 score from 70.0% to 87.5% simply by letting the model reason for longer.

That speed was also the problem. Researchers at ETH Zurich reported strong signs that AIME 2024 problems had already leaked into training data, which meant high scores could reflect memorisation as much as reasoning. The response has been to treat each AIME sitting as disposable once a year or two has passed — scoring only the newest exam, or routing evaluation through continuously updated platforms such as MathArena — while accepting that, as with any competition whose problems circulate freely online, the clock on contamination starts again the moment a new sitting is published.