Stanford HAI releases 2024 AI Index Report

Stanford HAI published the seventh edition of its annual AI Index, a compilation of capability, investment and adoption data intended as a reference point for the state of the field over the preceding year. The report drew on benchmark results, funding data, model releases and survey research to summarise 2023 rather than announce anything new itself.

Among its more widely cited figures, the report estimated that training OpenAI’s GPT-4 had cost roughly $78 million in compute, and training Google’s Gemini Ultra roughly $191 million — rare public estimates of frontier training costs, which labs did not disclose directly. It found generative AI investment had grown nearly eightfold from 2022 to reach $25.2 billion in 2023, even as total AI investment overall declined, and that industry produced 51 notable models that year against academia’s 15, with the United States responsible for 61 of the notable models tracked worldwide, ahead of the EU’s 21 and China’s 15. The report also noted AI systems had surpassed human performance on some benchmarks in image classification, visual reasoning and English-language understanding, while continuing to lag on tasks like competition-level mathematics and visual commonsense reasoning.

As an annual aggregation rather than a single event, the Index’s significance lay less in any individual figure than in providing a citable, methodologically consistent baseline that journalists, policymakers and researchers drew on throughout the following year — several of its cost and investment estimates were repeated widely in coverage and testimony without independent verification, given the absence of comparable disclosure from the labs themselves.