Epoch AI publishes 'Training compute of frontier AI models grows by 4-5x per year'

Epoch AI, a research organisation that tracks compute and capability trends in AI, published an updated estimate that the computing power used to train what it classes as “notable” AI models has grown by roughly 4 to 5 times a year since 2010. The figure drew on a database of 333 compute estimates for models released between 2010 and May 2024 — about triple the size of the sample behind the organisation’s original 2022 estimate — with models qualifying as notable by criteria including citation count, benchmark performance, historical significance or wide deployment.

Epoch fitted several statistical models to the data — simple exponential, kinked exponential, discontinuous and hyperbolic growth curves — and selected among them using standard model-comparison methods (the Bayesian Information Criterion and cross-validation), rather than reporting a single curve fit by eye. The analysis excluded DeepMind’s AlphaGo-lineage models as statistical outliers because of their distinctive reinforcement-learning training regime, and it separated the general trend from a narrower one covering language models specifically, dated from the Transformer architecture’s introduction in 2017 onward.

The report was explicit about its own limits: the underlying database remained incomplete, with many known models missing a reliable compute figure; the fitted trend was sensitive to which outliers were included and where any structural break was placed; and the data showed evidence of a slowdown around 2018 that Epoch said it could not fully explain.

The 4–5x figure became a standard reference point in subsequent debate over AI timelines, cited by forecasters projecting how quickly training runs might approach physical, financial or data limits, including later public scenario work such as the AI 2027 forecast. It functioned less as a claim about any single model than as the empirical backbone other groups’ extrapolations built on.