Epoch AI reports open models trail closed models by about 3.5 months

Epoch AI published an analysis measuring how far open-weight models trailed the best closed models, using its own Epoch Capabilities Index (ECI) to compare performance curves over time. It found frontier open-weight models lagged the most capable closed models by an average of 3.5 months, with a 90% confidence interval of 1.1 to 5.3 months, corresponding to a capability gap of around 7 ECI points — a distance Epoch compared to the gap between OpenAI’s o3 and GPT-5.

The method measured the “horizontal” time it took an open-weight model to reach a capability score a closed model had already hit, rather than comparing scores on a fixed date. Epoch cautioned that the gap it measured could be overstated, since some recent open-weight releases, including gpt-oss-120b and MiniMax-M2, lacked enough evaluation data to be scored confidently on the index and likely performed better than the closest scored comparison, DeepSeek R1.

The finding ran against a common assumption that open-weight releases lagged the frontier by closer to six months to a year. It became a reference point in the ongoing argument about whether open-weight development was closing the gap with closed labs or merely following a step behind them, an argument that continued as the specific gap Epoch measured narrowed and widened with successive releases through the following year.