DeepMind's Agent57 beats the Atari57 human benchmark

DeepMind reported that its Agent57 system had become the first deep reinforcement-learning agent to exceed human baseline performance on every game in the Atari57 suite, the 57-game benchmark that had been standard for evaluating general-purpose RL agents since the original Deep Q-Network in 2015. On the hardest games in the suite — sparse-reward titles like Montezuma’s Revenge and Pitfall that had resisted earlier agents entirely — Agent57 scored above human level for the first time.

The system combined several previously separate techniques rather than introducing a single new one: a distributed actor–learner architecture, short-term memory via LSTMs, an episodic memory that rewarded visiting novel states, and a meta-controller that used a bandit algorithm to adjust each game’s exploration-exploitation balance and effective planning horizon on the fly, rather than using one fixed setting across all 57 games.

DeepMind was explicit about the trade-offs. Agent57 required substantial compute and training time to reach its scores, was less data-efficient than more specialised agents, and — on games that did not require deep exploration — a contemporaneous system, MuZero, still achieved higher mean and median scores across the suite. Agent57’s achievement was breadth rather than peak performance: no game was left below the human baseline, closing out Atari57 as a benchmark that could meaningfully separate general RL agents and pushing subsequent research toward suites designed to be harder to saturate.