2020: the year scale became a plan

For all that the wider world was preoccupied elsewhere, 2020 was the year the modern AI recipe came into focus. In January, OpenAI’s paper on scaling laws put numbers behind a hunch that had been circulating for years: that a language model’s ability climbs in a smooth, predictable way as you add more data, more parameters and more computing power. In May the same lab showed what that meant in practice with GPT-3, a model large enough that it could pick up new tasks from a handful of examples in its prompt, without being retrained for each one. And in November, DeepMind’s AlphaFold 2 all but solved the fifty-year problem of predicting how proteins fold, a result biologists had thought was still years away. Between them, these gave the field a thesis — that intelligence might be, in large part, a matter of scale — and its first genuine proof that the thesis paid off beyond the lab.

The year also set the pattern for the arguments that would follow. GPT-3 was reached not through open release but through a paid interface, an early sign of the openness-versus-control tension that has never really gone away. And in December, the researcher Timnit Gebru left Google after a dispute over a paper questioning the costs of ever-larger language models — their energy use, the biases baked into their training data, and the tendency to mistake fluency for understanding. Her departure raised a question the field still has not settled: whether the companies building these systems can be trusted to air their downsides. The capabilities, for now, remained narrow and easily tripped up, and the excitement ran well ahead of anything most people would notice in their daily lives.