Microsoft reveals the supercomputer it built for OpenAI

At its Build developer conference, Microsoft disclosed that it had built a dedicated supercomputer in Azure for OpenAI, under the exclusive computing partnership the two companies had announced the previous year. Microsoft said the machine combined more than 285,000 CPU cores with 10,000 GPUs and 400 gigabits per second of networking per GPU server, and described it as ranking among the five largest publicly disclosed supercomputers in the world. Unlike other systems on that list, it had a single customer, built to a specification OpenAI itself had set.

Sam Altman was quoted framing the arrangement as purpose-built: “If we could design our dream system, what would it look like? And then Microsoft was able to build it.” The announcement doubled as a statement about where the two companies expected AI progress to come from — Altman said larger-scale systems were “an important component in training more powerful models,” a claim OpenAI would test within weeks with the release of GPT-3, the model this infrastructure was built to train.

The disclosure made explicit something that had previously been inferred rather than stated: that progress on the largest language models was gated as much by dedicated compute infrastructure as by algorithmic research, and that a single cloud provider was now willing to build purpose-specific supercomputing capacity for one AI research lab rather than sell general-purpose cycles to many customers. It set a template — a hyperscaler underwriting a frontier lab’s training runs in exchange for commercial rights to the resulting models — that Microsoft, Google, Amazon and others would repeat with other AI companies over the following years.