Microsoft unveils Maia 300, its third AI chip

Microsoft published an architectural deep dive on Maia 300, the third generation of its custom AI accelerator for Azure datacentres, built around what the company calls a “Software Defined Locally Accessed Dataflow Architecture.” Each chip is a multi-chiplet design manufactured on TSMC’s N2P process, with compute dies separated from memory-controller and I/O dies, and carries eight stacks of HBM4 memory offering 288GB of capacity and 18 terabytes per second of bandwidth. Microsoft said the design also uses lower-precision number formats to cut the amount of data moved during training and inference.

Microsoft’s own figures put Maia 300 at seven times the rack-scale token throughput of Maia 200 in a full 96-accelerator rack, with two-and-a-half times the performance per watt and memory bandwidth, and a third more memory capacity. A two-tier networking fabric — electrical links within a rack, optical links across racks — can connect up to 12,288 accelerators into a single “superpod,” which Microsoft said lets it place the prefill and decode stages of inference on separate, independently scaled groups of chips.

The post named internal workloads including Copilot, GitHub Copilot and Microsoft Foundry as early users, but did not disclose external customers or a general-availability date. Maia sits alongside Microsoft’s continued purchases of Nvidia and AMD accelerators, part of the wider build-out of AI-specific silicon by cloud providers seeking to reduce their dependence on merchant chip suppliers.