AMD launches Instinct MI325X accelerator

AMD announced its Instinct MI325X accelerator at its “Advancing AI” event, its latest attempt to take AI training and inference workloads from Nvidia. The chip carried 256GB of HBM3E memory and 6TB/s of memory bandwidth, which AMD said gave it 1.8 times the memory capacity and 1.3 times the bandwidth of Nvidia’s competing H200, along with 1.3 times the peak theoretical FP16 and FP8 compute.

The MI325X used the same CDNA 3 architecture as AMD’s earlier MI300X, with memory as the main upgrade — a bet that inference workloads, particularly for large context windows and bigger models, were increasingly memory-bound rather than compute-bound. AMD said it had reduced the chip’s memory from an earlier-announced 288GB to 256GB before launch. First production shipments were expected by the end of 2024, with broader availability from server makers including Dell, HPE, Lenovo and Supermicro from early 2025.

AMD’s specification claims were self-reported and not independently benchmarked at announcement. The launch nonetheless marked AMD’s most direct attempt yet to compete on the metric — memory capacity — where it had consistently claimed an edge over Nvidia, as both companies moved to annual accelerator release cadences.