Microsoft and NVIDIA train Megatron-Turing NLG at 530 billion parameters
The largest dense language model publicly described at the time, and a demonstration of multi-thousand-GPU training.
Microsoft, NVIDIAModels & capabilities · Compute & infrastructure