DeepMind's Chinchilla paper rewrites the scaling laws
Existing large models were badly under-trained: for a fixed compute budget, parameters and training tokens should scale together.
Google DeepMindIdeas & essays · Benchmarks & progress · Models & capabilities