Training Compute-Optimal Large Language Models
At a glance
Chinchilla showed how smaller models trained on more data can better use a fixed computing budget.
In one paragraph
Showed how to balance model size and training data to make better use of a fixed computing budget. Its Chinchilla model outperformed the much larger Gopher model using the same training compute, demonstrating the value of training smaller models on more data.
Authors 22
- Jordan Hoffmann
- Sebastian Borgeaud
- Arthur Mensch
- Elena Buchatskaya
- Trevor Cai
- Eliza Rutherford
- Diego de las Casas
- Lisa Anne Hendricks
- Johannes Welbl
- Aidan Clark
- Tom Hennigan
- Eric Noland
- Katie Millican
- George van den Driessche
- Bogdan Damoc
- Aurelia Guy
- Simon Osindero
- Karén Simonyan
- Erich Elsen
- Jack Rae
- Oriol Vinyals
- Laurent Sifre
