Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
At a glance
Expanded network capacity by routing each input to a small set of specialist subnetworks.
In one paragraph
Introduced a way to build much larger neural networks by routing each input to a few specialist subnetworks, increasing capacity without a proportional increase in computation.
Authors 7
- Noam Shazeer
- Azalia Mirhoseini
- Krzysztof Maziarz
- Andy Davis
- Quoc V. Le
- Geoffrey Hinton
- Jeff Dean
