LoRA: Low-Rank Adaptation of Large Language Models
At a glance
LoRA adapts large models by training small additions rather than all their parameters.
In one paragraph
Introduced a way to adapt large language models to new tasks by training small additions while keeping the original model unchanged. LoRA greatly reduces the memory and training needed compared with updating the entire model, without adding delay when the adapted model is used.
Authors 8
- Edward Hu
- Yelong Shen
- Phillip Wallis
- Zeyuan Allen-Zhu
- Yuanzhi Li
- Shean Wang
- Lu Wang
- Weizhu Chen
