Attention Is All You Need
The Transformer uses attention to translate text while allowing faster training.
2017
294,018
Works
Selected research papers and books, linked to their authors.
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
BERT learns word meaning from the text on both sides.
2018
192,498
Language Models are Few-Shot Learners
GPT-3 performs tasks from prompts and a few examples without task-specific training.
2020
97,522
Latent Dirichlet Allocation
Discovers topics in document collections by treating each document as a mixture of themes.
2003
66,139
Efficient Estimation of Word Representations in Vector Space
Word2vec efficiently learns relationships between words from large text collections.
2013
56,905
GloVe: Global Vectors for Word Representation
GloVe captures relationships between words by learning from how often they occur together.
2014
52,655
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Showed that better training choices could improve BERT without changing its core architecture.
2019
47,647
Neural Machine Translation by Jointly Learning to Align and Translate
Attention helps neural translation models focus on relevant words as each word is generated.
2014
45,398
LoRA: Low-Rank Adaptation of Large Language Models
LoRA adapts large models by training small additions rather than all their parameters.
2022
45,365
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Showed that examples with reasoning steps can help language models solve harder problems.
2022
42,906