XLNet: Generalized Autoregressive Pretraining for Language Understanding
XLNet learns language context by predicting words in varied orders.
2019
13,388
Works
Selected research papers and books, linked to their authors.
Direct Preference Optimization: Your Language Model is Secretly a Reward Model
DPO trains language models directly on comparisons of preferred and rejected answers.
2023
13,245
A Neural Probabilistic Language Model
Learned relationships between words with a neural network to better predict the next word.
2003
13,226
Effective Approaches to Attention-based Neural Machine Translation
Improved translation by helping models focus on relevant parts of the source sentence.
2015
12,715
PaLM: Scaling Language Modeling with Pathways
Presented PaLM, a large language model demonstrating gains in reasoning, translation and coding.
2022
10,281
Scaling Laws for Neural Language Models
Measured predictable links between language-model performance, model size, data and computation.
2020
9,547
Emergent Abilities of Large Language Models
Examined abilities that appeared in larger language models but were absent in smaller ones.
2022
7,607
Training Compute-Optimal Large Language Models
Chinchilla showed how smaller models trained on more data can better use a fixed computing budget.
2022
5,834
Constitutional AI: Harmlessness from AI Feedback
Trained AI assistants to critique and improve their answers using a written set of principles.
2022
5,687
Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference
Presented Chatbot Arena, comparing language models through people's votes on anonymous answers.
2024
2,140