Training Language Models to Follow Instructions with Human Feedback
InstructGPT uses human feedback to train language models to follow instructions.
2022
33,638
Works
Selected research papers and books, linked to their authors.
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Examined the environmental costs, biases and risks of misleading fluency in large language models.
2021
18,656
Direct Preference Optimization: Your Language Model is Secretly a Reward Model
DPO trains language models directly on comparisons of preferred and rejected answers.
2023
13,245
Superintelligence: Paths, Dangers, Strategies
A book examining how AI could surpass human intelligence and how its risks might be reduced.
2014
13,195
Deep reinforcement learning from human preferences
Taught AI agents tasks using people's comparisons of their behavior rather than hand-written rewards.
2017
9,013
Concrete Problems in AI Safety
Outlined five practical research problems for preventing accidents in AI systems.
2016
6,581
Constitutional AI: Harmlessness from AI Feedback
Trained AI assistants to critique and improve their answers using a written set of principles.
2022
5,687
The Alignment Problem: Machine Learning and Human Values
A book exploring AI bias, unintended goals and efforts to align machine learning with human values.
2020
1,358
Consciousness in Artificial Intelligence: Insights from the Science of Consciousness
Proposed indicators for assessing AI consciousness, drawn from scientific theories of consciousness.
2023
939
How to Catch an AI Liar: Lie Detection in Black-Box LLMs by Asking Unrelated Questions
Detected deliberately false model answers by asking unrelated follow-up questions.
2024
142