Entry

Learning to summarize from human feedback

Nisan Stiennon, Long Ouyang, Jeff Wu, Daniel M. Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, Paul Christiano

Synopsis

Trains summarization policies against a reward model fit to human comparisons. Resulting summaries beat reference summaries and supervised models 10x larger.

Keywords

·RLHF ·summarization ·preference learning ·reward model

Open paper ↗ arXiv ↗ Report issue ↗

Related entries

Deep Reinforcement Learning from Human Preferences

June 12, 2017 · NeurIPS 2017
How RLHF Amplifies Sycophancy

February 1, 2026 · arXiv
Strategyproof Reinforcement Learning from Human Feedback

March 12, 2025 · NeurIPS 2025
Language Models Learn to Mislead Humans via RLHF

September 19, 2024 · ICLR 2025
Towards Understanding Sycophancy in Language Models

October 20, 2023 · ICLR 2024
Training Language Models to Follow Instructions with Human Feedback

March 4, 2022 · NeurIPS 2022