Extragradient Preference Optimization (EGPO): Beyond Last-Iterate Convergence for Nash Learning from Human Feedback

RL Theory LLM

Runlong Zhou, Maryam Fazel, Simon S. Du

COLM 2025

We propose EGPO algorithm for Nash learning from human feedback, achieving a last-iterate linear convergence and a simple online IPO implementation.