Research Focus

Fundamental hardness of multi-step planning Theory: For problems with long planning horizons, I design algorithms with regrets independent of the horizon, showing that multi-step problems can be no harder than single-step problems.
Instance-dependent regrets Theory: I design algorithms with regrets depending on quantities that characterize each problem instance, while still recovering worst-case guarantees.

Related Publications

Asymptotically Optimal Regret for Reinforcement Learning without Horizon Dependence RL Theory
Sharp Variance-Dependent Bounds in Reinforcement Learning: Best of Both Worlds in Stochastic and Deterministic Environments RL Theory ICML 2023
Stochastic Shortest Path: Minimax, Parameter-Free and Towards Horizon-Free Regret RL Theory NeurIPS 2021 (Spotlight, 3% acceptance rate)

Knowledge acquisition and manipulation: I study failure patterns during knowledge retrieval in LLMs and design theory-inspired approaches to improve performance.

Related Publications

CASCADE Your Datasets for Cross-Mode Knowledge Retrieval of Language Models LLM COLM 2025

Reasoning: I research infrastructure, data, and algorithmic perspectives for building accurate and efficient RL systems for LLMs.
Reinforcement learning from human feedback Theory: I design RLHF algorithms with provably faster convergence rates and better benchmark performance, and study the separation between reward feedback and preference feedback.
Agent: I train LLMs to autonomously explore environments using agentic scaffolds.

Related Publications

RLAX: Large-Scale, Distributed Reinforcement Learning for Large Language Models on TPUs RL LLM
Extragradient Preference Optimization (EGPO): Beyond Last-Iterate Convergence for Nash Learning from Human Feedback RL Theory LLM COLM 2025
The Crucial Role of Samplers in Online Direct Preference Optimization RL Theory LLM ICLR 2025
Reflect-RL: Two-Player Online RL Fine-Tuning for LMs RL LLM ACL 2024