The Year in AI Papers
The year/Topics/RL for reasoning

Research collection

RL for reasoning

Reinforcement learning applied to train reasoning ability, including RLVR, verifiable rewards, reward design, and exploration for reasoning models.

Papers
136
Research labs
7
Official code
85

1–50 of 136 papers in this collection

01

Independent research

DAPD: Dual-Anchored Policy Distillation

The paper identifies information asymmetry as the root cause of privilege illusion in on-policy self-distillation (OPSD) for language models. In OPSD, a teacher conditioned on privileged information (e.g., a reference completion) supervises a student that lacks this information at inference, causing the student to behave as if the privileged information…

Jianyu Wu, Yizhou Wang, Encheng Su, Chen Tang, et al.
Published
Aug 2026
Citations
0
Code
31 stars
Read summary Original
02

Independent research

CoRT: Counterfactual Replay for Token-Level Rubric-Guided Policy Optimization

CoRT (Counterfactual Replay for Token-level credit weighting) is a method for rubric-conditioned GRPO that addresses the uniform token-level credit assignment in standard GRPO. It rescores the same sampled response under a criteria-free prompt, computes tokenwise log-likelihood contrasts, and maps these to bounded, response-normalized weights. These…

Bo-Wen Zhang, Junwei He, Wen Wang, Song-Lin Lv, et al.
Published
Jul 2026
Citations
0
Code
Not linked
Read summary Original
03

Independent research

From RLVR to RLSVR: Task Transformation Induces Self-Verifiable Rewards for Open-Ended LLM Self-Improvement

This paper introduces RLSVR (Reinforcement Learning with Self-Verifiable Rewards), a training paradigm that extends RLVR to open-ended tasks by transforming them into proxy environments with automatically verifiable rewards, inspired by self-supervised learning. The authors instantiate RLSVR with SpyRL, an information-asymmetric self-play framework based…

Qinsi Wang, Jing Shi, Huazheng Wang, Kun Wan, et al.
Published
Jul 2026
Citations
0
Code
165 stars
Read summary Original
04

Independent research

LongStraw: Long-Context RL Beyond 2M Tokens under a Fixed GPU Budget

LongStraw is a system for long-context GRPO post-training under fixed GPU budgets, addressing the memory bottleneck of multi-million-token prompts by virtualizing resident state, replaying responses, and executing distributed gradients. It captures the shared prompt without autograd, retains only architecture-required state (e.g., recurrent state, KV…

Changhai Zhou, Kieran Liu, Yuhua Zhou, Qian Qiao, et al.
Published
Jul 2026
Citations
0
Code
43 stars
Read summary Original
05

Independent research

SEED: Self-Evolving On-Policy Distillation for Agentic Reinforcement Learning

SEED (SElf-Evolving On-Policy Distillation) is a framework for agentic reinforcement learning that converts completed on-policy trajectories into natural-language hindsight skills and distills their behavioral effect back into the policy model. It addresses the supervision gap between sparse trajectory-level rewards and token-level policy learning in…

Jinyang Wu, Shuo Yang, Zhengxi Lu, Fan Zhang, et al.
Published
Jul 2026
Citations
3
Code
218 stars
Read summary Original
06

Independent research

Ring-Zero: Scaling Zero RL to a Trillion Parameters for Emergent Reasoning

The paper presents Ring-Zero, a stable and efficient pipeline for scaling zero reinforcement learning (RL) with verifiable rewards to a trillion-parameter model (Ling-2.5-1T-Base, 1T parameters, 63B activated) without human-annotated data. The pipeline includes three RL stages and a self-distillation phase, using clipped importance sampling,…

Xinyu Tang, Qianggang Cao, Yurou Liu, Yuliang Zhan, et al.
Published
Jul 2026
Citations
0
Code
Not linked
Read summary Original
07

Independent research

Read It Back: Pretrained MLLMs Are Zero-Shot Reward Models for Text-to-Image Generation

The paper introduces SpectraReward, a training-free reward function for text-to-image reinforcement learning (RL) that converts pretrained multimodal large language models (MLLMs) into reward models. Instead of asking the MLLM to judge images or answer questions, SpectraReward measures how well the original prompt can be recovered from a generated image…

Runhui Huang, Qihui Zhang, Zhe Liu, Yu Gao, et al.
Published
Jul 2026
Citations
0
Code
Not linked
Read summary Original
08

Independent research

Weak-to-Strong Generalization via Direct On-Policy Distillation

The paper introduces Direct On-Policy Distillation (Direct-OPD), a method to transfer the policy shift induced by reinforcement learning (RL) on a small, weak teacher model to a stronger student model, avoiding the high cost of running RL directly on the larger model. Instead of imitating the post-RL teacher's final policy, Direct-OPD uses the log-ratio…

Shiyuan Feng, Huan-ang Gao, Haohan Chi, Hanlin Wu, et al.
Published
Jul 2026
Citations
4
Code
Not linked
Read summary Original
09

Independent research

OmniOpt: Taxonomy, Geometry, and Benchmarking of Modern Optimizers

OmniOpt is a survey and benchmark framework for modern optimizers, addressing the fragmented landscape of over one hundred methods. It introduces a five-stage meta-pipeline (S0-S5) to describe optimizer updates, and a four-axis decomposition (update domain, state estimator, geometry/precondition operator, finalization) based on linear minimization oracles…

Siyuan Li, Jiabao Pan, Yumou Liu, Zhuoli Ouyang, et al.
Published
Jul 2026
Citations
0
Code
36 stars
Read summary Original
10

Independent research

DOPD: Dual On-policy Distillation

The paper introduces DOPD (Dual On-policy Distillation), an advantage-aware dual distillation paradigm for large language models (LLMs) and vision-language models (VLMs). It addresses a failure mode called 'privilege illusion,' where adding privileged information to teacher or student policies creates an apparent performance gap that stems from information…

Xinlei Yu, Gen Li, Qingyi Si, Guibin Zhang, et al.
Published
Jun 2026
Citations
1
Code
Not linked
Read summary Original
11

Independent research

The Mirage of Optimizing Training Policies: Monotonic Inference Policies as the Real Objective for LLM Reinforcement Learning

The paper addresses training-inference mismatch in LLM reinforcement learning, where separate training and inference engines produce inconsistent probabilities for the same trajectories, causing off-policyness and training instability. The authors identify an objective misalignment: improving the training policy does not guarantee improvement of the…

Jing Liang, Hongyao Tang, Yi Ma, Yancheng He, et al.
Published
Jun 2026
Citations
0
Code
Not linked
Read summary Original
12

NVIDIA

Zone of Proximal Policy Optimization: Teacher in Prompts, Not Gradients

The paper introduces Zone of Proximal Policy Optimization (ZPPO), a post-training method for small vision-language models (VLMs) that transfers knowledge from a larger teacher without imitating its logits or injecting its responses into the policy gradient. ZPPO addresses two failure modes: distillation's brittleness in the small-student regime and RL's…

Byung-Kwan Lee, Ximing Lu, Shizhe Diao, Minki Kang, et al.
Published
Jun 2026
Citations
2
Code
Not linked
Read summary Original
13

Independent research

Learning from the Self-future: On-policy Self-distillation for dLLMs

This paper introduces d-OPSD, the first on-policy self-distillation (OPSD) framework tailored for diffusion large language models (dLLMs). Existing OPSD methods are autoregressive-centric, using left-to-right prefix conditioning and token-level supervision, which conflicts with dLLMs' arbitrary-order generation. d-OPSD makes two key contributions: it…

Yifu Luo, Zeyu Chen, Haoyu Wang, Xinhao Hu, et al.
Published
Jun 2026
Citations
1
Code
16 stars
Read summary Original
14

Independent research

VibeThinker-3B: Exploring the Frontier of Verifiable Reasoning in Small Language Models

VibeThinker-3B is a 3B-parameter dense model from Sina Weibo Inc. that achieves frontier-level performance on verifiable reasoning tasks, matching or exceeding much larger models. Built on Qwen2.5-Coder-3B, it uses a post-training pipeline with curriculum SFT, multi-domain RL (math, code, STEM), Long2Short Math RL for efficiency, offline self-distillation,…

Sen Xu, Shixi Liu, Wei Wang, Jixin Min, et al.
Published
Jun 2026
Citations
1
Code
1.5K stars
Read summary Original
15

MiniMax

MaxProof: Scaling Mathematical Proof with Generative-Verifier RL and Population-Level Test-Time Scaling

MaxProof is a population-level test-time scaling framework for competition-level mathematical proof, developed for the MiniMax-M3 model series. The M3 model is trained with three proof-oriented capabilities: proof generation via verifier-guided reinforcement learning, proof verification through aligned error finding, and critique-conditioned proof repair.…

Jiacheng Chen, Xinyu Zhang, Shunkai Zhang, Yanmohan Wang, et al.
Published
Jun 2026
Citations
1
Code
Not linked
Read summary Original
16

Independent research

APPO: Agentic Procedural Policy Optimization

The paper introduces APPO (Agentic Procedural Policy Optimization), a reinforcement learning algorithm for LLM agents that shifts branching and credit assignment from coarse units like tool-call boundaries to fine-grained decision points in the generated sequence. A pilot study showed that influential decision points are distributed throughout the thinking…

Xucong Wang, Ziyu Ma, Yong Wang, Yuxiang Ji, et al.
Published
Jun 2026
Citations
0
Code
73 stars
Read summary Original
17

Independent research

Z-Reward: Beyond Scalar Rewards by Internalizing Reasoning into Score Distributions

Z-Reward is a teacher-student framework for text-to-image reward modeling that represents visual preference as a reasoning-conditioned score distribution rather than a scalar. The teacher, a 27B VLM, is trained with Group-wise Direct Score Optimization (GDSO), which combines GRPO-style policy-gradient rewards with direct supervision on score distributions…

Xin Jin, Huanqia Cai, Zhen Li, Zechao Zhan, et al.
Published
Jun 2026
Citations
0
Code
Not linked
Read summary Original
18

arXiv.org

Trust-Region Behavior Blending for On-Policy Distillation

The paper introduces Trust-Region behavior Blending (TRB), a warmup method for on-policy distillation (OPD) that addresses the issue of poor early student rollouts. TRB replaces the early rollout policy with a teacher-guided behavior policy that is constrained to stay within a student-centered KL trust region, while keeping the per-prefix reverse-KL OPD…

Daniil Plyusov, Alexey Gorbatovski, Alexey Malakhov, Nikita Balagansky, et al.
Published
May 2026
Citations
0
Code
Not linked
Read summary Original
19

NVIDIA

Agent Explorative Policy Optimization for Multimodal Agentic Reasoning

The paper introduces AXPO (Agent eXplorative Policy Optimization) to address the Thinking-Acting Gap in multimodal agentic reasoning, where tool use is under-trained compared to thinking. Under GRPO, tool use occurs in only ~30% of rollouts, and tool-using subgroups are all-wrong on ~40% of questions, suppressing learning signals. AXPO fixes the thinking…

Minki Kang, Shizhe Diao, Ryo Hachiuma, Sung Ju Hwang, et al.
Published
May 2026
Citations
2
Code
Not linked
Read summary Original
20

arXiv.org

ProRL: Effective Reinforcement Learning for Proactive Recommendation via Rectified Policy Gradient Estimation

The paper introduces ProRL, a reinforcement learning framework for proactive recommender systems (PRS) that guide user preferences toward target items via intermediate recommendation paths. The authors identify two deficiencies in standard policy gradient estimation for PRS: (1) a length shortcut, where positive-mean step-level rewards cause gradients to…

Hongru Hou, Tiehua Mei, Denghui Geng, Jinhui Huang, et al.
Published
May 2026
Citations
0
Code
46 stars
Read summary Original
21

arXiv.org

DVAO: Dynamic Variance-adaptive Advantage Optimization for Multi-reward Reinforcement Learning

The paper proposes Dynamic Variance-adaptive Advantage Optimization (DVAO) to improve multi-reward reinforcement learning for LLMs, addressing flaws in standard scalarization methods. Reward Combination (RC) causes training instability due to large advantage magnitudes, while Advantage Combination (AC) uses static weights and ignores cross-objective…

Guochao Jiang, Jingyi Song, Guofeng Quan, Chuzhan Hao, et al.
Published
May 2026
Citations
1
Code
Not linked
Read summary Original
22

arXiv.org

DelTA: Discriminative Token Credit Assignment for Reinforcement Learning from Verifiable Rewards

The paper introduces DelTA, a method for improving reinforcement learning from verifiable rewards (RLVR) in large language models. The authors show that sequence-level RLVR updates act as an implicit linear discriminator over token-gradient vectors, determining which token probabilities increase or decrease. Standard RLVR forms this discriminator using…

Kaiyi Zhang, Wei Wu, Yankai Lin
Published
May 2026
Citations
0
Code
17 stars
Read summary Original
23

arXiv.org

GoLongRL: Capability-Oriented Long Context Reinforcement Learning with Multitask Alignment

GoLongRL is a fully open-source, capability-oriented post-training recipe for long-context reinforcement learning with verifiable rewards (RLVR). It introduces a dataset of 22,965 samples spanning 9 task types, each paired with its natural evaluation metric (e.g., EM, F1, NDCG), and a four-phase construction pipeline. The dataset combines curated…

Minxuan Lv, Tiehua Mei, Tanlong Du, Junmin Chen, et al.
Published
May 2026
Citations
2
Code
75 stars
Read summary Original
24

arXiv.org

Achieving Gold-Medal-Level Olympiad Reasoning via Simple and Unified Scaling

The paper introduces SU-01, a 30B-A3B reasoning model achieving gold-medal-level performance on mathematical and physical olympiads through a simple, unified post-training recipe. The recipe comprises three stages: (1) SFT with a reverse-perplexity curriculum on 338K rigorous proof trajectories to instill proof-search and self-checking behaviors; (2) a…

Yafu Li, Runzhe Zhan, Haoran Zhang, Shunkai Zhang, et al.
Published
May 2026
Citations
4
Code
94 stars
Read summary Original
25

arXiv.org

Learning to Foresee: Unveiling the Unlocking Efficiency of On-Policy Distillation

This paper investigates why on-policy distillation (OPD) is more efficient than reinforcement learning (RL) for post-training large language models. The authors identify two properties of OPD's parameter updates: Functional Redundancy Avoidance, where OPD suppresses updates in low-utility modules (e.g., embeddings, peripheral layers) and concentrates on…

Yuchen Cai, Ding Cao, Liang Lin, Chunxi Luo, et al.
Published
May 2026
Citations
4
Code
73 stars
Read summary Original
26

arXiv.org

Anti-Self-Distillation for Reasoning RL via Pointwise Mutual Information

The paper identifies a structural bias in on-policy self-distillation for math reasoning, where the per-token signal (conditional pointwise mutual information between the next token and privileged context) rewards shortcut tokens (e.g., 'Given', 'succeeds') and penalizes deliberation tokens (e.g., 'Wait', 'Let'). To fix this, the authors propose…

Guobin Shen, Xiang Cheng, Chenxiao Zhao, Lei Huang, et al.
Published
May 2026
Citations
6
Code
33 stars
Read summary Original
27

Google DeepMind

RubricEM: Meta-RL with Rubric-guided Policy Decomposition beyond Verifiable Rewards

RubricEM is a reinforcement learning framework for training deep research agents on open-ended tasks where verifiable rewards are unavailable. It treats rubrics as a shared interface structuring policy execution, judge feedback, and agent memory. The framework decomposes trajectories into four rubric-guided stages (Plan, Research, Review, Answer) and uses…

Gaotang Li, Bhavana Dalvi Mishra, Zifeng Wang, Jun Yan, et al.
Published
May 2026
Citations
3
Code
Not linked
Read summary Original
28

arXiv.org

Flow-OPD: On-Policy Distillation for Flow Matching Models

Flow-OPD is a novel post-training framework for Flow Matching text-to-image models, addressing reward sparsity and gradient interference in multi-task alignment. It uses a two-stage strategy: first, domain-specialized teachers are trained via single-reward GRPO; then, a student model is initialized via a Flow-based Cold-Start (SFT or model merging) and…

Zhen Fang, Wenxuan Huang, Yu Zeng, Yiming Zhao, et al.
Published
May 2026
Citations
11
Code
274 stars
Read summary Original
29

arXiv.org

Listwise Policy Optimization: Group-based RLVR as Target-Projection on the LLM Response Simplex

This paper introduces Listwise Policy Optimization (LPO), a framework for reinforcement learning with verifiable rewards (RLVR) in large language models. The authors show that existing group-based policy gradient methods (e.g., GRPO, Dr.GRPO, MaxRL) implicitly perform approximate target-projection on a response simplex, where the target is a…

Yun Qu, Qi Wang, Yixiu Mao, Heming Zou, et al.
Published
May 2026
Citations
1
Code
Not linked
Read summary Original
30

arXiv.org

Leveraging Verifier-Based Reinforcement Learning in Image Editing

The paper introduces Edit-R1, a framework for image editing that uses a verifier-based Reasoning Reward Model (RRM) trained with reinforcement learning. The RRM decomposes editing instructions into principles, evaluates edited images against each, and produces interpretable scores. Training involves a cold-start SFT phase with filtered chain-of-thought…

Hanzhong Guo, Jie Wu, Jie Liu, Yu Gao, et al.
Published
Apr 2026
Citations
5
Code
Not linked
Read summary Original
31

arXiv.org

Co-Evolving Policy Distillation

The paper proposes Co-Evolving Policy Distillation (CoPD) to consolidate multiple expert capabilities into a single model. It analyzes two standard paradigms: mixed RLVR, which suffers from capability divergence cost, and the static pipeline of training experts then performing on-policy distillation (OPD), which fails to fully absorb teacher capabilities…

Naibin Gu, Chenxu Yang, Qingyi Si, Chuanyu Qin, et al.
Published
Apr 2026
Citations
3
Code
Not linked
Read summary Original
32

arXiv.org

World-R1: Reinforcing 3D Constraints for Text-to-Video Generation

World-R1 is a framework that improves 3D consistency in text-to-video generation by using reinforcement learning (RL) instead of architectural changes. It builds on the Wan 2.1 model and uses Flow-GRPO to optimize the model with rewards from 3D foundation models (Depth Anything 3) and vision-language models (Qwen3-VL). The reward system includes meta-view…

Weijie Wang, Xiaoxuan He, Youping Gu, Yifan Yang, et al.
Published
Apr 2026
Citations
7
Code
414 stars
Read summary Original
33

arXiv.org

Near-Future Policy Optimization

The paper introduces Near-Future Policy Optimization (NPO), a mixed-policy reinforcement learning with verifiable rewards (RLVR) method that improves training by using trajectories from a near-future checkpoint of the same training run. The authors formalize a quality-variance trade-off, defining effective learning signal S = Q/V, where Q is the fraction…

Chuanyu Qin, Chenxu Yang, Qingyi Si, Naibin Gu, et al.
Published
Apr 2026
Citations
2
Code
Not linked
Read summary Original
34

arXiv.org

DiPO: Disentangled Perplexity Policy Optimization for Fine-grained Exploration-Exploitation Trade-Off

The paper introduces DiPO (Disentangled Perplexity Policy Optimization), a method for fine-grained exploration-exploitation trade-off in Reinforcement Learning with Verifiable Rewards (RLVR) for LLMs. It identifies two dilemmas in GRPO-based methods: extreme sample groups (all-correct or all-error) yield zero advantage, and perplexity (PPL) distribution…

Xiaofan Li, Ming Yang, Zhiyuan Ma, Shichao Ma, et al.
Published
Apr 2026
Citations
0
Code
Not linked
Read summary Original
35

arXiv.org

KnowRL: Boosting LLM Reasoning via Reinforcement Learning with Minimal-Sufficient Knowledge Guidance

KnowRL is a reinforcement learning (RL) training framework that addresses reward sparsity in large language model (LLM) reasoning by treating hint design as a minimal-sufficient guidance problem. Instead of injecting long solution prefixes or abstract templates, KnowRL decomposes guidance into atomic knowledge points (KPs) and selects compact,…

Linhao Yu, Tianmeng Yang, Siyu Ding, Renren Jin, et al.
Published
Apr 2026
Citations
8
Code
58 stars
Read summary Original
36

arXiv.org

The Past Is Not Past: Memory-Enhanced Dynamic Reward Shaping

The paper introduces MEDS, a framework that enhances reinforcement learning for large language models by dynamically penalizing recurrent error patterns. It addresses the issue of reduced sampling diversity, where policies repeatedly generate similar erroneous behaviors. MEDS stores historical behavioral signals by reusing layer-wise logits from the final…

Yang Liu, Enxi Wang, Yufei Gao, Weixin Zhang, et al.
Published
Apr 2026
Citations
1
Code
142 stars
Read summary Original
37

arXiv.org

RAGEN-2: Reasoning Collapse in Agentic RL

RAGEN-2 identifies a failure mode in multi-turn LLM agent reinforcement learning called template collapse, where reasoning appears diverse within inputs but becomes input-agnostic across inputs, invisible to entropy-based metrics. The authors decompose reasoning quality into within-input diversity (conditional entropy) and cross-input distinguishability…

Zihan Wang, Chi Gui, Xing Jin, Qineng Wang, et al.
Published
Apr 2026
Citations
10
Code
2.8K stars
Read summary Original
38

arXiv.org

Self-Distilled RLVR

The paper analyzes on-policy self-distillation (OPSD) for LLM post-training, where a model serves as both teacher and student, with the teacher using privileged information. It proves that OPSD's information asymmetry creates an irreducible mutual information gap, causing privileged information leakage and performance degradation after early gains. To…

Chenxu Yang, Chuanyu Qin, Qingyi Si, Minghui Chen, et al.
Published
Apr 2026
Citations
56
Code
Not linked
Read summary Original
39

arXiv.org

GrandCode: Achieving Grandmaster Level in Competitive Programming via Agentic Reinforcement Learning

GrandCode is a multi-agent reinforcement learning (RL) system for competitive programming, introduced by the DeepReinforce Team. It orchestrates modules for hypothesis proposal, solution generation, summarization, and test-case generation, jointly optimized via post-training and online test-time RL. A key innovation is Agentic GRPO, which combines…

DeepReinforce Team, Xiaoya Li, Guoyin Wang, Songqiao Su, et al.
Published
Apr 2026
Citations
3
Code
Not linked
Read summary Original
40

arXiv.org

All Roads Lead to Rome: Incentivizing Divergent Thinking in Vision-Language Models

This paper investigates the behavioral differences between Reinforcement Learning (RL) and base Vision-Language Models (VLMs). The authors find that RL models, such as those trained with Group Relative Policy Optimization (GRPO), exhibit deeper but narrower reasoning, while base models show broader and more diverse thinking patterns. They identify a…

Xinyu Tian, Shu Zou, Zhaoyuan Yang, Mengqi He, et al.
Published
Apr 2026
Citations
0
Code
148 stars
Read summary Original
41

arXiv.org

Why Does Self-Distillation (Sometimes) Degrade the Reasoning Capability of LLMs?

This paper investigates why self-distillation can degrade the mathematical reasoning performance of LLMs, despite often improving other domains. The authors find that self-distillation suppresses 'epistemic verbalization'—the expression of uncertainty during reasoning—which is crucial for robust problem-solving. Through controlled experiments, they show…

Jeonghye Kim, Xufang Luo, Minbeom Kim, Sangmook Lee, et al.
Published
Mar 2026
Citations
61
Code
75 stars
Read summary Original
42

arXiv.org

LongCat-Flash-Prover: Advancing Native Formal Reasoning via Agentic Tool-Integrated Reinforcement Learning

LongCat-Flash-Prover is a 560-billion-parameter open-source Mixture-of-Experts (MoE) model for native formal reasoning in Lean4, developed by Meituan LongCat Team. It decomposes formal reasoning into auto-formalization, sketching, and proving, and uses a Hybrid-Experts Iteration Framework to synthesize high-quality trajectories with tool feedback. The…

Jianing Wang, Jianfei Zhang, Qi Guo, Linsen Guo, et al.
Published
Mar 2026
Citations
5
Code
93 stars
Read summary Original
43

Qwen

FIPO: Eliciting Deep Reasoning with Future-KL Influenced Policy Optimization

FIPO (Future-KL Influenced Policy Optimization) is a reinforcement learning algorithm that addresses the coarse-grained credit assignment problem in GRPO-style training for large language models. By incorporating a discounted Future-KL divergence into the policy update, FIPO re-weights token advantages based on their influence on subsequent trajectory…

Chiyu Ma, Shuo Yang, Kexin Huang, Jinda Lu, et al.
Published
Mar 2026
Citations
18
Code
130 stars
Read summary Original
44

NVIDIA

Nemotron-Cascade 2: Post-Training LLMs with Cascade RL and Multi-Domain On-Policy Distillation

Nemotron-Cascade 2 is an open 30B Mixture-of-Experts (MoE) model with 3B activated parameters, achieving best-in-class reasoning and agentic capabilities. It is the second open-weight LLM to achieve Gold Medal-level performance in the 2025 IMO, IOI, and ICPC World Finals, with 20x fewer parameters than DeepSeek-V3.2-Speciale-671B-A37B. The model builds on…

Zhuolin Yang, Zihan Liu, Yang Chen, Wenliang Dai, et al.
Published
Mar 2026
Citations
20
Code
Not linked
Read summary Original
45

arXiv.org

Astrolabe: Steering Forward-Process Reinforcement Learning for Distilled Autoregressive Video Models

Astrolabe is an online reinforcement learning (RL) framework designed to align distilled autoregressive (AR) video models with human preferences without re-distillation. It addresses limitations of existing RL methods, which require expensive reverse-process optimization or lack active exploration. Astrolabe uses a forward-process RL formulation based on…

Songchun Zhang, Zeyue Xue, Siming Fu, Jie Huang, et al.
Published
Mar 2026
Citations
10
Code
151 stars
Read summary Original
46

arXiv.org

TRUST-SQL: Tool-Integrated Multi-Turn Reinforcement Learning for Text-to-SQL over Unknown Schemas

TRUST-SQL addresses Text-to-SQL parsing under the Unknown Schema setting, where databases have hundreds of tables and noisy metadata, making full schema prefilling impractical. The task is formulated as a Partially Observable Markov Decision Process, and the agent follows a four-phase protocol (Explore, Propose, Generate, Confirm) to ground reasoning in…

Ai Jian, Xiaoyun Zhang, Wanrou Du, Jingqing Ruan, et al.
Published
Mar 2026
Citations
3
Code
Not linked
Read summary Original
47

arXiv.org

Video-CoE: Reinforcing Video Event Prediction via Chain of Events

The paper introduces Video-CoE, a method to improve video event prediction (VEP) in multimodal large language models (MLLMs). The authors first evaluate leading MLLMs on VEP benchmarks, identifying two main failure causes: lack of logical reasoning for future events and insufficient use of visual information. To address these, they propose the Chain of…

Qile Su, Jing Tang, Rui Chen, Lei Sun, et al.
Published
Mar 2026
Citations
2
Code
Not linked
Read summary Original
48

arXiv.org

AI Can Learn Scientific Taste

The paper introduces Reinforcement Learning from Community Feedback (RLCF), a paradigm that uses citation-based community signals to train AI models for scientific taste—the ability to judge and propose high-impact research ideas. They built SciJudgeBench with 720K field- and time-matched paper abstract pairs, trained Scientific Judge via GRPO to predict…

Jingqi Tong, Mingzhe Li, Hangcheng Li, Yongzhuo Yang, et al.
Published
Mar 2026
Citations
4
Code
428 stars
Read summary Original
49

NVIDIA

MM-Zero: Self-Evolving Multi-Model Vision Language Models From Zero Data

MM-Zero is a reinforcement learning framework that enables zero-data self-evolution for Vision Language Models (VLMs). It introduces a tri-role system—Proposer, Coder, and Solver—all initialized from the same base model and trained sequentially using Group Relative Policy Optimization (GRPO). The Proposer generates visual descriptions and questions, the…

Zongxia Li, Hongyang Du, Chengsong Huang, Xiyang Wu, et al.
Published
Mar 2026
Citations
12
Code
79 stars
Read summary Original
50

arXiv.org

How Far Can Unsupervised RLVR Scale LLM Training?

This paper investigates the scalability of Unsupervised Reinforcement Learning with Verifiable Rewards (URLVR) for LLM training. It categorizes URLVR methods into intrinsic (certainty-based and ensemble-based) and external rewards. The authors establish a unified theoretical framework showing that all intrinsic methods converge to sharpening the model's…

Bingxiang He, Yuxin Zuo, Zeyuan Liu, Shangziqi Zhao, et al.
Published
Mar 2026
Citations
24
Code
1.1K stars
Read summary Original