The Year in AI Papers
The year/Topics/Reasoning

Topic area

Reasoning

Every collection across reasoning.

Papers
177
Research labs
7
Official code
113

1–50 of 177 papers in this topic area

01

Independent research

DAPD: Dual-Anchored Policy Distillation

The paper identifies information asymmetry as the root cause of privilege illusion in on-policy self-distillation (OPSD) for language models. In OPSD, a teacher conditioned on privileged information (e.g., a reference completion) supervises a student that lacks this information at inference, causing the student to behave as if the privileged information…

Jianyu Wu, Yizhou Wang, Encheng Su, Chen Tang, et al.
Published
Aug 2026
Citations
0
Code
31 stars
Read summary Original
02

Independent research

Mental World Modeling

The paper introduces Mental World Modeling (MWM), a framework for world models that jointly represent physical and mental dynamics to predict human decisions. MWM maintains a coupled physical-mental world state, renders a target-specific partial observation, and simulates how candidate actions update both components. The authors implement MENTIS, a…

Hao Fei, Yiran Zhao
Published
Jul 2026
Citations
0
Code
16 stars
Read summary Original
03

Independent research

CoRT: Counterfactual Replay for Token-Level Rubric-Guided Policy Optimization

CoRT (Counterfactual Replay for Token-level credit weighting) is a method for rubric-conditioned GRPO that addresses the uniform token-level credit assignment in standard GRPO. It rescores the same sampled response under a criteria-free prompt, computes tokenwise log-likelihood contrasts, and maps these to bounded, response-normalized weights. These…

Bo-Wen Zhang, Junwei He, Wen Wang, Song-Lin Lv, et al.
Published
Jul 2026
Citations
0
Code
Not linked
Read summary Original
04

Independent research

From RLVR to RLSVR: Task Transformation Induces Self-Verifiable Rewards for Open-Ended LLM Self-Improvement

This paper introduces RLSVR (Reinforcement Learning with Self-Verifiable Rewards), a training paradigm that extends RLVR to open-ended tasks by transforming them into proxy environments with automatically verifiable rewards, inspired by self-supervised learning. The authors instantiate RLSVR with SpyRL, an information-asymmetric self-play framework based…

Qinsi Wang, Jing Shi, Huazheng Wang, Kun Wan, et al.
Published
Jul 2026
Citations
0
Code
165 stars
Read summary Original
05

Independent research

LongStraw: Long-Context RL Beyond 2M Tokens under a Fixed GPU Budget

LongStraw is a system for long-context GRPO post-training under fixed GPU budgets, addressing the memory bottleneck of multi-million-token prompts by virtualizing resident state, replaying responses, and executing distributed gradients. It captures the shared prompt without autograd, retains only architecture-required state (e.g., recurrent state, KV…

Changhai Zhou, Kieran Liu, Yuhua Zhou, Qian Qiao, et al.
Published
Jul 2026
Citations
0
Code
43 stars
Read summary Original
06

Independent research

SEED: Self-Evolving On-Policy Distillation for Agentic Reinforcement Learning

SEED (SElf-Evolving On-Policy Distillation) is a framework for agentic reinforcement learning that converts completed on-policy trajectories into natural-language hindsight skills and distills their behavioral effect back into the policy model. It addresses the supervision gap between sparse trajectory-level rewards and token-level policy learning in…

Jinyang Wu, Shuo Yang, Zhengxi Lu, Fan Zhang, et al.
Published
Jul 2026
Citations
3
Code
218 stars
Read summary Original
07

Independent research

Ring-Zero: Scaling Zero RL to a Trillion Parameters for Emergent Reasoning

The paper presents Ring-Zero, a stable and efficient pipeline for scaling zero reinforcement learning (RL) with verifiable rewards to a trillion-parameter model (Ling-2.5-1T-Base, 1T parameters, 63B activated) without human-annotated data. The pipeline includes three RL stages and a self-distillation phase, using clipped importance sampling,…

Xinyu Tang, Qianggang Cao, Yurou Liu, Yuliang Zhan, et al.
Published
Jul 2026
Citations
0
Code
Not linked
Read summary Original
08

Independent research

Read It Back: Pretrained MLLMs Are Zero-Shot Reward Models for Text-to-Image Generation

The paper introduces SpectraReward, a training-free reward function for text-to-image reinforcement learning (RL) that converts pretrained multimodal large language models (MLLMs) into reward models. Instead of asking the MLLM to judge images or answer questions, SpectraReward measures how well the original prompt can be recovered from a generated image…

Runhui Huang, Qihui Zhang, Zhe Liu, Yu Gao, et al.
Published
Jul 2026
Citations
0
Code
Not linked
Read summary Original
09

Independent research

Weak-to-Strong Generalization via Direct On-Policy Distillation

The paper introduces Direct On-Policy Distillation (Direct-OPD), a method to transfer the policy shift induced by reinforcement learning (RL) on a small, weak teacher model to a stronger student model, avoiding the high cost of running RL directly on the larger model. Instead of imitating the post-RL teacher's final policy, Direct-OPD uses the log-ratio…

Shiyuan Feng, Huan-ang Gao, Haohan Chi, Hanlin Wu, et al.
Published
Jul 2026
Citations
4
Code
Not linked
Read summary Original
10

Independent research

OmniOpt: Taxonomy, Geometry, and Benchmarking of Modern Optimizers

OmniOpt is a survey and benchmark framework for modern optimizers, addressing the fragmented landscape of over one hundred methods. It introduces a five-stage meta-pipeline (S0-S5) to describe optimizer updates, and a four-axis decomposition (update domain, state estimator, geometry/precondition operator, finalization) based on linear minimization oracles…

Siyuan Li, Jiabao Pan, Yumou Liu, Zhuoli Ouyang, et al.
Published
Jul 2026
Citations
0
Code
36 stars
Read summary Original
11

Independent research

DOPD: Dual On-policy Distillation

The paper introduces DOPD (Dual On-policy Distillation), an advantage-aware dual distillation paradigm for large language models (LLMs) and vision-language models (VLMs). It addresses a failure mode called 'privilege illusion,' where adding privileged information to teacher or student policies creates an apparent performance gap that stems from information…

Xinlei Yu, Gen Li, Qingyi Si, Guibin Zhang, et al.
Published
Jun 2026
Citations
1
Code
Not linked
Read summary Original
12

Independent research

The Mirage of Optimizing Training Policies: Monotonic Inference Policies as the Real Objective for LLM Reinforcement Learning

The paper addresses training-inference mismatch in LLM reinforcement learning, where separate training and inference engines produce inconsistent probabilities for the same trajectories, causing off-policyness and training instability. The authors identify an objective misalignment: improving the training policy does not guarantee improvement of the…

Jing Liang, Hongyao Tang, Yi Ma, Yancheng He, et al.
Published
Jun 2026
Citations
0
Code
Not linked
Read summary Original
13

NVIDIA

Zone of Proximal Policy Optimization: Teacher in Prompts, Not Gradients

The paper introduces Zone of Proximal Policy Optimization (ZPPO), a post-training method for small vision-language models (VLMs) that transfers knowledge from a larger teacher without imitating its logits or injecting its responses into the policy gradient. ZPPO addresses two failure modes: distillation's brittleness in the small-student regime and RL's…

Byung-Kwan Lee, Ximing Lu, Shizhe Diao, Minki Kang, et al.
Published
Jun 2026
Citations
2
Code
Not linked
Read summary Original
14

Independent research

Learning from the Self-future: On-policy Self-distillation for dLLMs

This paper introduces d-OPSD, the first on-policy self-distillation (OPSD) framework tailored for diffusion large language models (dLLMs). Existing OPSD methods are autoregressive-centric, using left-to-right prefix conditioning and token-level supervision, which conflicts with dLLMs' arbitrary-order generation. d-OPSD makes two key contributions: it…

Yifu Luo, Zeyu Chen, Haoyu Wang, Xinhao Hu, et al.
Published
Jun 2026
Citations
1
Code
16 stars
Read summary Original
15

Independent research

LoopCoder-v2: Only Loop Once for Efficient Test-Time Computation Scaling

The paper investigates loop-count selection in Parallel Loop Transformers (PLT), which use cross-loop position offsets (CLP) and shared-KV gated sliding-window attention to keep latency and memory constant regardless of loop count. The authors train LoopCoder-v2, a 7B PLT coder, from scratch on 18T tokens with loop counts R=1,2,3,4, under matched training…

Jian Yang, Shawn Guo, Wei Zhang, Tianyu Zheng, et al.
Published
Jun 2026
Citations
1
Code
Not linked
Read summary Original
16

Independent research

VibeThinker-3B: Exploring the Frontier of Verifiable Reasoning in Small Language Models

VibeThinker-3B is a 3B-parameter dense model from Sina Weibo Inc. that achieves frontier-level performance on verifiable reasoning tasks, matching or exceeding much larger models. Built on Qwen2.5-Coder-3B, it uses a post-training pipeline with curriculum SFT, multi-domain RL (math, code, STEM), Long2Short Math RL for efficiency, offline self-distillation,…

Sen Xu, Shixi Liu, Wei Wang, Jixin Min, et al.
Published
Jun 2026
Citations
1
Code
1.5K stars
Read summary Original
17

MiniMax

MaxProof: Scaling Mathematical Proof with Generative-Verifier RL and Population-Level Test-Time Scaling

MaxProof is a population-level test-time scaling framework for competition-level mathematical proof, developed for the MiniMax-M3 model series. The M3 model is trained with three proof-oriented capabilities: proof generation via verifier-guided reinforcement learning, proof verification through aligned error finding, and critique-conditioned proof repair.…

Jiacheng Chen, Xinyu Zhang, Shunkai Zhang, Yanmohan Wang, et al.
Published
Jun 2026
Citations
1
Code
Not linked
Read summary Original
18

Independent research

APPO: Agentic Procedural Policy Optimization

The paper introduces APPO (Agentic Procedural Policy Optimization), a reinforcement learning algorithm for LLM agents that shifts branching and credit assignment from coarse units like tool-call boundaries to fine-grained decision points in the generated sequence. A pilot study showed that influential decision points are distributed throughout the thinking…

Xucong Wang, Ziyu Ma, Yong Wang, Yuxiang Ji, et al.
Published
Jun 2026
Citations
0
Code
73 stars
Read summary Original
19

Independent research

Z-Reward: Beyond Scalar Rewards by Internalizing Reasoning into Score Distributions

Z-Reward is a teacher-student framework for text-to-image reward modeling that represents visual preference as a reasoning-conditioned score distribution rather than a scalar. The teacher, a 27B VLM, is trained with Group-wise Direct Score Optimization (GDSO), which combines GRPO-style policy-gradient rewards with direct supervision on score distributions…

Xin Jin, Huanqia Cai, Zhen Li, Zechao Zhan, et al.
Published
Jun 2026
Citations
0
Code
Not linked
Read summary Original
20

arXiv.org

Trust-Region Behavior Blending for On-Policy Distillation

The paper introduces Trust-Region behavior Blending (TRB), a warmup method for on-policy distillation (OPD) that addresses the issue of poor early student rollouts. TRB replaces the early rollout policy with a teacher-guided behavior policy that is constrained to stay within a student-centered KL trust region, while keeping the per-prefix reverse-KL OPD…

Daniil Plyusov, Alexey Gorbatovski, Alexey Malakhov, Nikita Balagansky, et al.
Published
May 2026
Citations
0
Code
Not linked
Read summary Original
21

arXiv.org

Self-Improving Language Models with Bidirectional Evolutionary Search

The paper introduces Bidirectional Evolutionary Search (BES), a search framework for self-improving language models and agents. BES addresses two limitations of existing methods (best-of-N and tree search): sparse verification signals and confinement to the model's own distribution. It couples forward search, which uses expansion and four evolution…

Guowei Xu, Zhenting Qi, Huangyuan Su, Weirui Ye, et al.
Published
May 2026
Citations
1
Code
166 stars
Read summary Original
22

NVIDIA

Agent Explorative Policy Optimization for Multimodal Agentic Reasoning

The paper introduces AXPO (Agent eXplorative Policy Optimization) to address the Thinking-Acting Gap in multimodal agentic reasoning, where tool use is under-trained compared to thinking. Under GRPO, tool use occurs in only ~30% of rollouts, and tool-using subgroups are all-wrong on ~40% of questions, suppressing learning signals. AXPO fixes the thinking…

Minki Kang, Shizhe Diao, Ryo Hachiuma, Sung Ju Hwang, et al.
Published
May 2026
Citations
2
Code
Not linked
Read summary Original
23

arXiv.org

ProRL: Effective Reinforcement Learning for Proactive Recommendation via Rectified Policy Gradient Estimation

The paper introduces ProRL, a reinforcement learning framework for proactive recommender systems (PRS) that guide user preferences toward target items via intermediate recommendation paths. The authors identify two deficiencies in standard policy gradient estimation for PRS: (1) a length shortcut, where positive-mean step-level rewards cause gradients to…

Hongru Hou, Tiehua Mei, Denghui Geng, Jinhui Huang, et al.
Published
May 2026
Citations
0
Code
46 stars
Read summary Original
24

arXiv.org

DVAO: Dynamic Variance-adaptive Advantage Optimization for Multi-reward Reinforcement Learning

The paper proposes Dynamic Variance-adaptive Advantage Optimization (DVAO) to improve multi-reward reinforcement learning for LLMs, addressing flaws in standard scalarization methods. Reward Combination (RC) causes training instability due to large advantage magnitudes, while Advantage Combination (AC) uses static weights and ignores cross-objective…

Guochao Jiang, Jingyi Song, Guofeng Quan, Chuzhan Hao, et al.
Published
May 2026
Citations
1
Code
Not linked
Read summary Original
25

arXiv.org

DelTA: Discriminative Token Credit Assignment for Reinforcement Learning from Verifiable Rewards

The paper introduces DelTA, a method for improving reinforcement learning from verifiable rewards (RLVR) in large language models. The authors show that sequence-level RLVR updates act as an implicit linear discriminator over token-gradient vectors, determining which token probabilities increase or decrease. Standard RLVR forms this discriminator using…

Kaiyi Zhang, Wei Wu, Yankai Lin
Published
May 2026
Citations
0
Code
17 stars
Read summary Original
26

arXiv.org

GoLongRL: Capability-Oriented Long Context Reinforcement Learning with Multitask Alignment

GoLongRL is a fully open-source, capability-oriented post-training recipe for long-context reinforcement learning with verifiable rewards (RLVR). It introduces a dataset of 22,965 samples spanning 9 task types, each paired with its natural evaluation metric (e.g., EM, F1, NDCG), and a four-phase construction pipeline. The dataset combines curated…

Minxuan Lv, Tiehua Mei, Tanlong Du, Junmin Chen, et al.
Published
May 2026
Citations
2
Code
75 stars
Read summary Original
27

arXiv.org

Active Learners as Efficient PRP Rerankers

The paper reframes Pairwise Ranking Prompting (PRP) reranking as active learning from noisy pairwise comparisons, arguing that classical sorting algorithms are mismatched with noisy, order-sensitive LLM judgments. The authors propose active rankers, particularly the Mohajer algorithm, as drop-in replacements for sorting in call-constrained settings. They…

Jeremías Figueiredo Paschmann, Juan Kaplan, Francisco Nattero, Santiago Barron, et al.
Published
May 2026
Citations
0
Code
7 stars
Read summary Original
28

arXiv.org

Achieving Gold-Medal-Level Olympiad Reasoning via Simple and Unified Scaling

The paper introduces SU-01, a 30B-A3B reasoning model achieving gold-medal-level performance on mathematical and physical olympiads through a simple, unified post-training recipe. The recipe comprises three stages: (1) SFT with a reverse-perplexity curriculum on 338K rigorous proof trajectories to instill proof-search and self-checking behaviors; (2) a…

Yafu Li, Runzhe Zhan, Haoran Zhang, Shunkai Zhang, et al.
Published
May 2026
Citations
4
Code
94 stars
Read summary Original
29

arXiv.org

Learning to Foresee: Unveiling the Unlocking Efficiency of On-Policy Distillation

This paper investigates why on-policy distillation (OPD) is more efficient than reinforcement learning (RL) for post-training large language models. The authors identify two properties of OPD's parameter updates: Functional Redundancy Avoidance, where OPD suppresses updates in low-utility modules (e.g., embeddings, peripheral layers) and concentrates on…

Yuchen Cai, Ding Cao, Liang Lin, Chunxi Luo, et al.
Published
May 2026
Citations
4
Code
73 stars
Read summary Original
30

arXiv.org

Anti-Self-Distillation for Reasoning RL via Pointwise Mutual Information

The paper identifies a structural bias in on-policy self-distillation for math reasoning, where the per-token signal (conditional pointwise mutual information between the next token and privileged context) rewards shortcut tokens (e.g., 'Given', 'succeeds') and penalizes deliberation tokens (e.g., 'Wait', 'Let'). To fix this, the authors propose…

Guobin Shen, Xiang Cheng, Chenxiao Zhao, Lei Huang, et al.
Published
May 2026
Citations
6
Code
33 stars
Read summary Original
31

Google DeepMind

RubricEM: Meta-RL with Rubric-guided Policy Decomposition beyond Verifiable Rewards

RubricEM is a reinforcement learning framework for training deep research agents on open-ended tasks where verifiable rewards are unavailable. It treats rubrics as a shared interface structuring policy execution, judge feedback, and agent memory. The framework decomposes trajectories into four rubric-guided stages (Plan, Research, Review, Answer) and uses…

Gaotang Li, Bhavana Dalvi Mishra, Zifeng Wang, Jun Yan, et al.
Published
May 2026
Citations
3
Code
Not linked
Read summary Original
32

arXiv.org

Soohak: A Mathematician-Curated Benchmark for Evaluating Research-level Math Capabilities of LLMs

SOOHAK is a new benchmark for evaluating research-level mathematical reasoning in LLMs, created by 64 mathematicians and comprising 340 Challenge and 99 Refusal problems, plus a 702-item SOOHAK-Mini subset. On the Challenge subset, frontier models like Gemini-3-Pro, GPT-5, and Claude-Opus-4.5 achieve Avg@3 scores of 30.39%, 26.37%, and 10.39% respectively,…

Guijin Son, Seungone Kim, Catherine Arnett, Hyunwoo Ko, et al.
Published
May 2026
Citations
4
Code
Not linked
Read summary Original
33

Google DeepMind

LLMs Improving LLMs: Agentic Discovery for Test-Time Scaling

The paper introduces AutoTTS, an environment-driven framework for automatically discovering test-time scaling (TTS) strategies for large language models, shifting the human role from hand-crafting heuristics to constructing discovery environments. The framework formulates width-depth TTS as controller synthesis over an offline replay environment built from…

Tong Zheng, Haolin Liu, Chengsong Huang, Huiwen Bao, et al.
Published
May 2026
Citations
0
Code
170 stars
Read summary Original
34

arXiv.org

Flow-OPD: On-Policy Distillation for Flow Matching Models

Flow-OPD is a novel post-training framework for Flow Matching text-to-image models, addressing reward sparsity and gradient interference in multi-task alignment. It uses a two-stage strategy: first, domain-specialized teachers are trained via single-reward GRPO; then, a student model is initialized via a Flow-based Cold-Start (SFT or model merging) and…

Zhen Fang, Wenxuan Huang, Yu Zeng, Yiming Zhao, et al.
Published
May 2026
Citations
11
Code
274 stars
Read summary Original
35

arXiv.org

Listwise Policy Optimization: Group-based RLVR as Target-Projection on the LLM Response Simplex

This paper introduces Listwise Policy Optimization (LPO), a framework for reinforcement learning with verifiable rewards (RLVR) in large language models. The authors show that existing group-based policy gradient methods (e.g., GRPO, Dr.GRPO, MaxRL) implicitly perform approximate target-projection on a response simplex, where the target is a…

Yun Qu, Qi Wang, Yixiu Mao, Heming Zou, et al.
Published
May 2026
Citations
1
Code
Not linked
Read summary Original
36

arXiv.org

Leveraging Verifier-Based Reinforcement Learning in Image Editing

The paper introduces Edit-R1, a framework for image editing that uses a verifier-based Reasoning Reward Model (RRM) trained with reinforcement learning. The RRM decomposes editing instructions into principles, evaluates edited images against each, and produces interpretable scores. Training involves a cold-start SFT phase with filtered chain-of-thought…

Hanzhong Guo, Jie Wu, Jie Liu, Yu Gao, et al.
Published
Apr 2026
Citations
5
Code
Not linked
Read summary Original
37

arXiv.org

Co-Evolving Policy Distillation

The paper proposes Co-Evolving Policy Distillation (CoPD) to consolidate multiple expert capabilities into a single model. It analyzes two standard paradigms: mixed RLVR, which suffers from capability divergence cost, and the static pipeline of training experts then performing on-policy distillation (OPD), which fails to fully absorb teacher capabilities…

Naibin Gu, Chenxu Yang, Qingyi Si, Chuanyu Qin, et al.
Published
Apr 2026
Citations
3
Code
Not linked
Read summary Original
38

arXiv.org

Recursive Multi-Agent Systems

RecursiveMAS is a recursive multi-agent framework that scales agent collaboration by treating the entire system as a unified latent-space recursive computation. It connects heterogeneous agents via lightweight RecursiveLink modules: inner links enable latent thought generation within each agent, and outer links transfer latent states across agents, forming…

Jiaru Zou, Rui Pan, Ruizhong Qiu, Pan Lu, et al.
Published
Apr 2026
Citations
3
Code
907 stars
Read summary Original
39

arXiv.org

Large Language Models Explore by Latent Distilling

The paper introduces Exploratory Sampling (ESamp), a decoding method for large language models (LLMs) that promotes semantic diversity during generation. ESamp trains a lightweight Latent Distiller (LD) at test time to predict the LLM's deep-layer hidden representations from its shallow-layer ones. The prediction error serves as a novelty signal,…

Yuanhao Zeng, Ao Lu, Lufei Li, Zheng Zhang, et al.
Published
Apr 2026
Citations
0
Code
45 stars
Read summary Original
40

arXiv.org

World-R1: Reinforcing 3D Constraints for Text-to-Video Generation

World-R1 is a framework that improves 3D consistency in text-to-video generation by using reinforcement learning (RL) instead of architectural changes. It builds on the Wan 2.1 model and uses Flow-GRPO to optimize the model with rewards from 3D foundation models (Depth Anything 3) and vision-language models (Qwen3-VL). The reward system includes meta-view…

Weijie Wang, Xiaoxuan He, Youping Gu, Yifan Yang, et al.
Published
Apr 2026
Citations
7
Code
414 stars
Read summary Original
41

arXiv.org

Near-Future Policy Optimization

The paper introduces Near-Future Policy Optimization (NPO), a mixed-policy reinforcement learning with verifiable rewards (RLVR) method that improves training by using trajectories from a near-future checkpoint of the same training run. The authors formalize a quality-variance trade-off, defining effective learning signal S = Q/V, where Q is the fraction…

Chuanyu Qin, Chenxu Yang, Qingyi Si, Naibin Gu, et al.
Published
Apr 2026
Citations
2
Code
Not linked
Read summary Original
42

Independent research

Xiaomi OneVL: One-Step Latent Reasoning and Planning with Vision-Language Explanation

OneVL is a unified Vision-Language-Action (VLA) and World Model framework for autonomous driving that performs one-step latent reasoning and planning with vision-language explanations. It addresses the latency of explicit Chain-of-Thought (CoT) reasoning by compressing reasoning into compact latent tokens supervised by dual auxiliary decoders: a language…

Jinghui Lu, Jiayi Guan, Zhijian Huang, Jinlong Li, et al.
Published
Apr 2026
Citations
3
Code
461 stars
Read summary Original
43

Annual Meeting of the Association for Computational Linguistics

LLaTiSA: Towards Difficulty-Stratified Time Series Reasoning from Visual Perception to Semantics

The paper introduces LLATISA, a Vision-Language Model (VLM) for time series reasoning (TSR), and HITSR, a hierarchical dataset of 83k samples. The authors formalize a four-level TSR taxonomy (L1: Numerical Read-out, L2: Pattern Perception, L3: Semantic Reasoning, L4: Predictive Inference) to address fragmented task definitions and unreliable benchmarks.…

Yueyang Ding, HaoPeng Zhang, Rui Dai, Yi Wang, et al.
Published
Apr 2026
Citations
1
Code
78 stars
Read summary Original
44

arXiv.org

DiPO: Disentangled Perplexity Policy Optimization for Fine-grained Exploration-Exploitation Trade-Off

The paper introduces DiPO (Disentangled Perplexity Policy Optimization), a method for fine-grained exploration-exploitation trade-off in Reinforcement Learning with Verifiable Rewards (RLVR) for LLMs. It identifies two dilemmas in GRPO-based methods: extreme sample groups (all-correct or all-error) yield zero advantage, and perplexity (PPL) distribution…

Xiaofan Li, Ming Yang, Zhiyuan Ma, Shichao Ma, et al.
Published
Apr 2026
Citations
0
Code
Not linked
Read summary Original
45

arXiv.org

KnowRL: Boosting LLM Reasoning via Reinforcement Learning with Minimal-Sufficient Knowledge Guidance

KnowRL is a reinforcement learning (RL) training framework that addresses reward sparsity in large language model (LLM) reasoning by treating hint design as a minimal-sufficient guidance problem. Instead of injecting long solution prefixes or abstract templates, KnowRL decomposes guidance into atomic knowledge points (KPs) and selects compact,…

Linhao Yu, Tianmeng Yang, Siyu Ding, Renren Jin, et al.
Published
Apr 2026
Citations
8
Code
58 stars
Read summary Original
46

arXiv.org

The Past Is Not Past: Memory-Enhanced Dynamic Reward Shaping

The paper introduces MEDS, a framework that enhances reinforcement learning for large language models by dynamically penalizing recurrent error patterns. It addresses the issue of reduced sampling diversity, where policies repeatedly generate similar erroneous behaviors. MEDS stores historical behavioral signals by reusing layer-wise logits from the final…

Yang Liu, Enxi Wang, Yufei Gao, Weixin Zhang, et al.
Published
Apr 2026
Citations
1
Code
142 stars
Read summary Original
47

arXiv.org

Rethinking Generalization in Reasoning SFT: A Conditional Analysis on Optimization, Data, and Model Capability

This preprint challenges the narrative that supervised fine-tuning (SFT) memorizes while reinforcement learning generalizes, showing that cross-domain generalization in reasoning SFT is conditional on optimization dynamics, training data, and base-model capability. Using math-only long chain-of-thought (CoT) SFT on pretrained base models, the authors find…

Qihan Ren, Peng Wang, Ruikun Cai, Shuai Shao, et al.
Published
Apr 2026
Citations
3
Code
109 stars
Read summary Original
48

arXiv.org

RAGEN-2: Reasoning Collapse in Agentic RL

RAGEN-2 identifies a failure mode in multi-turn LLM agent reinforcement learning called template collapse, where reasoning appears diverse within inputs but becomes input-agnostic across inputs, invisible to entropy-based metrics. The authors decompose reasoning quality into within-input diversity (conditional entropy) and cross-input distinguishability…

Zihan Wang, Chi Gui, Xing Jin, Qineng Wang, et al.
Published
Apr 2026
Citations
10
Code
2.8K stars
Read summary Original
49

arXiv.org

Self-Distilled RLVR

The paper analyzes on-policy self-distillation (OPSD) for LLM post-training, where a model serves as both teacher and student, with the teacher using privileged information. It proves that OPSD's information asymmetry creates an irreducible mutual information gap, causing privileged information leakage and performance degradation after early gains. To…

Chenxu Yang, Chuanyu Qin, Qingyi Si, Minghui Chen, et al.
Published
Apr 2026
Citations
56
Code
Not linked
Read summary Original
50

arXiv.org

GrandCode: Achieving Grandmaster Level in Competitive Programming via Agentic Reinforcement Learning

GrandCode is a multi-agent reinforcement learning (RL) system for competitive programming, introduced by the DeepReinforce Team. It orchestrates modules for hypothesis proposal, solution generation, summarization, and test-case generation, jointly optimized via post-training and online test-time RL. A key innovation is Agentic GRPO, which combines…

DeepReinforce Team, Xiaoya Li, Guoyin Wang, Songqiao Su, et al.
Published
Apr 2026
Citations
3
Code
Not linked
Read summary Original