The Year in AI Papers
The year/Topics/Reasoning

Topic area

Reasoning

Every collection across reasoning.

Papers
177
Research labs
7
Official code
113

51–100 of 177 papers in this topic area

51

arXiv.org

The Latent Space: Foundation, Evolution, Mechanism, Ability, and Outlook

This survey provides a unified overview of latent space in language-based models, arguing that it is emerging as a native computational substrate beyond explicit token-level processing. It addresses the fragmentation in the field by organizing research into five sequential perspectives: Foundation, Evolution, Mechanism, Ability, and Outlook. The survey…

Xinlei Yu, Zhangquan Chen, Yongbo He, Tianyu Fu, et al.
Published
Apr 2026
Citations
30
Code
956 stars
Read summary Original
52

arXiv.org

All Roads Lead to Rome: Incentivizing Divergent Thinking in Vision-Language Models

This paper investigates the behavioral differences between Reinforcement Learning (RL) and base Vision-Language Models (VLMs). The authors find that RL models, such as those trained with Group Relative Policy Optimization (GRPO), exhibit deeper but narrower reasoning, while base models show broader and more diverse thinking patterns. They identify a…

Xinyu Tian, Shu Zou, Zhaoyuan Yang, Mengqi He, et al.
Published
Apr 2026
Citations
0
Code
148 stars
Read summary Original
53

arXiv.org

Why Does Self-Distillation (Sometimes) Degrade the Reasoning Capability of LLMs?

This paper investigates why self-distillation can degrade the mathematical reasoning performance of LLMs, despite often improving other domains. The authors find that self-distillation suppresses 'epistemic verbalization'—the expression of uncertainty during reasoning—which is crucial for robust problem-solving. Through controlled experiments, they show…

Jeonghye Kim, Xufang Luo, Minbeom Kim, Sangmook Lee, et al.
Published
Mar 2026
Citations
61
Code
75 stars
Read summary Original
54

arXiv.org

LongCat-Flash-Prover: Advancing Native Formal Reasoning via Agentic Tool-Integrated Reinforcement Learning

LongCat-Flash-Prover is a 560-billion-parameter open-source Mixture-of-Experts (MoE) model for native formal reasoning in Lean4, developed by Meituan LongCat Team. It decomposes formal reasoning into auto-formalization, sketching, and proving, and uses a Hybrid-Experts Iteration Framework to synthesize high-quality trajectories with tool feedback. The…

Jianing Wang, Jianfei Zhang, Qi Guo, Linsen Guo, et al.
Published
Mar 2026
Citations
5
Code
93 stars
Read summary Original
55

Qwen

FIPO: Eliciting Deep Reasoning with Future-KL Influenced Policy Optimization

FIPO (Future-KL Influenced Policy Optimization) is a reinforcement learning algorithm that addresses the coarse-grained credit assignment problem in GRPO-style training for large language models. By incorporating a discounted Future-KL divergence into the policy update, FIPO re-weights token advantages based on their influence on subsequent trajectory…

Chiyu Ma, Shuo Yang, Kexin Huang, Jinda Lu, et al.
Published
Mar 2026
Citations
18
Code
130 stars
Read summary Original
56

NVIDIA

Nemotron-Cascade 2: Post-Training LLMs with Cascade RL and Multi-Domain On-Policy Distillation

Nemotron-Cascade 2 is an open 30B Mixture-of-Experts (MoE) model with 3B activated parameters, achieving best-in-class reasoning and agentic capabilities. It is the second open-weight LLM to achieve Gold Medal-level performance in the 2025 IMO, IOI, and ICPC World Finals, with 20x fewer parameters than DeepSeek-V3.2-Speciale-671B-A37B. The model builds on…

Zhuolin Yang, Zihan Liu, Yang Chen, Wenliang Dai, et al.
Published
Mar 2026
Citations
20
Code
Not linked
Read summary Original
57

arXiv.org

Astrolabe: Steering Forward-Process Reinforcement Learning for Distilled Autoregressive Video Models

Astrolabe is an online reinforcement learning (RL) framework designed to align distilled autoregressive (AR) video models with human preferences without re-distillation. It addresses limitations of existing RL methods, which require expensive reverse-process optimization or lack active exploration. Astrolabe uses a forward-process RL formulation based on…

Songchun Zhang, Zeyue Xue, Siming Fu, Jie Huang, et al.
Published
Mar 2026
Citations
10
Code
151 stars
Read summary Original
58

arXiv.org

TRUST-SQL: Tool-Integrated Multi-Turn Reinforcement Learning for Text-to-SQL over Unknown Schemas

TRUST-SQL addresses Text-to-SQL parsing under the Unknown Schema setting, where databases have hundreds of tables and noisy metadata, making full schema prefilling impractical. The task is formulated as a Partially Observable Markov Decision Process, and the agent follows a four-phase protocol (Explore, Propose, Generate, Confirm) to ground reasoning in…

Ai Jian, Xiaoyun Zhang, Wanrou Du, Jingqing Ruan, et al.
Published
Mar 2026
Citations
3
Code
Not linked
Read summary Original
59

arXiv.org

Video-CoE: Reinforcing Video Event Prediction via Chain of Events

The paper introduces Video-CoE, a method to improve video event prediction (VEP) in multimodal large language models (MLLMs). The authors first evaluate leading MLLMs on VEP benchmarks, identifying two main failure causes: lack of logical reasoning for future events and insufficient use of visual information. To address these, they propose the Chain of…

Qile Su, Jing Tang, Rui Chen, Lei Sun, et al.
Published
Mar 2026
Citations
2
Code
Not linked
Read summary Original
60

arXiv.org

AI Can Learn Scientific Taste

The paper introduces Reinforcement Learning from Community Feedback (RLCF), a paradigm that uses citation-based community signals to train AI models for scientific taste—the ability to judge and propose high-impact research ideas. They built SciJudgeBench with 720K field- and time-matched paper abstract pairs, trained Scientific Judge via GRPO to predict…

Jingqi Tong, Mingzhe Li, Hangcheng Li, Yongzhuo Yang, et al.
Published
Mar 2026
Citations
4
Code
428 stars
Read summary Original
61

arXiv.org

Efficient Reasoning with Balanced Thinking

Large Reasoning Models (LRMs) often overthink (redundant steps on simple problems) or underthink (insufficient exploration), causing inefficiency and inaccuracy. Existing mitigation methods, like suppressing reflection keywords, can induce underthinking. The paper proposes REBALANCE, a training-free framework to balance these extremes. It uses stepwise…

Yulin Li, Tengyao Tu, Li Ding, Junjie Wang, et al.
Published
Mar 2026
Citations
3
Code
333 stars
Read summary Original
62

NVIDIA

MM-Zero: Self-Evolving Multi-Model Vision Language Models From Zero Data

MM-Zero is a reinforcement learning framework that enables zero-data self-evolution for Vision Language Models (VLMs). It introduces a tri-role system—Proposer, Coder, and Solver—all initialized from the same base model and trained sequentially using Group Relative Policy Optimization (GRPO). The Proposer generates visual descriptions and questions, the…

Zongxia Li, Hongyang Du, Chengsong Huang, Xiyang Wu, et al.
Published
Mar 2026
Citations
12
Code
79 stars
Read summary Original
63

arXiv.org

How Far Can Unsupervised RLVR Scale LLM Training?

This paper investigates the scalability of Unsupervised Reinforcement Learning with Verifiable Rewards (URLVR) for LLM training. It categorizes URLVR methods into intrinsic (certainty-based and ensemble-based) and external rewards. The authors establish a unified theoretical framework showing that all intrinsic methods converge to sharpening the model's…

Bingxiang He, Yuxin Zuo, Zeyuan Liu, Shangziqi Zhao, et al.
Published
Mar 2026
Citations
24
Code
1.1K stars
Read summary Original
64

arXiv.org

Thinking in Uncertainty: Mitigating Hallucinations in MLRMs with Latent Entropy-Aware Decoding

The paper introduces Latent Entropy-Aware Decoding (LEAD), a plug-and-play decoding strategy to mitigate hallucinations in multimodal large reasoning models (MLRMs). The authors observe that transition words (e.g., because, however, wait) are associated with high-entropy states and hallucinations. They propose entropy-aware reasoning mode switching: in…

Zhongxing Xu, Zhonghua Wang, Zhe Qian, Dachuan Shi, et al.
Published
Mar 2026
Citations
10
Code
94 stars
Read summary Original
65

arXiv.org

BandPO: Bridging Trust Regions and Ratio Clipping via Probability-Aware Bounds for LLM Reinforcement Learning

The paper introduces BandPO, a method for LLM reinforcement learning that replaces the fixed clipping bounds of PPO/GRPO with dynamic, probability-aware bounds derived from f-divergence trust regions. The authors identify a bottleneck in canonical clipping: fixed bounds limit the upward update margin for low-probability actions, suppressing high-advantage…

Yuan Li, Bo Wang, Yufei Gao, Yuqian Yao, et al.
Published
Mar 2026
Citations
2
Code
49 stars
Read summary Original
66

arXiv.org

Bootstrapping Exploration with Group-Level Natural Language Feedback in Reinforcement Learning

The paper introduces GOLF, a reinforcement learning (RL) framework that leverages group-level natural language (NL) feedback to improve exploration efficiency in training large language models (LLMs). Unlike standard RL that relies solely on scalar rewards, GOLF aggregates two complementary feedback sources: external critiques (pinpointing errors or…

Lei Huang, Xiang Cheng, Chenxiao Zhao, Guobin Shen, et al.
Published
Mar 2026
Citations
2
Code
18 stars
Read summary Original
67

arXiv.org

T2S-Bench & Structure-of-Thought: Benchmarking and Prompting Comprehensive Text-to-Structure Reasoning

The paper introduces Structure of Thought (SoT), a prompting technique that guides LLMs to construct intermediate text structures (nodes and links) before answering, consistently improving performance across eight text-processing tasks and three model families. Building on this, the authors present T2S-Bench, the first benchmark for evaluating…

Qinsi Wang, Hancheng Ye, Jinhee Kim, Jinghan Ke, et al.
Published
Mar 2026
Citations
1
Code
24 stars
Read summary Original
68

Together AI

V1: Unifying Generation and Self-Verification for Parallel Reasoners

The paper introduces V1, a framework that unifies generation and self-verification for parallel reasoning in LLMs. It identifies that pointwise self-verification suffers from calibration collapse, while self-aggregation methods like RSA lead to diversity collapse. V1 comprises two components: V1-Infer, an uncertainty-guided pairwise ranking algorithm using…

Harman Singh, Xiuyu Li, Kusha Sareen, Monishwaran Maheswaran, et al.
Published
Mar 2026
Citations
7
Code
39 stars
Read summary Original
69

Independent research

Edit in 2D, Verify in 3D: Reinforcement Learning for Multi-view Consistent Scene Editing

The paper introduces RL3DEdit, a single-pass framework for multi-view consistent 3D scene editing that uses reinforcement learning (RL) to augment a 2D diffusion editor (FLUX-Kontext) with 3D consistency priors. The authors observe that while generating multi-view consistent images is challenging, verifying consistency is tractable, making RL a suitable…

Jiyuan Wang, Chunyu Lin, Lei Sun, Zhi Cao, et al.
Published
Mar 2026
Citations
7
Code
203 stars
Read summary Original
70

arXiv.org

Heterogeneous Agent Collaborative Reinforcement Learning

The paper introduces Heterogeneous Agent Collaborative Reinforcement Learning (HACRL), a new RLVR problem where heterogeneous LLM agents share verified rollouts during training to mutually improve, while executing independently at inference. Unlike LLM-based MARL, HACRL does not require coordinated deployment, and unlike distillation, it enables…

Zhixia Zhang, Zixuan Huang, Gongxun Li, Huaiyang Wang, et al.
Published
Mar 2026
Citations
12
Code
Not linked
Read summary Original
71

arXiv.org

CUDA Agent: Large-Scale Agentic RL for High-Performance CUDA Kernel Generation

CUDA Agent is a large-scale agentic reinforcement learning (RL) system for generating high-performance CUDA kernels. It addresses the gap between LLMs and compiler-based systems like torch.compile by combining a scalable data synthesis pipeline (crawling seed operators, LLM-based combinatorial fusion, and rubric-based filtering to create…

Weinan Dai, Hanlin Wu, Qiying Yu, Huan-ang Gao, et al.
Published
Feb 2026
Citations
32
Code
1.1K stars
Read summary Original
72

arXiv.org

From Scale to Speed: Adaptive Test-Time Scaling for Image Editing

The paper introduces ADE-CoT, a test-time scaling framework for image editing that improves efficiency and performance. It addresses three issues with applying Image-CoT to editing: inefficient fixed sampling budgets, unreliable early-stage verification using general MLLM scores, and redundant results from large-scale sampling. ADE-CoT uses…

Xiangyan Qu, Zhenlong Yuan, Jing Tang, Rui Chen, et al.
Published
Feb 2026
Citations
6
Code
Not linked
Read summary Original
73

arXiv.org

Experiential Reinforcement Learning

The paper introduces Experiential Reinforcement Learning (ERL), a training paradigm for language models that embeds an explicit experience–reflection–consolidation loop into reinforcement learning. In ERL, the model first generates an initial attempt, receives environmental feedback, and then produces a self-reflection that guides a refined second attempt.…

Taiwei Shi, Sihao Chen, Bowen Jiang, Linxin Song, et al.
Published
Feb 2026
Citations
12
Code
76 stars
Read summary Original
74

arXiv.org

Learning beyond Teacher: Generalized On-Policy Distillation with Reward Extrapolation

The paper introduces Generalized On-Policy Distillation (G-OPD), a framework that extends standard on-policy distillation (OPD) by adding a flexible reference model and a reward scaling factor λ. The authors show that OPD is a special case of dense KL-constrained RL with equal reward and KL weights. In G-OPD, λ controls the relative weight of the reward…

Wenkai Yang, Weijie Liu, Ruobing Xie, Kai Yang, et al.
Published
Feb 2026
Citations
87
Code
278 stars
Read summary Original
75

arXiv.org

Composition-RL: Compose Your Verifiable Prompts for Reinforcement Learning of Large Language Models

Composition-RL addresses the issue of 'solve all' prompts (pass rate 1) that become prevalent during RLVR training, reducing effective data size. The method automatically composes multiple existing verifiable prompts into new, harder ones using Sequential Prompt Composition (SPC), which modifies one prompt with a variable from another's answer and links…

Xin Xu, Clive Bai, Kai Yang, Tianhao Chen, et al.
Published
Feb 2026
Citations
3
Code
137 stars
Read summary Original
76

arXiv.org

VESPO: Variational Sequence-Level Soft Policy Optimization for Stable Off-Policy LLM Training

VESPO addresses high variance in off-policy reinforcement learning for large language models by deriving a principled importance weight reshaping kernel from a variational formulation. It recasts weight reshaping as a measure change to an implicit proposal distribution, solving a KL-regularized objective with a variance constraint to obtain a closed-form…

Guobin Shen, Chenxiao Zhao, Xiang Cheng, Lei Huang, et al.
Published
Feb 2026
Citations
7
Code
35 stars
Read summary Original
77

arXiv.org

Chain of Mindset: Reasoning with Adaptive Cognitive Modes

The paper introduces Chain of Mindset (CoM), a training-free agentic framework for LLM reasoning that enables step-level adaptive switching among four cognitive mindsets: Spatial, Convergent, Divergent, and Algorithmic. A Meta-Agent dynamically selects the optimal mindset based on the evolving reasoning state, while a bidirectional Context Gate filters…

Tianyi Jiang, Arctanx An, Hengyi Feng, Naixin Zhai, et al.
Published
Feb 2026
Citations
1
Code
65 stars
Read summary Original
78

arXiv.org

Does Your Reasoning Model Implicitly Know When to Stop Thinking?

The paper investigates why large reasoning models (LRMs) produce lengthy, redundant chains of thought (CoTs) despite shorter chains often being more accurate. The authors discover that LRMs implicitly know when to stop thinking, but this capability is obscured by current sampling paradigms like pass@1. They introduce SAGE (Self-Aware Guided Efficient…

Zixuan Huang, Xin Xia, Yuxi Ren, Jianbin Zheng, et al.
Published
Feb 2026
Citations
20
Code
Not linked
Read summary Original
79

arXiv.org

Weak-Driven Learning: How Weak Agents make Strong Agents Stronger

The paper introduces Weak-Driven Learning, a post-training paradigm that repurposes weak historical checkpoints as corrective signals to strengthen a strong language model, contrasting with knowledge distillation that imitates stronger teachers. The authors instantiate this with WMSS (Weak agents Make Strong agents Stronger), which combines Weak-Driven…

Zehao Chen, Gongxun Li, Tianxiang Ai, Zixuan Huang, et al.
Published
Feb 2026
Citations
4
Code
79 stars
Read summary Original
80

arXiv.org

F-GRPO: Don't Let Your Policy Learn the Obvious and Forget the Rare

The paper introduces F-GRPO, a method to mitigate distribution sharpening in Reinforcement Learning with Verifiable Rewards (RLVR) using group-relative objectives like GRPO, DAPO, and CISPO. The authors derive a non-monotonic tail-miss probability, showing that intermediate group sizes can maximize active updates that miss rare-correct solutions, while…

Daniil Plyusov, Alexey Gorbatovski, Boris Shaposhnikov, Viacheslav Sinii, et al.
Published
Feb 2026
Citations
8
Code
Not linked
Read summary Original
81

arXiv.org

Length-Unbiased Sequence Policy Optimization: Revealing and Controlling Response Length Variation in RLVR

The paper analyzes response length variation in Reinforcement Learning with Verifiable Rewards (RLVR) for large language models, focusing on the length bias in GRPO and GSPO objectives. GRPO averages token contributions per trajectory, causing shorter correct responses to receive larger gradient updates, while GSPO's sequence-level clipping and Clip-Higher…

Fanfan Liu, Youyang Yin, Peng Shi, Siqi Yang, et al.
Published
Feb 2026
Citations
6
Code
8 stars
Read summary Original
82

arXiv.org

On the Entropy Dynamics in Reinforcement Fine-Tuning of Large Language Models

This paper establishes a theoretical framework for analyzing entropy dynamics during reinforcement fine-tuning (RFT) of large language models. The authors derive a first-order expression for entropy change from a single logit update, showing that the direction of entropy change is determined by the update direction and a discriminator score S* = p_k(H +…

Shumin Wang, Yuexiang Xie, Wenhao Zhang, Yuchang Sun, et al.
Published
Feb 2026
Citations
7
Code
Not linked
Read summary Original
83

NVIDIA

Golden Goose: A Simple Trick to Synthesize Unlimited RLVR Tasks from Unverifiable Internet Text

The paper introduces Golden Goose, a method to synthesize unlimited Reinforcement Learning with Verifiable Rewards (RLVR) tasks from unverifiable internet text by converting them into multiple-choice fill-in-the-middle questions. Given a source text, an LLM masks a contiguous span of crucial reasoning steps (the ground-truth answer) and generates diverse,…

Ximing Lu, David Acuna, Jaehun Jung, Jian Hu, et al.
Published
Jan 2026
Citations
6
Code
Not linked
Read summary Original
84

arXiv.org

Harder Is Better: Boosting Mathematical Reasoning via Difficulty-Aware GRPO and Multi-Aspect Question Reformulation

The paper introduces MathForge, a framework to enhance mathematical reasoning in large language models via reinforcement learning with verifiable rewards (RLVR). It addresses a lack of emphasis on harder questions in existing methods, both algorithmically and data-wise. Algorithmically, it proposes Difficulty-Aware Group Policy Optimization (DGPO), which…

Yanqi Dai, Yuxiang Ji, Xiao Zhang, Yong Wang, et al.
Published
Jan 2026
Citations
23
Code
128 stars
Read summary Original
85

Together AI

Learning to Discover at Test Time

The paper introduces TTT-Discover, a method that performs reinforcement learning at test time to solve scientific discovery problems. Unlike prior test-time scaling methods that use a frozen LLM for search, TTT-Discover continues to train the LLM on the specific test problem, using an entropic objective and PUCT-based state reuse to prioritize the most…

Mert Yuksekgonul, Daniel Koceja, Xinhao Li, Federico Bianchi, et al.
Published
Jan 2026
Citations
71
Code
613 stars
Read summary Original
86

Annual Meeting of the Association for Computational Linguistics

MAXS: Meta-Adaptive Exploration with LLM Agents

The paper introduces MAXS, a meta-adaptive reasoning framework for LLM agents that addresses two issues: locally myopic generation and trajectory instability. MAXS uses a lookahead strategy to extend reasoning paths a few steps ahead, estimating the advantage value of tool usage, and combines step consistency variance and inter-step trend slopes to select…

Jian Zhang, Zhiyuan Wang, Zhangqi Wang, Yu He, et al.
Published
Jan 2026
Citations
1
Code
Not linked
Read summary Original
87

arXiv.org

Distribution-Aligned Sequence Distillation for Superior Long-CoT Reasoning

This report introduces DASD-4B-Thinking, a lightweight open-source reasoning model that achieves state-of-the-art performance on math, science, and code benchmarks, outperforming larger models (e.g., 32B) using only 448K training samples. The authors critique the common sequence-level distillation paradigm (SFT on teacher-generated responses) for three…

Shaotian Yan, Kaiyuan Liu, Chen Shen, Bing Wang, et al.
Published
Jan 2026
Citations
13
Code
105 stars
Read summary Original
88

Annual Meeting of the Association for Computational Linguistics

Rewarding the Rare: Uniqueness-Aware RL for Creative Problem Solving in LLMs

The paper introduces Uniqueness-Aware Reinforcement Learning (UARL) to address exploration collapse in RL-trained LLMs, where policies prematurely converge to a few dominant reasoning patterns, improving pass@1 but limiting pass@k. UARL operates at the rollout level, using an LLM judge to cluster solutions by high-level strategy and reweighting GRPO…

Zhiyuan Hu, Yucheng Wang, Yufei He, Jiaying Wu, et al.
Published
Jan 2026
Citations
9
Code
Not linked
Read summary Original
89

arXiv.org

Your Group-Relative Advantage Is Biased

This paper identifies a fundamental bias in group-relative advantage estimation used by RLVR algorithms like GRPO. The authors prove that this estimator systematically underestimates advantages for hard prompts and overestimates them for easy prompts, with bias increasing as prompt difficulty deviates from 0.5 and with smaller group sizes. To address this,…

Fengkai Yang, Zherui Chen, Xiaohan Wang, Xiaodong Lu, et al.
Published
Jan 2026
Citations
27
Code
Not linked
Read summary Original
90

Qwen

ArenaRL: Scaling RL for Open-Ended Agents via Tournament-based Relative Ranking

ArenaRL is a reinforcement learning framework for open-ended LLM agents that replaces pointwise scalar rewards with tournament-based relative ranking to overcome discriminative collapse, where reward models fail to distinguish subtle differences among high-quality trajectories. It introduces a process-aware pairwise evaluation mechanism and a seeded…

Qiang Zhang, Boli Chen, Fanrui Zhang, Ruixue Ding, et al.
Published
Jan 2026
Citations
9
Code
274 stars
Read summary Original
91

arXiv.org

The Molecular Structure of Thought: Mapping the Topology of Long Chain-of-Thought Reasoning

The paper proposes that effective long chain-of-thought (Long CoT) reasoning in LLMs requires stable molecular-like structures formed by three interaction types: Deep-Reasoning (covalent-like), Self-Reflection (hydrogen-bond-like), and Self-Exploration (van der Waals-like). Distillation from strong reasoning LLMs (e.g., DeepSeek-R1, QwQ, OpenAI-OSS)…

Qiguang Chen, Yantao Du, Ziniu Li, Jinhao Liu, et al.
Published
Jan 2026
Citations
3
Code
Not linked
Read summary Original
92

Annual Meeting of the Association for Computational Linguistics

PaCoRe: Learning to Scale Test-Time Compute with Parallel Coordinated Reasoning

PaCoRe (Parallel Coordinated Reasoning) is a training-and-inference framework that scales test-time compute (TTC) beyond sequential reasoning limits by using parallel exploration coordinated via message passing. It runs multiple rounds: each round generates parallel reasoning trajectories, compacts them into context-bounded messages, and synthesizes these…

Jingcheng Hu, Yinmin Zhang, Shijie Shang, Xiaobo Yang, et al.
Published
Jan 2026
Citations
16
Code
338 stars
Read summary Original
93

NVIDIA

GDPO: Group reward-Decoupled Normalization Policy Optimization for Multi-reward RL Optimization

The paper identifies a flaw in applying Group Relative Policy Optimization (GRPO) to multi-reward reinforcement learning: normalizing the summed reward causes distinct reward combinations to collapse into identical advantage values, reducing training signal resolution and causing suboptimal convergence or early failure. To address this, the authors propose…

Shih-Yang Liu, Xin Dong, Ximing Lu, Shizhe Diao, et al.
Published
Jan 2026
Citations
114
Code
495 stars
Read summary Original
94

arXiv.org

MMFormalizer: Multimodal Autoformalization in the Wild

The paper introduces MMFORMALIZER, a framework for multimodal autoformalization that translates natural language and visual inputs into formal LEAN statements. It addresses challenges in grounding physical concepts by using recursive grounding and dimensional analysis, with adaptive termination to ensure visual evidence supports abstractions. The authors…

Jing Xiong, Qi Han, Yunta Hsieh, Hui Shen, et al.
Published
Jan 2026
Citations
0
Code
Not linked
Read summary Original
95

arXiv.org

Entropy-Adaptive Fine-Tuning: Resolving Confident Conflicts to Mitigate Forgetting

The paper investigates why Supervised Fine-Tuning (SFT) causes catastrophic forgetting while on-policy Reinforcement Learning (RL) preserves general capabilities. The authors identify a distributional gap: SFT data contains 'Confident Conflicts'—tokens with low probability and low entropy, where the model is confident but forced to learn a divergent label,…

Muxi Diao, Lele Yang, Wuxuan Gong, Yutong Zhang, et al.
Published
Jan 2026
Citations
12
Code
105 stars
Read summary Original
96

arXiv.org

Dynamic Large Concept Models: Latent Reasoning in an Adaptive Semantic Space

The paper introduces Dynamic Large Concept Models (DLCM), a hierarchical language modeling framework that learns variable-length semantic concepts from latent representations and performs reasoning in a compressed concept space, shifting computation from tokens to concepts. DLCM uses a four-stage pipeline: encoding, dynamic segmentation via learned…

Xingwei Qu, Shaowen Wang, Zihao Huang, Kai Hua, et al.
Published
Dec 2025
Citations
12
Code
Not linked
Read summary Original
97

arXiv.org

DiffThinker: Towards Generative Multimodal Reasoning with Diffusion Models

DiffThinker is a diffusion-based framework that introduces Generative Multimodal Reasoning, reformulating multimodal reasoning as a native image-to-image generative task rather than text-centric symbolic mapping. Built on Qwen-Image-Edit with Flow Matching and a Multimodal Diffusion Transformer, it directly produces solution images that are parsed into…

Zefeng He, Xiaoye Qu, Yafu Li, Tong Zhu, et al.
Published
Dec 2025
Citations
12
Code
185 stars
Read summary Original
98

arXiv.org

Latent Implicit Visual Reasoning

The paper introduces Latent Implicit Visual Reasoning (LIVR), a method to enhance visual reasoning in Large Multimodal Models (LMMs) without explicit intermediate supervision. LIVR adds latent tokens to the model's vocabulary and trains them via a two-stage visual bottlenecking approach: Stage 1 forces visual information to pass through latent tokens by…

Kelvin Li, Chuyi Shang, Leonid Karlinsky, Rogerio Feris, et al.
Published
Dec 2025
Citations
8
Code
Not linked
Read summary Original
99

Google DeepMind

Emergent temporal abstractions in autoregressive models enable hierarchical reinforcement learning

This paper introduces a method for hierarchical reinforcement learning (RL) in autoregressive models by discovering and using temporally-abstract actions within the model's internal representations. The authors show that autoregressive models pretrained on next-token prediction learn latent representations of subgoals in their residual stream activations.…

Seijin Kobayashi, Yanick Schimpf, Maximilian Schlegel, Angelika Steger, et al.
Published
Dec 2025
Citations
5
Code
Not linked
Read summary Original
100

arXiv.org

Bottom-up Policy Optimization: Your Language Model Policy Secretly Contains Internal Policies

This paper introduces Bottom-up Policy Optimization (BuPO), a novel reinforcement learning (RL) paradigm for large language models (LLMs) that optimizes internal layer policies before the full language model policy. The authors decompose the LLM policy into Internal Layer Policies and Internal Modular Policies via the Transformer's residual stream. Entropy…

Yuqiao Tan, Minzheng Wang, Shizhu He, Huanxuan Liao, et al.
Published
Dec 2025
Citations
21
Code
60 stars
Read summary Original