Research collection
Techniques that improve how a model deliberates: chain-of-thought, latent or implicit reasoning, self-consistency, thinking budgets, and knowing when to stop thinking.
Independent research
The paper introduces Mental World Modeling (MWM), a framework for world models that jointly represent physical and mental dynamics to predict human decisions. MWM maintains a coupled physical-mental world state, renders a target-specific partial observation, and simulates how candidate actions update both components. The authors implement MENTIS, a…
Hao Fei, Yiran ZhaoIndependent research
The paper investigates loop-count selection in Parallel Loop Transformers (PLT), which use cross-loop position offsets (CLP) and shared-KV gated sliding-window attention to keep latency and memory constant regardless of loop count. The authors train LoopCoder-v2, a 7B PLT coder, from scratch on 18T tokens with loop counts R=1,2,3,4, under matched training…
Jian Yang, Shawn Guo, Wei Zhang, Tianyu Zheng, et al.arXiv.org
The paper introduces Bidirectional Evolutionary Search (BES), a search framework for self-improving language models and agents. BES addresses two limitations of existing methods (best-of-N and tree search): sparse verification signals and confinement to the model's own distribution. It couples forward search, which uses expansion and four evolution…
Guowei Xu, Zhenting Qi, Huangyuan Su, Weirui Ye, et al.arXiv.org
The paper reframes Pairwise Ranking Prompting (PRP) reranking as active learning from noisy pairwise comparisons, arguing that classical sorting algorithms are mismatched with noisy, order-sensitive LLM judgments. The authors propose active rankers, particularly the Mohajer algorithm, as drop-in replacements for sorting in call-constrained settings. They…
Jeremías Figueiredo Paschmann, Juan Kaplan, Francisco Nattero, Santiago Barron, et al.arXiv.org
SOOHAK is a new benchmark for evaluating research-level mathematical reasoning in LLMs, created by 64 mathematicians and comprising 340 Challenge and 99 Refusal problems, plus a 702-item SOOHAK-Mini subset. On the Challenge subset, frontier models like Gemini-3-Pro, GPT-5, and Claude-Opus-4.5 achieve Avg@3 scores of 30.39%, 26.37%, and 10.39% respectively,…
Guijin Son, Seungone Kim, Catherine Arnett, Hyunwoo Ko, et al.Google DeepMind
The paper introduces AutoTTS, an environment-driven framework for automatically discovering test-time scaling (TTS) strategies for large language models, shifting the human role from hand-crafting heuristics to constructing discovery environments. The framework formulates width-depth TTS as controller synthesis over an offline replay environment built from…
Tong Zheng, Haolin Liu, Chengsong Huang, Huiwen Bao, et al.arXiv.org
RecursiveMAS is a recursive multi-agent framework that scales agent collaboration by treating the entire system as a unified latent-space recursive computation. It connects heterogeneous agents via lightweight RecursiveLink modules: inner links enable latent thought generation within each agent, and outer links transfer latent states across agents, forming…
Jiaru Zou, Rui Pan, Ruizhong Qiu, Pan Lu, et al.arXiv.org
The paper introduces Exploratory Sampling (ESamp), a decoding method for large language models (LLMs) that promotes semantic diversity during generation. ESamp trains a lightweight Latent Distiller (LD) at test time to predict the LLM's deep-layer hidden representations from its shallow-layer ones. The prediction error serves as a novelty signal,…
Yuanhao Zeng, Ao Lu, Lufei Li, Zheng Zhang, et al.Independent research
OneVL is a unified Vision-Language-Action (VLA) and World Model framework for autonomous driving that performs one-step latent reasoning and planning with vision-language explanations. It addresses the latency of explicit Chain-of-Thought (CoT) reasoning by compressing reasoning into compact latent tokens supervised by dual auxiliary decoders: a language…
Jinghui Lu, Jiayi Guan, Zhijian Huang, Jinlong Li, et al.Annual Meeting of the Association for Computational Linguistics
The paper introduces LLATISA, a Vision-Language Model (VLM) for time series reasoning (TSR), and HITSR, a hierarchical dataset of 83k samples. The authors formalize a four-level TSR taxonomy (L1: Numerical Read-out, L2: Pattern Perception, L3: Semantic Reasoning, L4: Predictive Inference) to address fragmented task definitions and unreliable benchmarks.…
Yueyang Ding, HaoPeng Zhang, Rui Dai, Yi Wang, et al.arXiv.org
This preprint challenges the narrative that supervised fine-tuning (SFT) memorizes while reinforcement learning generalizes, showing that cross-domain generalization in reasoning SFT is conditional on optimization dynamics, training data, and base-model capability. Using math-only long chain-of-thought (CoT) SFT on pretrained base models, the authors find…
Qihan Ren, Peng Wang, Ruikun Cai, Shuai Shao, et al.arXiv.org
This survey provides a unified overview of latent space in language-based models, arguing that it is emerging as a native computational substrate beyond explicit token-level processing. It addresses the fragmentation in the field by organizing research into five sequential perspectives: Foundation, Evolution, Mechanism, Ability, and Outlook. The survey…
Xinlei Yu, Zhangquan Chen, Yongbo He, Tianyu Fu, et al.arXiv.org
Large Reasoning Models (LRMs) often overthink (redundant steps on simple problems) or underthink (insufficient exploration), causing inefficiency and inaccuracy. Existing mitigation methods, like suppressing reflection keywords, can induce underthinking. The paper proposes REBALANCE, a training-free framework to balance these extremes. It uses stepwise…
Yulin Li, Tengyao Tu, Li Ding, Junjie Wang, et al.arXiv.org
The paper introduces Latent Entropy-Aware Decoding (LEAD), a plug-and-play decoding strategy to mitigate hallucinations in multimodal large reasoning models (MLRMs). The authors observe that transition words (e.g., because, however, wait) are associated with high-entropy states and hallucinations. They propose entropy-aware reasoning mode switching: in…
Zhongxing Xu, Zhonghua Wang, Zhe Qian, Dachuan Shi, et al.arXiv.org
The paper introduces Structure of Thought (SoT), a prompting technique that guides LLMs to construct intermediate text structures (nodes and links) before answering, consistently improving performance across eight text-processing tasks and three model families. Building on this, the authors present T2S-Bench, the first benchmark for evaluating…
Qinsi Wang, Hancheng Ye, Jinhee Kim, Jinghan Ke, et al.Together AI
The paper introduces V1, a framework that unifies generation and self-verification for parallel reasoning in LLMs. It identifies that pointwise self-verification suffers from calibration collapse, while self-aggregation methods like RSA lead to diversity collapse. V1 comprises two components: V1-Infer, an uncertainty-guided pairwise ranking algorithm using…
Harman Singh, Xiuyu Li, Kusha Sareen, Monishwaran Maheswaran, et al.arXiv.org
The paper introduces ADE-CoT, a test-time scaling framework for image editing that improves efficiency and performance. It addresses three issues with applying Image-CoT to editing: inefficient fixed sampling budgets, unreliable early-stage verification using general MLLM scores, and redundant results from large-scale sampling. ADE-CoT uses…
Xiangyan Qu, Zhenlong Yuan, Jing Tang, Rui Chen, et al.arXiv.org
The paper introduces Chain of Mindset (CoM), a training-free agentic framework for LLM reasoning that enables step-level adaptive switching among four cognitive mindsets: Spatial, Convergent, Divergent, and Algorithmic. A Meta-Agent dynamically selects the optimal mindset based on the evolving reasoning state, while a bidirectional Context Gate filters…
Tianyi Jiang, Arctanx An, Hengyi Feng, Naixin Zhai, et al.arXiv.org
The paper investigates why large reasoning models (LRMs) produce lengthy, redundant chains of thought (CoTs) despite shorter chains often being more accurate. The authors discover that LRMs implicitly know when to stop thinking, but this capability is obscured by current sampling paradigms like pass@1. They introduce SAGE (Self-Aware Guided Efficient…
Zixuan Huang, Xin Xia, Yuxi Ren, Jianbin Zheng, et al.Annual Meeting of the Association for Computational Linguistics
The paper introduces MAXS, a meta-adaptive reasoning framework for LLM agents that addresses two issues: locally myopic generation and trajectory instability. MAXS uses a lookahead strategy to extend reasoning paths a few steps ahead, estimating the advantage value of tool usage, and combines step consistency variance and inter-step trend slopes to select…
Jian Zhang, Zhiyuan Wang, Zhangqi Wang, Yu He, et al.arXiv.org
This report introduces DASD-4B-Thinking, a lightweight open-source reasoning model that achieves state-of-the-art performance on math, science, and code benchmarks, outperforming larger models (e.g., 32B) using only 448K training samples. The authors critique the common sequence-level distillation paradigm (SFT on teacher-generated responses) for three…
Shaotian Yan, Kaiyuan Liu, Chen Shen, Bing Wang, et al.arXiv.org
The paper proposes that effective long chain-of-thought (Long CoT) reasoning in LLMs requires stable molecular-like structures formed by three interaction types: Deep-Reasoning (covalent-like), Self-Reflection (hydrogen-bond-like), and Self-Exploration (van der Waals-like). Distillation from strong reasoning LLMs (e.g., DeepSeek-R1, QwQ, OpenAI-OSS)…
Qiguang Chen, Yantao Du, Ziniu Li, Jinhao Liu, et al.arXiv.org
The paper introduces MMFORMALIZER, a framework for multimodal autoformalization that translates natural language and visual inputs into formal LEAN statements. It addresses challenges in grounding physical concepts by using recursive grounding and dimensional analysis, with adaptive termination to ensure visual evidence supports abstractions. The authors…
Jing Xiong, Qi Han, Yunta Hsieh, Hui Shen, et al.arXiv.org
The paper introduces Dynamic Large Concept Models (DLCM), a hierarchical language modeling framework that learns variable-length semantic concepts from latent representations and performs reasoning in a compressed concept space, shifting computation from tokens to concepts. DLCM uses a four-stage pipeline: encoding, dynamic segmentation via learned…
Xingwei Qu, Shaowen Wang, Zihao Huang, Kai Hua, et al.arXiv.org
DiffThinker is a diffusion-based framework that introduces Generative Multimodal Reasoning, reformulating multimodal reasoning as a native image-to-image generative task rather than text-centric symbolic mapping. Built on Qwen-Image-Edit with Flow Matching and a Multimodal Diffusion Transformer, it directly produces solution images that are parsed into…
Zefeng He, Xiaoye Qu, Yafu Li, Tong Zhu, et al.arXiv.org
The paper introduces Latent Implicit Visual Reasoning (LIVR), a method to enhance visual reasoning in Large Multimodal Models (LMMs) without explicit intermediate supervision. LIVR adds latent tokens to the model's vocabulary and trains them via a two-stage visual bottlenecking approach: Stage 1 forces visual information to pass through latent tokens by…
Kelvin Li, Chuyi Shang, Leonid Karlinsky, Rogerio Feris, et al.arXiv.org
The paper introduces the Laws of Reasoning (LORE), a framework formalizing desired reasoning behaviors in Large Reasoning Models (LRMs). LORE posits a compute law (reasoning compute scales linearly with question complexity) and an accuracy law (accuracy decays exponentially with complexity). Since complexity is hard to measure, the laws are approximated…
Junyu Zhang, Yifan Sun, Tianang Leng, Jingyan Shen, et al.arXiv.org
T-pro 2.0 is an open-weight Russian LLM for hybrid reasoning and efficient inference, supporting direct answering and reasoning-trace generation. It uses a Cyrillic-dense tokenizer and an adapted EAGLE speculative-decoding pipeline to reduce latency. The authors release the model weights, the T-Wix 500k instruction corpus, the T-Math reasoning benchmark,…
Dmitrii Stoianov, Danil Taranets, Olga Tsymboi, Ramil Latypov, et al.arXiv.org
LatentMAS is a training-free framework enabling multi-agent systems (MAS) to collaborate entirely in the continuous latent space, bypassing text-based communication. Each agent generates latent thoughts via auto-regressive last-layer hidden states, and a shared latent working memory (KV caches) transfers these representations losslessly to subsequent…
Jiaru Zou, Ruizhong Qiu, Gaotang Li, Xiyuan Yang, et al.arXiv.org
The paper introduces Think-at-Hard (TaH), a looped transformer that selectively applies latent iterations to improve reasoning in small language models. The authors identify a 'latent overthinking' problem where always iterating can flip correct predictions into errors. An oracle policy that iterates only on mispredicted tokens boosts accuracy by up to…
Tianyu Fu, Yichen You, Zekai Chen, Guohao Dai, et al.arXiv.org
The paper introduces MIRA (Multimodal Imagination for Reasoning Assessment), a benchmark designed to evaluate multimodal large language models (MLLMs) on tasks that require generating or using intermediate visual images (e.g., sketches, diagrams) for successful reasoning, mirroring human 'drawing to think'. MIRA contains 546 problems across 20 task types…
Yiyang Zhou, Haoqin Tu, Zijun Wang, Zeyu Wang, et al.arXiv.org
ThinkMorph is a unified multimodal model fine-tuned on ~24K interleaved reasoning traces across four tasks (Jigsaw Assembly, Spatial Navigation, Visual Search, Chart Refocus) to enable complementary text-image chain-of-thought reasoning. It outperforms its base model Bagel-7B by an average of 34.74% on vision-centric benchmarks, with gains of 85.84% on…
Jiawei Gu, Yunzhuo Hao, Huichen Will Wang, Linjie Li, et al.arXiv.org
The paper introduces AutoDeco, a novel architecture that makes LLM generation truly end-to-end by learning to predict token-level temperature and top-p values during decoding. This replaces manual hyperparameter tuning with a dynamic, self-regulating process. AutoDeco uses lightweight heads on top of frozen base models, trained with a differentiable soft…
Zhichao Wang, Dongyang Ma, Xinting Huang, Deng Cai, et al.arXiv.org
The paper introduces Ouro, a family of pre-trained Looped Language Models (LoopLM) that integrate iterative computation in latent space during pre-training, using an entropy-regularized objective for adaptive depth allocation. Trained on 7.7T tokens, Ouro 1.4B and 2.6B models match or exceed the performance of 4B and 8B standard transformers, achieving…
Rui-Jie Zhu, Zixuan Wang, Kai Hua, Tianyu Zhang, et al.Neural Information Processing Systems
This paper introduces a theoretical framework for sampling-based test-time scaling in LLM reasoning, decomposing reasoning error into estimation error and model error. It analyzes self-consistency (SC) and perplexity (PPL), finding SC has linear estimation error convergence while PPL has exponential but suffers from high model error and degradation when…
Zhi Zhou, Yuhao Tan, Zenan Li, Yuan Yao, et al.arXiv.org
The paper introduces MM-HELIX, a benchmark of 1,260 samples across 42 multimodal tasks (algorithms, graphs, puzzles, games) with five difficulty levels, designed to evaluate long-chain reflective reasoning in MLLMs. Evaluation of 23 models shows significant deficits; even GPT-5 achieves only 58.1% accuracy, and open-source models lag far behind. To improve…
Xiangyu Zhao, Junming Lin, Tianhao Liang, Yifan Zhou, et al.arXiv (Cornell University)
The paper introduces Tiny Recursive Model (TRM), a simplified recursive reasoning approach that outperforms the Hierarchical Reasoning Model (HRM) and many large language models on hard puzzle tasks. TRM uses a single tiny 2-layer network (7M parameters) that recursively improves its answer, eliminating the need for fixed-point theorems, hierarchical…
Alexia Jolicoeur-MartineauarXiv.org
The paper introduces REverse-Engineered Reasoning (REER), a new paradigm for instilling deep reasoning in LLMs for open-ended, non-verifiable tasks like creative writing, where reinforcement learning (RL) and instruction distillation fail due to lack of clear rewards and high costs. REER works backwards from known good solutions, using a gradient-free…
Haozhe Wang, Haoran Que, Qixin Xu, Minghao Liu, et al.Conference on Empirical Methods in Natural Language Processing
The paper introduces Drivelology, a linguistic phenomenon of syntactically coherent but pragmatically paradoxical utterances that encode implicit meaning requiring contextual inference, moral reasoning, or emotional interpretation. The authors construct DRIVELHUB, a benchmark dataset of over 1,200 curated examples across six languages (English, Mandarin,…
Yang Wang, Chenghao Xiao, Chia-Yi Hsiao, Zi Yan Chang, et al.Meta AI
Deep Think with Confidence (DeepConf) is a test-time method that improves LLM reasoning efficiency and accuracy by using model-internal confidence signals to filter low-quality reasoning traces. It addresses the diminishing returns and high computational overhead of self-consistency with majority voting. DeepConf operates in offline mode (filtering…
Yichao Fu, Xuewei Wang, Yuandong Tian, Jiawei ZhaoAnnual Meeting of the Association for Computational Linguistics
ReasonRank is a reasoning-intensive listwise passage reranker that outperforms existing baselines on reasoning-heavy IR benchmarks. The authors address the scarcity of reasoning-intensive training data by proposing an automated synthesis framework that collects queries from diverse domains (complex QA, coding, math, web search) and uses DeepSeek-R1 to…
Wenhan Liu, Xinyu Ma, Weiwei Sun, Yutao Zhu, et al.
