Paper 2510.04871
Less is More: Recursive Reasoning with Tiny Networks
- Published
- Oct 2025
- Research lab
- Independent
- Citations
- 4
- GitHub
- 6.6K stars
01 In brief
Summary
The paper introduces Tiny Recursive Model (TRM), a simplified recursive reasoning approach that outperforms the Hierarchical Reasoning Model (HRM) and many large language models on hard puzzle tasks.
TRM uses a single tiny 2-layer network (7M parameters) that recursively improves its answer, eliminating the need for fixed-point theorems, hierarchical structures, and complex biological justifications.
Key simplifications include backpropagating through the full recursion process, reinterpreting latent features as a solution and reasoning state, using a single network, and simplifying the halting mechanism.
TRM achieves 87.4% test accuracy on Sudoku-Extreme, 85.3% on Maze-Hard, 44.6% on ARC-AGI-1, and 7.8% on ARC-AGI-2, surpassing HRM (27M parameters) and most LLMs with far fewer parameters.
The authors find that smaller networks generalize better due to reduced overfitting, and that an MLP-based architecture works best for small fixed context lengths.
They also incorporate Exponential Moving Average for stability and find that backpropagating through all recursions is optimal.
TRM demonstrates that recursive reasoning with tiny networks can be highly effective, though scaling laws and theoretical explanations for its success remain open questions.
The approach is supervised and deterministic, with potential extensions to generative tasks.
02 From the paper
Abstract
Hierarchical Reasoning Model (HRM) is a novel approach using two small neural networks recursing at different frequencies. This biologically inspired method beats Large Language models (LLMs) on hard puzzle tasks such as Sudoku, Maze, and ARC-AGI while trained with small models (27M parameters) on small data (around 1000 examples). HRM holds great promise for solving hard problems with small networks, but it is not yet well understood and may be suboptimal. We propose Tiny Recursive Model (TRM), a much simpler recursive reasoning approach that achieves significantly higher generalization than HRM, while using a single tiny network with only 2 layers. With only 7M parameters, TRM obtains 45% test-accuracy on ARC-AGI-1 and 8% on ARC-AGI-2, higher than most LLMs (e.g., Deepseek R1, o3-mini, Gemini 2.5 Pro) with less than 0.01% of the parameters.