Paper 2604.25917
Recursive Multi-Agent Systems
- Published
- Apr 2026
- Research lab
- Independent
- Citations
- 3
- GitHub
- 907 stars
01 In brief
Summary
RecursiveMAS is a recursive multi-agent framework that scales agent collaboration by treating the entire system as a unified latent-space recursive computation.
It connects heterogeneous agents via lightweight RecursiveLink modules: inner links enable latent thought generation within each agent, and outer links transfer latent states across agents, forming a collaboration loop.
An inner-outer loop training algorithm co-optimizes the system through shared gradient-based credit assignment across recursion rounds.
Theoretical analyses show RecursiveMAS is more efficient than text-based MAS and maintains stable gradients during training.
Empirically, RecursiveMAS is instantiated across four collaboration patterns (sequential, mixture, distillation, deliberation) and evaluated on nine benchmarks spanning mathematics, science, medicine, search, and code generation.
Compared to advanced baselines, it achieves an average accuracy improvement of 8.3%, 1.2x–2.4x inference speedup, and 34.6%–75.6% token usage reduction.
The framework is structure-agnostic, generalizing across diverse MAS patterns, and shows scaling benefits with deeper recursion.
Training cost analysis shows RecursiveMAS uses lower GPU memory, fewer trainable parameters, and lower cost than LoRA or full fine-tuning while achieving higher accuracy.
Ablations confirm the effectiveness of the residual RecursiveLink design and a moderate latent thought length (around 80 steps) is sufficient for effective collaboration.
Semantic distribution analysis shows deeper recursion aligns generated answers with ground truth, and case studies demonstrate iterative refinement correcting early errors.
Overall, RecursiveMAS provides a scalable and efficient approach to multi-agent collaboration in latent space.
- RecursiveMAS connects agents via inner and outer RecursiveLink modules, enabling latent-space collaboration without text decoding.
- The inner-outer loop training algorithm co-optimizes the whole…
02 From the paper
Abstract
Recursive or looped language models have recently emerged as a new scaling axis by iteratively refining the same model computation over latent states to deepen reasoning. We extend such scaling principle from a single model to multi-agent systems, and ask: Can agent collaboration itself be scaled through recursion? To this end, we introduce RecursiveMAS, a recursive multi-agent framework that casts the entire system as a unified latent-space recursive computation. RecursiveMAS connects heterogeneous agents as a collaboration loop through the lightweight RecursiveLink module, enabling in-distribution latent thoughts generation and cross-agent latent state transfer. To optimize our framework, we develop an inner-outer loop learning algorithm for iterative whole-system co-optimization through shared gradient-based credit assignment across recursion rounds. Theoretical analyses of runtime complexity and learning dynamics establish that RecursiveMAS is more efficient than standard text-based MAS and maintains stable gradients during recursive training. Empirically, we instantiate RecursiveMAS under 4 representative agent collaboration patterns and evaluate across 9 benchmarks spanning mathematics, science, medicine, search, and code generation. In comparison with advanced single/multi-agent and recursive computation baselines, RecursiveMAS consistently delivers an average accuracy improvement of 8.3%, together with 1.2$\times$-2.4$\times$ end-to-end inference speedup, and 34.6%-75.6% token usage reduction. Code and Data are provided in https://recursivemas.github.io.