The Year in AI Papers
The year/Google DeepMind

Paper 2605.08083

LLMs Improving LLMs: Agentic Discovery for Test-Time Scaling

Tong Zheng, Haolin Liu, Chengsong Huang, Huiwen Bao, Sheng Zhang, Rui Liu, Runpeng Dai, Ruibo Chen, Chenxi Liu, Tianyi Xiong, Xidong Wu, Hongming Zhang, et al.

Published
May 2026
Research lab
Google DeepMind
Citations
0
GitHub
170 stars

01 In brief

Summary

The paper introduces AutoTTS, an environment-driven framework for automatically discovering test-time scaling (TTS) strategies for large language models, shifting the human role from hand-crafting heuristics to constructing discovery environments.

The framework formulates width-depth TTS as controller synthesis over an offline replay environment built from pre-collected reasoning trajectories and probe signals, enabling cheap and deterministic evaluation without repeated LLM calls.

To make search tractable, AutoTTS uses beta parameterization, where a single scalar beta deterministically derives all internal hyperparameters, and provides fine-grained execution trace feedback to help the agent diagnose failures.

Experiments on mathematical reasoning benchmarks (AIME24, AIME25, HMMT25) across four Qwen3 models show the discovered controller improves the accuracy-cost Pareto frontier over hand-crafted baselines (SC@64, ASC, ESC, Parallel-Probe), generalizes to held-out benchmarks and model scales, and transfers to a different model family (DeepSeek-R1-Distill-Llama-8B) and a non-math benchmark (GPQA-Diamond).

The entire discovery process costs only $39.9 and 160 minutes.

Ablations confirm that beta parameterization and execution traces are essential for preventing overfitting and guiding effective search.

The discovered controller, Confidence Momentum Controller (CMC), uses trend-based stopping via EMA momentum, coupled width-depth control, alignment-aware depth allocation, and conservative branch abandonment.

02 From the paper

Abstract

Test-time scaling (TTS) has become an effective approach for improving large language model performance by allocating additional computation during inference. However, existing TTS strategies are largely hand-crafted: researchers manually design reasoning patterns and tune heuristics by intuition, leaving much of the computation-allocation space unexplored. We propose an environment-driven framework, AutoTTS, that changes what researchers design: from individual TTS heuristics to environments where TTS strategies can be discovered automatically. The key to AutoTTS lies in environment construction: the discovery environment must make the control space tractable and provide cheap, frequent feedback for TTS search. As a concrete instantiation, we formulate width--depth TTS as controller synthesis over pre-collected reasoning trajectories and probe signals, where controllers decide when to branch, continue, probe, prune, or stop and can be evaluated cheaply without repeated LLM calls. We further introduce beta parameterization to make the search tractable and fine-grained execution trace feedback to improve discovery efficiency by helping the agent diagnose why a TTS program fails. Experiments on mathematical reasoning benchmarks show that the discovered strategies improve the overall accuracy--cost tradeoff over strong manually designed baselines. The discovered strategies generalize to held-out benchmarks and model scales, while the entire discovery costs only $39.9 and 160 minutes. Our data, and code will be open-source at https://github.com/zhengkid/AutoTTS.