Paper 2601.16175
Learning to Discover at Test Time
- Published
- Jan 2026
- Research lab
- Together AI
- Citations
- 71
- GitHub
- 613 stars
01 In brief
Summary
The paper introduces TTT-Discover, a method that performs reinforcement learning at test time to solve scientific discovery problems.
Unlike prior test-time scaling methods that use a frozen LLM for search, TTT-Discover continues to train the LLM on the specific test problem, using an entropic objective and PUCT-based state reuse to prioritize the most promising solutions.
The method is evaluated on problems in mathematics, GPU kernel engineering, algorithm design, and biology, achieving new state-of-the-art results in almost all of them using the open model gpt-oss-120b.
Key results include improving the upper bound for Erdős' minimum overlap problem to 0.380876, achieving a 2x faster GPU kernel in the GPUMode TriMul competition, winning AtCoder heuristic contests, and improving denoising scores in single-cell analysis.
The method's success is attributed to its focus on maximizing the best solution rather than average performance, and its ability to learn from its own search attempts.
The paper also includes ablations showing the importance of the entropic objective and PUCT reuse, and expert reviews validating the results.
The cost of a training run is a few hundred dollars, and the code is publicly available.
The method is limited to problems with continuous rewards, and future work will address sparse or binary rewards and non-verifiable domains.
The paper also discusses related work in continual learning and test-time training, and compares TTT-Discover to concurrent methods like ThetaEvolve, showing significant improvements with the same model and compute budget.
The authors report results for every problem they attempted, and the method sets new…
02 From the paper
Abstract
How can we use AI to discover a new state of the art for a scientific problem? Prior work in test-time scaling, such as AlphaEvolve, performs search by prompting a frozen LLM. We perform reinforcement learning at test time, so the LLM can continue to train, but now with experience specific to the test problem. This form of continual learning is quite special, because its goal is to produce one great solution rather than many good ones on average, and to solve this very problem rather than generalize to other problems. Therefore, our learning objective and search subroutine are designed to prioritize the most promising solutions. We call this method Test-Time Training to Discover (TTT-Discover). Following prior work, we focus on problems with continuous rewards. We report results for every problem we attempted, across mathematics, GPU kernel engineering, algorithm design, and biology. TTT-Discover sets the new state of the art in almost all of them: (i) Erd\H{o}s'minimum overlap problem and an autocorrelation inequality; (ii) a GPUMode kernel competition (up to $2\times$ faster than prior art); (iii) past AtCoder algorithm competitions; and (iv) denoising problem in single-cell analysis. Our solutions are reviewed by experts or the organizers. All our results are achieved with an open model, OpenAI gpt-oss-120b, and can be reproduced with our publicly available code, in contrast to previous best results that required closed frontier models. Our test-time training runs are performed using Tinker, an API by Thinking Machines, with a cost of only a few hundred dollars per problem.