Paper 2511.19399
DR Tulu: Reinforcement Learning with Evolving Rubrics for Deep Research
- Published
- Nov 2025
- Research lab
- Independent
- Citations
- 75
- GitHub
- 692 stars
01 In brief
Summary
The paper introduces DR Tulu-8B, the first fully open model trained end-to-end for long-form deep research tasks, using a new method called Reinforcement Learning with Evolving Rubrics (RLER).
RLER constructs and maintains rubrics that co-evolve with the policy model during training, incorporating newly explored information from search and contrasting model responses to provide better fact-checking and more discriminative on-policy feedback.
DR Tulu-8B is initialized from Qwen3-8B, fine-tuned on high-quality user data, and then trained via RLER.
Across four long-form deep research benchmarks (SQAv2, HealthBench, ResearchQA, DeepResearchBench), DR Tulu-8B outperforms existing open deep research agents by 15.6% on average and matches or exceeds proprietary systems like OpenAI Deep Research, while being significantly smaller and cheaper (1000x cheaper per query).
The model also adapts its search tool usage per task, and the authors release all data, code, and models, including an extensible deep research library and evaluation suite.
02 From the paper
Abstract
Deep research agents perform multi-step research to produce long-form, well-attributed answers. However, most open deep research agents are trained on easily verifiable short-form QA tasks via reinforcement learning with verifiable rewards, which does not extend to realistic long-form tasks. We address this with Reinforcement Learning with Evolving Rubrics (RLER), where rubrics are constructed and maintained to co-evolve with the policy model during training. This allows the rubrics to incorporate newly explored information from search and contrasting model responses, enabling better fact checking and more discriminative on-policy feedback. Using RLER, we develop Deep Research Tulu (DR Tulu-8B), the first fully open model that is directly trained for open-ended, long-form deep research. Across four long-form deep research benchmarks in science, healthcare, and general domains, DR Tulu substantially outperforms existing open deep research agents (by 15.6% over Tongyi DR on average) and matches or exceeds proprietary deep research agents (by 0.7% over OpenAI DR on average), while being significantly smaller and cheaper per query (1000x cheaper than OpenAI DR per query).