Paper 2511.21689
ToolOrchestra: Elevating Intelligence via Efficient Model and Tool Orchestration
01 In brief
Summary
The paper introduces ToolOrchestra, a method for training small language models as orchestrators that coordinate diverse tools, including basic tools (web search, code interpreter), specialized LLMs (math, coding), and generalist LLMs (GPT-5, Claude Opus 4.1).
The resulting 8B-parameter Orchestrator model is trained end-to-end with reinforcement learning (GRPO) using rewards for outcome correctness, efficiency (cost and latency), and user preferences.
On Humanity's Last Exam (HLE), Orchestrator achieves 37.1% accuracy, outperforming GPT-5 (35.1%) while being 2.5x more cost-efficient.
On FRAMES and Tau2-Bench, it surpasses GPT-5 with about 30% of the cost.
The model also generalizes to unseen tools and pricing configurations, and better adheres to user preferences than baselines.
The authors contribute ToolScale, a synthetic dataset for training, and demonstrate that orchestration of smaller models with intelligent tools is more effective and efficient than monolithic approaches.
The work addresses biases in prompted orchestrators, such as over-reliance on specific models, and highlights the potential of composite AI systems for scalable tool-augmented reasoning.
The paper includes detailed analysis of tool use, cost-effectiveness, and generalization, showing Orchestrator's superior performance-cost trade-off across benchmarks.
The authors conclude that orchestration with lightweight models paves the way for practical, scalable agentic systems.
The study was conducted at NVIDIA and the University of Hong Kong, with code, model, and data to be released.
The paper is dated November 27, 2025, and is available on arXiv (2511.21689).
The work was supported by NVIDIA, and the authors used GPT-5 for writing polish.
The paper includes extensive appendices on data synthesis, tool descriptions…
02 From the paper
Abstract
Large language models are powerful generalists, yet solving deep and complex problems such as those of the Humanity's Last Exam (HLE) remains both conceptually challenging and computationally expensive. We show that small orchestrators managing other models and a variety of tools can both push the upper bound of intelligence and improve efficiency in solving difficult agentic tasks. We introduce ToolOrchestra, a method for training small orchestrators that coordinate intelligent tools. ToolOrchestra explicitly uses reinforcement learning with outcome-, efficiency-, and user-preference-aware rewards. Using ToolOrchestra, we produce Orchestrator, an 8B model that achieves higher accuracy at lower cost than previous tool-use agents while aligning with user preferences on which tools are to be used for a given query. On HLE, Orchestrator achieves a score of 37.1%, outperforming GPT-5 (35.1%) while being 2.5x more efficient. On tau2-Bench and FRAMES, Orchestrator surpasses GPT-5 by a wide margin while using only about 30% of the cost. Extensive analysis shows that Orchestrator achieves the best trade-off between performance and cost under multiple metrics, and generalizes robustly to unseen tools. These results demonstrate that composing diverse tools with a lightweight orchestration model is both more efficient and more effective than existing methods, paving the way for practical and scalable tool-augmented reasoning systems.