The Year in AI Papers
The year/Independent research

Paper 2509.06501

WebExplorer: Explore and Evolve for Training Long-Horizon Web Agents

Junteng Liu, Yunji Li, Chi Zhang, Jingyang Li, Aili Chen, Ke Ji, Weiyu Cheng, Zijia Wu, Chengyu Du, Qidi Xu, Jiayuan Song, Zhengmao Zhu, et al.

Published
Sep 2025
Research lab
Independent
Citations
74
GitHub
120 stars

01 In brief

Summary

WebExplorer introduces a data synthesis framework for training long-horizon web agents.

It uses model-based exploration, where LLMs iteratively search and browse from a seed entity to construct an information space, and iterative long-to-short query evolution, which removes salient clues and adds obfuscation to increase query difficulty.

This produces the WebExplorer-QA dataset (~40K pairs), which is more challenging than prior datasets (e.g., Claude-4-Sonnet accuracy drops from 86.6% to 67.1% after evolution).

Training on Qwen3-8B uses supervised fine-tuning (13K samples) followed by GRPO reinforcement learning (12K samples) with progressive context expansion to 128K tokens and 100 tool-call turns.

The resulting WebExplorer-8B achieves state-of-the-art results at its scale: 15.7% on BrowseComp-en, 32.0% on BrowseComp-zh, 62.7% on WebWalkerQA, 75.7% on FRAMES, and 17.3% on HLE, outperforming larger models like WebSailor-72B.

RL training increases average tool calls from ~11 to over 16, correlating with improved performance.

02 From the paper

Abstract

The paradigm of Large Language Models (LLMs) has increasingly shifted toward agentic applications, where web browsing capabilities are fundamental for retrieving information from diverse online sources. However, existing open-source web agents either demonstrate limited information-seeking abilities on complex tasks or lack transparent implementations. In this work, we identify that the key challenge lies in the scarcity of challenging data for information seeking. To address this limitation, we introduce WebExplorer: a systematic data generation approach using model-based exploration and iterative, long-to-short query evolution. This method creates challenging query-answer pairs that require multi-step reasoning and complex web navigation. By leveraging our curated high-quality dataset, we successfully develop advanced web agent WebExplorer-8B through supervised fine-tuning followed by reinforcement learning. Our model supports 128K context length and up to 100 tool calling turns, enabling long-horizon problem solving. Across diverse information-seeking benchmarks, WebExplorer-8B achieves the state-of-the-art performance at its scale. Notably, as an 8B-sized model, WebExplorer-8B is able to effectively search over an average of 16 turns after RL training, achieving higher accuracy than WebSailor-72B on BrowseComp-en/zh and attaining the best performance among models up to 100B parameters on WebWalkerQA and FRAMES. Beyond these information-seeking tasks, our model also achieves strong generalization on the HLE benchmark even though it is only trained on knowledge-intensive QA data. These results highlight our approach as a practical path toward long-horizon web agents.