Paper 2606.02404
K-BrowseComp: A Web Browsing Agent Benchmark Grounded in Korean Contexts
- Published
- Jun 2026
- Research lab
- Independent
- Citations
- 1
- GitHub
- 14 stars
01 In brief
Summary
K-BROWSECOMP is a new benchmark for evaluating web-browsing agents in Korean contexts, consisting of 400 problems.
The 300-problem K-BROWSECOMP-VERIFIED subset was manually constructed and validated by native Korean speakers.
On this subset, frontier LLMs like GPT-5.5, DeepSeek-V4-Pro, and GLM-5.1 achieve only 30.00–45.67% accuracy, a significant drop from their performance on the English BrowseComp benchmark.
Korean LLMs from the government-funded 'Proprietary AI Foundation Model' program score even lower, between 0.00% and 10.33%.
The paper also introduces a 100-problem SYNTHETIC split generated using a failure-mode-targeted approach, where the strongest model reaches only 26.00% accuracy.
Analysis of model trajectories reveals that failures often occur after relevant evidence is retrieved, due to issues like premature candidate commitment, failure to merge evidence branches, and loss of role bindings.
The benchmark is released publicly to support the development of Korean web-browsing agents.
02 From the paper
Abstract
Frontier model evaluations are shifting from foundational capabilities (e.g., instruction following and reasoning) toward compositional, agentic ones, but Korean agentic benchmarks remain scarce. We introduce K-BrowseComp, a web-browsing agent benchmark grounded in Korean contexts, consisting of 400 problems. The 300-problem K-BrowseComp-Verified subset is manually constructed and validated by native Korean speakers. On this subset, frontier LLMs, including GPT-5.5, DeepSeek-V4-Pro, and GLM-5.1, reach only 30.00--45.67\%, a substantial drop from BrowseComp, while Korean LLMs released through Korea's Proprietary AI Foundation Model program obtain only 0.00--10.33\%. We further construct a 100-problem synthetic split using hard few-shot exemplars and failure-mode-targeted generation to exploit the asymmetry between solving and creating web browsing problems. On the adversarially filtered synthetic diagnostic split, the strongest model reaches only 26.00\%, and we report this split separately as a targeted stress test. We publicly release our data and code.