The Year in AI Papers
The year/Independent research

Paper 2509.15221

ScaleCUA: Scaling Open-Source Computer Use Agents with Cross-Platform Data

Zhaoyang Liu, Jingjing Xie, Zichen Ding, Zehao Li, Bowen Yang, Zhenyu Wu, Xuehui Wang, Qiushi Sun, Shi Liu, Weiyun Wang, Shenglong Ye, Qingyun Li, et al.

Published
Sep 2025
Research lab
Independent
Citations
46
GitHub
1.1K stars

01 In brief

Summary

ScaleCUA introduces a large-scale, cross-platform dataset and model family for computer use agents (CUAs), addressing data scarcity via a dual-loop pipeline combining automated agents and human experts across six platforms (Windows, macOS, Linux, Android, iOS, Web).

The dataset includes 471K GUI understanding examples, 17.1M grounding annotations, and 19K trajectories.

Trained on Qwen2.5-VL, ScaleCUA supports three inference modes: grounding, direct action, and reasoned action.

It achieves state-of-the-art results on MMBench-GUI L1-Hard (94.4%), OSWorld-G (60.6%), and WebArena-Lite-v2 (47.4%), with strong gains over baselines (+26.6 on WebArena-Lite-v2, +10.7 on ScreenSpot-Pro).

Ablations show data augmentation, weak-semantic trajectories, raw coordinates, and higher resolutions improve grounding, while general data mixing trades off GUI vs.

general performance.

Limitations include data quality gaps, lack of advanced agentic mechanisms, and rudimentary memory.

02 From the paper

Abstract

Vision-Language Models (VLMs) have enabled computer use agents (CUAs) that operate GUIs autonomously, showing great potential, yet progress is limited by the lack of large-scale, open-source computer use data and foundation models. In this work, we introduce ScaleCUA, a step toward scaling open-source CUAs. It offers a large-scale dataset spanning 6 operating systems and 3 task domains, built via a closed-loop pipeline uniting automated agents with human experts. Trained on this scaled-up data, ScaleCUA can operate seamlessly across platforms. Specifically, it delivers strong gains over baselines (+26.6 on WebArena-Lite-v2, +10.7 on ScreenSpot-Pro) and sets new state-of-the-art results (94.4% on MMBench-GUI L1-Hard, 60.6% on OSWorld-G, 47.4% on WebArena-Lite-v2). These findings underscore the power of data-driven scaling for general-purpose computer use agents. We will release data, models, and code to advance future research: https://github.com/OpenGVLab/ScaleCUA.