Paper 2605.19769
OpenComputer: Verifiable Software Worlds for Computer-Use Agents
- Published
- May 2026
- Research lab
- Independent
- Citations
- 6
- GitHub
- 31 stars
01 In brief
Summary
OpenComputer is a verifier-grounded framework for constructing verifiable software worlds for computer-use agents.
It integrates four components: app-specific state verifiers, a self-evolving verification layer, a task-generation pipeline, and an evaluation harness.
The framework covers 33 desktop applications and 1,000 finalized tasks.
Experiments show that hard-coded verifiers align more closely with human adjudication than LLM-as-judge evaluation, especially for fine-grained application state.
Frontier agents like GPT-5.4 achieve a 68.3% success rate, while open-source models show sharp drops from OSWorld-Verified scores, indicating a persistent gap in robust computer automation.
The self-evolving verification layer repairs 89.4% of checker-side errors, improving human-checker agreement from 85.2% to 94.1%.
GUI agents outperform CLI agents on a shared subset, but CLI agents are faster.
The framework is released as extensible infrastructure for training and evaluating computer-use agents.
02 From the paper
Abstract
We present OpenComputer, a verifier-grounded framework for constructing verifiable software worlds for computer-use agents. OpenComputer integrates four components: (1) app-specific state verifiers that expose structured inspection endpoints over real applications, (2) a self-evolving verification layer that improves verifier reliability using execution-grounded feedback, (3) a task-generation pipeline that synthesizes realistic and machine-checkable desktop tasks, and (4) an evaluation harness that records full trajectories and computes auditable partial-credit rewards. In its current form, OpenComputer covers 33 desktop applications and 1,000 finalized tasks spanning browsers, office tools, creative software, development environments, file managers, and communication applications. Experiments show that OpenComputer's hard-coded verifiers align more closely with human adjudication than LLM-as-judge evaluation, especially when success depends on fine-grained application state. Frontier agents struggle with end-to-end completion despite partial progress, and open-source models exhibit sharp drops from their OSWorld-Verified scores, exposing a persistent gap in robust computer automation.