Paper 2605.16257
DexJoCo: A Benchmark and Toolkit for Task-Oriented Dexterous Manipulation on MuJoCo
- Published
- May 2026
- Research lab
- Independent
- Citations
- 5
- GitHub
- 171 stars
01 In brief
Summary
DexJoCo is a benchmark and toolkit for task-oriented dexterous manipulation on MuJoCo, featuring 11 functionally grounded tasks that evaluate tool-use, bimanual coordination, long-horizon execution, and reasoning.
It includes a low-cost teleoperation system using Rokoko gloves and HTC Vive trackers, with a retargeting module (GeoRT) to map human hand motions to the Allegro Hand.
The dataset comprises 1.1K human demonstration trajectories, with support for domain randomization (object placement, table height, camera poses, lighting, textures).
The benchmark evaluates five policies: ACT, Diffusion Policy (DP-T and DP-C), π0.5, and GR00T N1.5.
Results show that π0.5 achieves the highest average success rate (52.5% under rand-obj), but DP-T is competitive on bimanual tasks.
DP-C excels at precise operations like button pressing.
All policies degrade under full visual randomization, and multi-task training reduces performance.
π0.5 shows stronger robustness to dynamics randomization.
Retaining pretrained action-head weights improves performance.
The VLA model fails to exhibit true language generalization, showing mode collapse.
Key limitations identified include lack of hand-centric foundation models, vision-only policies insufficient for contact-rich tasks, and the need for more realistic simulation for sim-to-real transfer.
02 From the paper
Abstract
Achieving human-level manipulation requires dexterous robotic hands capable of complex object interactions. Advancing such capabilities further demands standardized benchmarks for systematic evaluation. However, existing dexterous benchmarks lack tasks that reflect the unique manipulation capabilities of dexterous hands over parallel grippers, as well as comprehensive evaluation pipelines. In this paper, we present DexJoCo, a benchmark and toolkit for task-oriented dexterous manipulation, comprising 11 functionally grounded tasks that evaluate tool-use, bimanual coordination, long-horizon execution, and reasoning. We develop a low-cost data collection system and collect 1.1K trajectories across these tasks, with support for domain randomization to assess robustness. We benchmark modern models under diverse settings, including visual and dynamics randomization, multi-task training, and action-head adaptation. Through extensive empirical analysis, we identify several important insights and common limitations of current policies in dexterous manipulation, highlighting key challenges for future research in dexterous hand robot learning. Project page available at: https://dexjoco.github.io