Paper 2607.27180
HumanCLAW: Can Vision-Language Models Act Through a Body?
- Published
- Jul 2026
- Research lab
- Independent
- Citations
- 0
- GitHub
- 58 stars
01 In brief
Summary
HumanCLAW is an evaluation framework that measures whether vision-language models (VLMs) can act through a physical body by decoupling action decision-making from low-level motor execution.
A harnessed VLM issues atomic skill commands (e.g., walk, turn, sit) at sub-second intervals, which are converted into continuous full-body motion by a skill-conditioned generator and executed in a half-physics simulator that preserves gravity, collisions, and object displacement while factoring out balance and motor errors.
The authors introduce HumanCLAW-Bench, comprising 1,218 egocentric find-navigate-interact episodes across 41 indoor scenes.
Testing nine state-of-the-art VLMs, none solves the benchmark; the best model (Gemini-3.1) achieves only a 16.8% success rate.
Error analysis reveals that perception is not the bottleneck—once a target is visible, models recognize it—but they lack embodied self-awareness: they fail to track their own body position, whether they have arrived at the goal, or whether they have collided with obstacles.
Ablations show that a verifier and mid-level reasoning are critical, while longer text or visual history does not improve performance.
The authors conclude that current VLMs reason about the scene but not about the body they control, and propose embodied self-awareness as a key missing capability.
02 From the paper
Abstract
Evaluating whether a vision-language model (VLM) can act through a physical body is challenging. The outcome of an action couples the VLM's decision with motor control. When a task fails, it is hard to tell whether the VLM made a bad choice or the motor controller simply failed to execute it, e.g., losing balance and falling. In this work, we introduce HumanCLAW, an evaluation framework that decouples action decision-making from low-level execution. At every step, a harnessed, off-the-shelf VLM issues an atomic skill command, and the command is translated into a sub-second chunk of continuous full-body motion with real physical consequences, including gravity and collisions. The body can therefore act freely in the physical world, while execution-side disturbances, balance and motor errors, are factored out. What remains measurable is the model's action intelligence: its moment-to-moment choice of what the body should execute next. Based on this framework, we build HumanCLAW-Bench: 1,218 long-horizon, egocentric find-navigate-interact episodes across 41 indoor scenes. We test nine state-of-the-art VLMs and find that none solves the benchmark; the best model reaches only a 16.8% success rate. Recognizing the target is not the bottleneck. What current VLMs lack is embodied self-awareness: they lose track of their own body, failing to tell where it is, whether it has reached the goal, or whether it has hit an obstacle.