Paper 2606.13673
SpatialClaw: Rethinking Action Interface for Agentic Spatial Reasoning
01 In brief
Summary
SpatialClaw is a training-free framework that improves spatial reasoning in vision-language models (VLMs) by using code as the action interface.
It maintains a persistent Python kernel pre-loaded with input frames and perception tools, allowing a VLM-backed agent to write and execute one code cell per step, inspect intermediate results (e.g., masks, depth maps), and revise its analysis.
This contrasts with single-pass code execution and structured tool-call interfaces, which offer less flexibility.
Evaluated on 20 spatial reasoning benchmarks, SpatialClaw achieves 59.9% average accuracy, outperforming the recent SpaceTools agent by +11.2 points, with consistent gains across six VLM backbones (Qwen and Gemma4 families, 26B-397B parameters) without any benchmark- or model-specific adaptation.
Ablations show the action interface itself drives gains, with the largest improvements in tasks requiring chained geometric computation across frames and viewpoints.
The framework also includes a planner, security sandbox, and error handling, and its gains are attributed to flexible composition of perception primitives and numerical operations.
02 From the paper
Abstract
Spatial reasoning, the ability to determine where objects are, how they relate, and how they move in 3D, remains a fundamental challenge for vision-language models (VLMs). Tool-augmented agents attempt to address this by augmenting VLMs with specialist perception modules, yet their effectiveness is bounded by the action interface through which those tools are invoked. In this work, we study how the design of this interface shapes the agent's capacity for open-ended spatial reasoning. Existing spatial agents either employ single-pass code execution, which commits to a full analysis strategy before any intermediate result is observed, or rely on a structured tool-call interface that often offers less flexibility for freely composing operations or tailoring the analysis to each task. Both designs offer limited flexibility for open-ended, complex 3D/4D spatial reasoning. We therefore propose SpatialClaw, a training-free framework for spatial reasoning that adopts code as the action interface. SpatialClaw maintains a stateful Python kernel pre-loaded with input frames and a suite of perception and geometry primitives, letting a VLM-backed agent write one executable cell per step conditioned on all prior outputs, enabling the agent to flexibly compose and manipulate perception results and adapt its analysis to both intermediate text and visual observations and the demands of each problem. Evaluated across 20 spatial reasoning benchmarks spanning a broad range of static and dynamic 3D/4D spatial reasoning tasks, SpatialClaw achieves 59.9% average accuracy, outperforming the recent spatial agent by +11.2 points, with consistent gains across six VLM backbones from two model families without any benchmark- or model-specific adaptation.