Paper 2511.05491
Visual Spatial Tuning
- Published
- Nov 2025
- Research lab
- Independent
- Citations
- 64
- GitHub
- 201 stars
01 In brief
Summary
The paper introduces Visual Spatial Tuning (VST), a framework to enhance the spatial perception and reasoning abilities of Vision-Language Models (VLMs) without adding specialized 3D encoders.
VST comprises two datasets: VST-P, with 4.1 million samples across 19 tasks covering single-image, multi-image, and video scenarios, and VST-R, with 135K samples for spatial reasoning, including chain-of-thought (CoT) data and rule-checkable data for reinforcement learning.
Training proceeds in three stages: supervised fine-tuning on VST-P, CoT cold start, and reinforcement learning with GRPO.
The method achieves state-of-the-art results on spatial benchmarks, including 87.8% on CVBench, 34.8% on MMSI-Bench, and 61.2% on VSIBench, while preserving general capabilities.
Ablations show that single-image, multi-image, and video data each contribute to spatial understanding, and that scaling model and data sizes improves performance.
The VST-tuned model also enhances Vision-Language-Action (VLA) models, yielding an 8.6% improvement on the LIBERO benchmark.
02 From the paper
Abstract
Capturing spatial relationships from visual inputs is a cornerstone of human-like general intelligence. Several previous studies have tried to enhance the spatial awareness of Vision-Language Models (VLMs) by adding extra expert encoders, which brings extra overhead and usually harms general capabilities. To enhance the spatial ability in general architectures, we introduce Visual Spatial Tuning (VST), a comprehensive framework to cultivate VLMs with human-like visuospatial abilities, from spatial perception to reasoning. We first attempt to enhance spatial perception in VLMs by constructing a large-scale dataset termed VST-P, which comprises 4.1 million samples spanning 19 skills across single views, multiple images, and videos. Then, we present VST-R, a curated dataset with 135K samples that instruct models to reason in space. In particular, we adopt a progressive training pipeline: supervised fine-tuning to build foundational spatial knowledge, followed by reinforcement learning to further improve spatial reasoning abilities. Without the side-effect to general capabilities, the proposed VST consistently achieves state-of-the-art results on several spatial benchmarks, including $34.8\%$ on MMSI-Bench and $61.2\%$ on VSIBench. It turns out that the Vision-Language-Action models can be significantly enhanced with the proposed spatial tuning paradigm, paving the way for more physically grounded AI.