Visual Spatial Tuning

摘要

Capturing spatial relationships from visual inputs is a cornerstone of human-like general intelligence, yet current Vision-Language Models (VLMs) still struggle to perform spatial understanding from everyday visual observations. To enhance the spatial ability within general architectures, we introduce Visual Spatial Tuning (VST), a comprehensive framework to cultivate VLMs with human-like visuospatial competence spanning spatial perception and reasoning. VST dataset comprises VST-P, a large-scale perception dataset with 4.1 million samples spanning 19 skills across single views, multiple images, and videos; and VSTR, a reasoning dataset with 135K samples that explicitly instruct models to reason in space. For spatial reasoning, we design Reconstruction with Text in Chain-of-Thought (RT-CoT), which represents scene layout as a textual intermediate and performs reasoning grounded on this reconstruction. We further propose a strategy termed prompting with BEV annotations to improve the faithfulness and coherence of spatial reasoning traces. Then, we adopt a progressive training pipeline: supervised fine-tuning to build foundational spatial perception abilities, followed by reinforcement learning to develop spatial reasoning, where a 3D-aware layout reward is proposed. Without the side-effect on general capabilities, the proposed VST consistently achieves state-of-the-art results on several spatial benchmarks, including 34.8% on MMSI-Bench and 61.2% on VSIBench. Moreover, VST substantially improves Vision-LanguageAction (VLA) models, paving the way for more physically grounded AI.