ProLaViT: Learning Progressive Latent Visual Thoughts in Structured Latent Space

Abstract

Multimodal Large Language Models (MLLMs) have achievedremarkable progress but still struggle with complex visual reasoningtasks requiring multi-step perception and logical deduction. While ex-plicit visual generation incurs prohibitive computational costs, existinglatent approaches often rely on external experts or lack rigorous cogni-tive logic. In this paper, we introduce ProLaViT (Progressive LatentVisual Thought), a framework empowering MLLMs to perform struc-tured visual derivation in the continuous latent space. Unlike worksdependent on heterogeneous external models, ProLaViT leverages anendogenous self-distillation mechanism, utilizing the model’s own vi-sual encoder to supervise latent thoughts. To facilitate this, we con-struct a scalable programmatic synthesis pipeline enabling the modelto internalize algorithmic precision without inference-time tools. We de-sign two reasoning paradigms: (1) Coarse-to-Fine Causal Chain for spa-tial tasks, guiding attention from global context to local targets. (2)Dialectical Reasoning Chain for logical tasks, incorporating counter-factual thinking for verification. Furthermore, we propose a Distance-Weighted Diversity Loss to impose topology-aware constraints, prevent-ing feature degeneration by enforcing semantic distinctiveness. Exten-sive experiments demonstrate that ProLaViT outperforms baselines onvision-centric benchmarks, achieving superior accuracy and interpretabil-ity with high efficiency.