Region-Aware Multimodal Large Language Model via SlowFast Tokenization and Pseudo-Mask Guidance for 3D CT Report Generation

Abstract

Current CT report generation frameworks predominantlyrely on global feature representations, often failing to capture region-specific details and potentially missing certain abnormalities. To over-come this limitation, we propose MedRegion-CT, a region-focused mul-timodal large language model framework featuring three key innova-tions. First, we revisit the SlowFast strategy to jointly model globaland fine-grained information and adapt it to the medical domain via aRegion-based SlowFast Tokenizer that extracts tokens guided by clin-ically meaningful regions. Second, generated pseudo-masks guide themodel to attend to diagnostically important anatomical regions, facilitat-ing a systematic understanding of the overall scan context. Third, quanti-tative lesion information, including size, diameter, and spatial location,is encoded as structured textual prompts, enabling context-aware andclinically informed report generation. To enable rigorous evaluation, wevalidate our framework on multi-institutional structured report genera-tion benchmarks. Experimental results demonstrate that MedRegion-CTachieves state-of-the-art performance, outperforming existing approachesin both linguistic quality and clinical accuracy. All code is publicly avail-able at: https://github.com/babbu3682/MedRegion-CT.