Cheers: Decoupling Patch Details from Semantic Representations Enables Unified Multimodal Comprehension and Generation

摘要

A recent cutting-edge topic in multimodal modeling is tounify visual comprehension and generation within a single model. How-ever, the two tasks demand mismatched decoding regimes and visualrepresentations, making it non-trivial to jointly optimize within a sharedfeature space. In this work, we present Cheers, a unified multimodalmodel that decouples patch-level details from semantic representations,thereby stabilizing semantics for multimodal understanding and improv-ing fidelity for image generation via gated detail residuals. Cheers in-cludes three key components: (i) a unified vision tokenizer that encodesand compresses image latent states into semantic tokens for efficient LLMconditioning, (ii) an LLM-based Transformer that unifies autoregressivedecoding for text generation and diffusion decoding for image generation,and (iii) a cascaded flow matching head that decodes visual semanticsfirst and then injects semantically gated detail residuals from the vi-sion tokenizer to refine high-frequency content. Experiments on popularbenchmarks demonstrate that Cheers matches or surpasses advancedUMMs in both visual understanding and generation. Notably, Cheersoutperforms the Tar-1.5B on the popular benchmarks GenEval and MM-Bench, while requiring only 20% of the training cost, indicating effectiveand efficient (i.e., 4× token compression) unified multimodal modeling.Code and checkpoints are available at github.com/AI9Stars/CHEERS.