ECCV 2026 Papers

Despite the advancements of Large Multimodal Models (LMMs)in RGB vision, their ability to generalize to unseen visual modalitiesremains a largely unexplored challenge. We argue that different visualmodalities are merely distinct samplings of the same physical world.Therefore, effective generalization requires models to possess both modality-agnostic perception of scene semantics and the adaptability to modality-specific characteristics. To achieve this, we propose a training frame-work, VVM-Tuning, to equip LMMs with these capabilities throughmodality synthesis and modality contexts. Specifically, we synthesize di-verse appearance-varied images from RGB scenes, training the model todisentangle invariant semantics from varying visual appearances, andalign these appearances with language for visual concepts decoupledfrom modalities. We then introduce modality contexts in the promptand use instruction tuning to assist the model in mapping these appear-ance variations back to modality-related attributes, enabling zero-shotadaptation to unseen modalities during inference. To facilitate researchin this direction, we introduce VVM-Bench, a comprehensive bench-mark featuring 6 real and synthetic modalities to evaluate semantic per-ception and modality understanding. Experiments demonstrate that, viaour training on synthetic modalities, 5 tested models exhibit consistentimprovements on both real-world and novel synthetic modalities withoutin-modality training. Source code and data will be publicly available athttps://github.com/Hunter-Will/VVM-Tuning.

Diffusion Transformers (DiTs) have become a leading back-bone for video generation, yet their quadratic attention cost remains amajor bottleneck. Sparse attention reduces this cost by computing only asubset of attention blocks. However, prior methods often either drop theremaining blocks which incurs information loss, or rely on learned predic-tors to approximate them, introducing training overhead and potentialoutput distribution shifting. In this paper, we show that the missingcontributions can be recovered without training: after semantic cluster-ing, keys and values within each block exhibit strong similarity and canbe well summarized by a small set of cluster centroids. Based on thisobservation, we introduce SVG-EAR, a parameter-free linear compensa-tion branch that uses the centroid to approximate skipped blocks andrecover their contributions. While centroid compensation is accurate formost blocks, it can fail on a small subset. Standard sparsification typi-cally selects blocks by attention scores, which indicate where the modelplaces its attention mass, but not where the approximation error wouldbe largest. SVG-EAR therefore performs error-aware routing: a lightweightprobe estimates the compensation error for each block, and we computeexactly the blocks with the highest error-to-cost ratio while compensatingfor skipped blocks. We provide theoretical guarantees that relate atten-tion reconstruction error to clustering quality, and empirically show thatSVG-EAR improves the quality-efficiency trade-off and increases through-put at the same generation fidelity on video diffusion tasks. Overall,SVG-EAR establishes a clear Pareto frontier over prior approaches, achiev-ing up to 1.77× and 1.93× speedups while maintaining PSNRs of up to29.759 and 31.043 on Wan2.2 and HunyuanVideo, respectively.

Concept Bottleneck Models (CBMs) promise transparent rea-soning by predicting through human-interpretable concepts, yet theireffectiveness fundamentally depends on how well visual and textual rep-resentations are aligned or matched. Existing vision–language CBMs of-ten rely on pre-aligned encoders or global cosine similarity, which ob-scures fine-grained concept localization and fails to reflect true seman-tic geometry. In this work, we rethink concept alignment as dynamiccross-modal transport process instead of static projection and proposeOptimal Transport Flow Concept Bottleneck Model (OTF-CBM). Itfirst learns a data-driven semantic cost via Inverse Optimal Transport tomeasure cross-modal distances, and then performs unbalanced optimal-transport-based flow matching to model semantic transitions betweenvisual patches and textual concepts. With velocity-based concept activa-tion, OTF-CBM captures interpretable geometric relations without ODEintegration. Experiments further show that OTF-CBM achieves greatclassification accuracy and concept faithfulness, offering a new geomet-ric and dynamical perspective for interpretable cross-modal reasoning.Our code can be found at github.com/ChenyangZhang00/OTF-CBM.

In this paper, we present BLOB-Q, a novel Mixed-PrecisionPost Training Quantization (MPQ) approach for Vision Transformers(ViTs). MPQ aims to assign different bit-widths across layers under amodel size budget. Most existing MPQ methods either rely on layerwisesensitivity heuristics or surrogate objectives. These approaches simplifyoptimization, but they no longer optimize the same model-level objec-tive that determines accuracy. Others solve the global MPQ problemby non-heuristic and non-analytical algorithms, such as ReinforcementLearning, genetic algorithms or gradient-based learning. However, theyare inefficient for modern large vision transformers. In this work, we revisitthe global MPQ problem and show that this problem can be solved byefficient analytical algorithms while still maintaining its global-optimality.This comes from two key insights. Firstly, through systematic signalanalysis on quantization errors, we discover that under 4„6-bit range,the quantization error on converged ViTs are indeed small perturbations.This encourages that the global MPQ objective is possible to be quadrat-ically decomposed for empirical ViT models. Secondly, motivated by thesmall perturbation observation, we further discover that the global MPQobjective satisfies an additivity property. Utilizing the additivity property,the NP-hard global MPQ optimization problem can be decomposed intosub-problems and solved practically, while still retain the global optimal-ity. Specifically, we solve the decomposed analytical optimization problemusing dynamic programming algorithm, which efficiently find the globallyoptimal solution with only linear time complexity. Extensive experimentson numerous ViT models demonstrate the effectiveness of our approach.Results show that BLOB-Q significantly improves state-of-the-art andcan further reduce the size of ViT models to 4 bits to 6 bits withouthurting ImageNet accuracy. Moreover, BLOB-Q is highly efficient, onlyrequiring on average sub-2 minutes for regular ViT-S to ViT-B models.

Precise medical image segmentation is crucial for clinical di-agnosis and treatment planning, yet relies heavily on expensive expertannotations. Semi-supervised medical image segmentation (SSMIS) of-fers a cost-effective solution but typically operates under the assump-tion of independent and identically distributed (i.i.d.) data, defaultingto random sampling. While statistically valid at scale, this strategy suf-fers from severe representation bias in low-data regimes, failing to cap-ture the heterogeneous medical data manifold. To address this, we pro-pose a highly data-efficient framework driven by distribution alignment.First, we introduce an offline Distribution-Aware Sample Selection strat-egy. By leveraging Vision Foundation Models (VFMs) and our designedDensity-K-Center algorithm, we explicitly identify representative struc-tural anchors, establishing a more representative labeled domain. Second,to bridge the remaining distribution gap, we propose the Memory-guidedCopy-Paste (MCP) module. Tailored for the inherent class imbalance inmedical scans, MCP leverages a semantic memory mechanism to retrievehistorically consistent priors for cross-domain alignment, encouragingsemantic consistency. Coupled with an easy-to-hard progressive sched-ule, this framework effectively mitigates early-stage pseudo-label noise.Extensive experiments on six diverse 2D and 3D datasets demonstratestrong segmentation performance, particularly in extremely low-labeledscenarios (e.g., 1/16 ratio).

Text-driven inversion of generative models is a core paradigmfor manipulating 2D or 3D content, unlocking numerous applicationssuch as text-based editing, style transfer, or inverse problems. How-ever, it relies on the assumption that generative models remain sen-sitive to natural language prompts. We demonstrate that for state-of-the-art native text-to-3D generative models, this assumption often col-lapses. We identify a critical failure mode where generation trajectoriesare drawn into latent “sink traps”: regions where the model becomesinsensitive to prompt modifications. In these regimes, changes to theinput text fail to alter internal representations in a way that altersthe output geometry. Crucially, we observe that this is not a limita-tion of the model’s geometric expressivity; the same generative mod-els possess the ability to produce a vast diversity of shapes but, as wedemonstrate, become insensitive to out-of-distribution text guidance. Weinvestigate this behavior by analyzing the sampling trajectories of thegenerative model, and find that complex geometries can still be repre-sented and produced by leveraging the model’s unconditional generativeprior. This leads to a more robust framework for text-based 3D shapeediting that bypasses latent sinks by decoupling a model’s geometricrepresentation power from its linguistic sensitivity. Our approach ad-dresses the limitations of current 3D pipelines and enables high-fidelitysemantic manipulation of out-of-distribution 3D shapes. Project web-page: https://daidedou.sorpi.fr/publication/beyondprompts

While Unified Multimodal Models (UMMs) show remarkablereasoning capabilities, their spatial intelligence remains limited to pas-sive 2D Question-Answering (QA). In this paper, we argue that true spa-tial intelligence demands active construction: not only recognizing a 3Dstructure in pixel space, but also building and modifying it. We introduceMindBlock, a benchmark that challenges models’ active generative con-struction in pixel space across two primary axes: Spatial Assembly,which evaluates step-by-step compositional and causal reasoning, andSpatial Structure, which probes spatial equivariance through localsub-component rotations and global viewpoint transformations. To movebeyond pixel-level metrics, we propose 3DGS-Eval, a novel validationprotocol using 3D Gaussian Splatting to reconstruct implicit scenes frommodel-generated multi-view images. This allows us to quantify structuralconsistency, verifying for the first time whether a model’s generative out-put admits a coherent internal 3D world model. Furthermore, we con-duct a deep-dive diagnostic analysis into the representational groundingof spatial logic, disentangling whether structural consistency relies ontextual Chain-of-Thought (CoT) as a symbolic scaffold, or emerges as anative spatial intuition within the generative latent space. Our findingsreveal a significant “perception-execution” gap: while current models cor-rectly identify the intended spatial state, they fail at active construction.They struggle to maintain spatial equivariance without explicit symbolicscaffolding. MindBlock provides a rigorous foundation for the next gen-eration of embodied, physically-grounded multimodal AI.

Visual reasoning may require models to interpret images/videosand respond to implicit text queries across diverse output formats, frompixel-level segmentation masks to natural language descriptions. Exist-ing approaches rely on supervised fine-tuning with task-specific architec-tures. For example, reasoning segmentation, grounding, summarization,and visual question answering each demand distinct model designs andtraining, preventing unified solutions and limiting cross-task/-modalitygeneralization. Hence, we propose DT-R1, a reinforcement learning frame-work that trains large language models (LLMs) to construct digital twin(DT) representations of complex multi-modal visual inputs and then rea-son over these high-level representations as a unified approach to visualreasoning. Specifically, we train DT-R1 using GRPO with a novel rewardthat validates both structural integrity and output accuracy. Evaluationsin six visual reasoning benchmarks, covering two modalities and four tasktypes, demonstrate that DT-R1 consistently achieves improvements overstate-of-the-art task-specific models. DT-R1 opens a new direction wherevisual reasoning emerges from reinforcement learning on with DT repre-sentations.

Continual visual question answering (VQA) requires learningnew question types while retaining previously acquired visual ground-ing and reasoning. Existing methods mainly reduce parameter interfer-ence, but do not explicitly control how sequential adaptation changesthe coupling between a question and its visual evidence. We proposeQ-STEER, a task-agnostic framework that separates plasticity from re-tention through a shared question-conditioned controller. For plasticity,probe-based MoE-LoRA self-expansion activates new expert slots onlywhen the current capacity is insufficient. For retention, a KL-based late-layer attention-drift signal conditions a low-rank correction to attentionlogits; the drift is diagnostic rather than an attention-matching loss. Oncontinual VQA v2, Q-STEER obtains 53.68 average performance and4.51 average forgetting, and reaches 51.00/51.48 on Novel/Seen compo-sitional splits. Experiments on a heterogeneous visual-instruction stream,together with seed, ablation, grounding, sensitivity, and overhead anal-yses, support the effectiveness of drift-aware steering and on-demandexpansion.

Proactive tamper localization embeds an imperceptible sig-nal into an image prior to distribution, enabling pixel-level manipulationdetection. Existing methods assume a spliced (SP) setting, where syn-thesized regions are composited onto the original background, leavingembedded signals intact. However, real-world diffusion-based inpaint-ing operates in a fully regenerated (FR) setting, where the entire im-age undergoes denoising, disrupting background signals and renderingexisting frameworks ineffective. We propose APT, a semi-fragile latent-space perturbation that embeds a dense, vector-wise localization signal.By aligning each spatial feature vector toward a fixed anchor direction,APT localizes tampering via the alignment disparity between synthesizedforeground and anchor-aligned background features after inpainting. Theproposed hard negative mining loss and noisy perturbation branch fur-ther enforce uniform alignment. Experiments on COCO demonstratethat APT achieves an FR IoU of 0.92, outperforming the strongest base-line (WAM, 0.84), while existing methods collapse to near-random per-formance (AUC ≈ 0.5), establishing APT as a practical forensic frame-work generalizable across tampering types unknown at test time.

Multimodal Large Language Models (MLLMs) achieve re-markable progress in widespread vision-language tasks, while achievingfaithful MLLM reasoning remains a critical challenge: generating con-tent grounded in comprehensive visual evidence without hallucination.This challenge often stems from the inherent perceptual limitations ofthe single visual encoder design in most MLLMs. While many workshave turned to integrating multiple vision experts, existing methods failto leverage the unique insights of diverse experts due to unspecializedmixture and alignment strategies. To resolve these challenges, we in-troduce MoSVE (Mixture of Specialized Vision Experts), a holisticframework built upon the core insight of inter-expert complementarity.To achieve fine-grained mixture, we introduce Query-guided Com-plementary Clustering, which selectively preserves text-critical andinformative visual perception without redundancy. To cultivate expertspecialization, we propose Complementary Rejection Fine-Tuning,which explicitly routes hard samples from the anchor MLLM to the mostvisually-disparate auxiliary expert, enhancing unique insights withouthomogenization. Extensive experiments demonstrate that MoSVE notonly mitigates hallucinations across the POPE, CHAIR and MMVP, butalso improves general multimodal reasoning on MMBench. Ultimately,MoSVE provides an advanced and efficient solution for faithful MLLM

Existing advanced video frame interpolation (VFI) methods strug-gle to achieve accurate per-pixel motion or object-level motion estimation. Thereasons lie in that pixel-level motion estimation allows for infinite possibilities,making it challenging to guarantee global motion consistency. Conversely, lo-cal pixel-level motion tends to be discontinuous whereas object-level motionestimation is typically optimized for global motion. Therefore, a hierarchicalmotion learning scheme is imperative to enhance the consistency and continu-ity of motion prediction in VFI. To this end, we marry the object-level motionto the pixel-level motion to construct a hierarchical motion estimation frame-work. This approach elaborately incorporates object priors derived from open-world knowledge models, such as Segment Anything Model (SAM), to facili-tate latent object-level motion learning. In particular, a hybrid contextual fea-ture extraction module (HCE) is employed to aggregate both pixel-wise and se-mantic representations. This is further reinforced by hierarchical motion loss(HML) and hierarchical feature loss (HFL), which respectively simulate the con-sistent and continuous motion patterns for individual objects. Our frameworksupports two configurations: a full variant with HCE, HML, and HFL for higheraccuracy, and an efficient variant with only HML and HFL, which introducesno additional inference overhead. Extensive experiments demonstrate consistentimprovements over state-of-the-art VFI methods across multiple benchmarks.https://github.com/hhhhhumengshun/SAM-VFI

Image Quality Assessment (IQA) in open-world environmentsremains challenging due to limited generalization and interpretability.Recent approaches based on multimodal large language models (MLLMs)introduce textual reasoning for quality prediction, yet their judgmentsrely heavily on semantically biased internal representations, making theminsensitive to low-level perceptual degradations. We propose IQA-T1, atool-based visual evidence reasoning framework that augments MLLMreasoning with explicit perceptual observations. During inference, themodel autonomously invokes specialized analysis tools to generate struc-tured visual evidence, such as noise residual maps, gradient statistics,and frequency spectra, which are progressively integrated into the rea-soning process. To support this paradigm, we construct Q-Tool, a datasetcontaining 11k multimodal reasoning chains grounded in tool-generatedevidence. Extensive experiments on seven IQA benchmarks show thatIQA-T1 achieves the best overall performance across datasets while pro-ducing interpretable and evidence-grounded quality assessments. Codeand dataset are available at https://github.com/zibuyu-02/IQA-T1.

Dataset condensation for action segmentation synthesizescompact, informative representations of long, untrimmed video datasets.The existing approach relies on Variational Autoencoders and an iter-ative latent optimization; it is computationally expensive and suffersfrom over-smoothed reconstructions and rigid temporal constraints. Thispaper proposes to shift the condensation paradigm from optimization-based inversion to deterministic latent mapping. By leveraging Denois-ing Diffusion Implicit Models, we represent action segments as continu-ous trajectories anchored by sparse latent points in the noise manifold.To maximize representational efficiency, we introduce an adaptive allo-cation mechanism that dynamically redistributes the anchoring budgetbased on segment-wise reconstruction difficulty. Extensive experimentsdemonstrate that our framework significantly outperforms state-of-the-art methods in segmentation performance across common datasets. No-tably, our approach achieves performance parity with real data trainingwhile maintaining a condensation ratio of 2.4% on Breakfast dataset.

Semi-supervised medical image segmentation (SSMIS) aimsto achieve accurate segmentation with limited labeled data. However,existing methods still struggle in multi-organ scenarios. Limited anno-tations exacerbate class imbalance and feature entanglement among co-occurring organs, leading to false activations and performance degrada-tion. To overcome these challenges, we propose a dual-space patch-drivencomplementary learning framework for semi-supervised multi-organ seg-mentation (DPCL). Our framework addresses these problems from bothimage and feature space. In the image space, we combine uncertaintyand class frequency to form a probabilistic sampling strategy for im-age decomposition, which generates informative patches to enhance localfeature perception and promote balanced learning across organs. In thefeature space, we design two complementary objectives to achieve ro-bust feature disentanglement. The patch-based contrastive loss focuseson decoupling features of co-occurring neighboring organs within lo-cal patches, mitigating semantic confusion among adjacent structures.Meanwhile, the global-based contrastive loss minimizes the gap betweenglobal and local semantics, improving model sensitivity to fine-graineddetails and enhancing consistency across different regions. These com-plementary losses collaboratively enhance inter-organ feature discrimi-nation while mitigating false activations. Extensive evaluations on fourpublic multi-organ medical segmentation datasets (Synapse, FeTA2021,SegTHOR, and ACDC) demonstrate that DPCL outperforms state-of-the-art methods, achieving substantial improvements in segmentationperformance.

Current generative models, including GANs and diffusionmodels, have reached an outstanding level of photorealism, posing sig-nificant risks to privacy and security. To ensure real-world applicability,deepfake detectors must generalise effectively to unseen generators. How-ever, most existing approaches rely on supervised training with both realand fake images, which limits their generalisation especially across gener-ators categories (e.g. GANs vs DMs). In this work, we introduce µFlow ,a one-class deepfake detector trained only on real images without relyingon pseudo-deepfakes or synthetic artifacts. Our approach builds on theobservation that averaging multiple images amplifies consistent genera-tive traces, producing highly discriminative feature representations. Weleverage this property by modelling the distribution of features extractedfrom averaged images and training a normalizing flow to align the featurespace of individual images with this distribution. This alignment yieldsa likelihood-based criterion that separates real and fake samples whilepromoting strong generalisation. We evaluate µFlow on a fully out-of-distribution setting, where both real and fake datasets are unseen duringtraining. Experimental results show that our method significantly out-performs SOTA detectors. Project page: opontorno.github.io/MuFlow.

Vision-Language-Action (VLA) models aim to unify per-ception, language understanding, and action generation, offering strongcross-task and cross-scene generalization with broad impact on embodiedAI. However, current VLA models often lack explicit step-by-step rea-soning, instead emitting final actions without considering affordance con-straints or geometric relations. Their post-training pipelines also rarelyreinforce reasoning quality, relying primarily on supervised fine-tuningwith weak reward design. To address these challenges, we present VLA-R1, a reasoning-enhanced VLA that integrates Reinforcement Learningfrom Verifiable Rewards (RLVR) with Group Relative Policy Optimiza-tion (GRPO) to systematically optimize both reasoning and execution.Specifically, we design an RLVR-based post-training strategy with ver-ifiable rewards for region alignment, trajectory consistency, and outputformatting, thereby strengthening reasoning robustness and executionaccuracy. Moreover, we develop VLA-CoT-13K, a high-quality datasetthat provides chain-of-thought supervision explicitly aligned with affor-dance and trajectory annotations. Furthermore, extensive evaluations onin-domain, out-of-domain, simulation, and real-robot platforms demon-strate that VLA-R1 achieves superior generalization and real-world per-formance compared to prior VLA methods. We plan to release the model,code, and dataset following the publication of this work.

World models have demonstrated significant promise for data synthesis in autonomous driving. However, existing methods predominantly concentrate on single-modality generation, typically focusing on either multi-camera video or LiDAR sequence synthesis. In this paper, we propose UniDriveDreamer, a single-stage unified multimodal world model for autonomous driving, which directly generates multimodal future observations without relying on intermediate representations or cascaded modules. Our framework introduces a LiDAR-specific variational autoencoder (VAE) designed to encode input LiDAR sequences, alongside a video VAE for multi-camera images. To ensure cross-modal compatibility and training stability, we propose Unified Latent Anchoring (ULA), which explicitly aligns the latent distributions of the two modalities. The aligned features are fused and processed by a diffusion transformer that jointly models their geometric correspondence and temporal evolution. Additionally, structured scene layout information is projected per modality as a conditioning signal to guide the synthesis. Extensive experiments demonstrate that UniDriveDreamer outperforms previous state-of-the-art methods in both video and LiDAR generation, while also yielding measurable improvements in downstream driving tasks.

We introduce a controlled subspace intervention framework to investigate how self-supervised Vision Transformers (ViTs) encode dense geometric information. While linear probing is widely used to assess geometric representations, it treats features as a black box, failing to disentangle the underlying topology. To address this issue, we decompose the weights of converged linear probes to isolate the low-rank subspaces containing explicit geometric signals using Singular Value Decomposition (SVD). Our perspective yields three key insights: (1) Pre-training objectives determine how features are encoded. DINOv2 aligns spatial features for efficient linear extraction, while Masked Autoencoders (MAE) tend to disperse these signals, requiring a broader spatial context. (2) Explicit geometric representations are highly compressible, suggesting dense predictive heads could potentially be constrained to low-rank subspaces with minimal performance loss. (3) The layer-wise task affinity suggests that geometric precision peaks at intermediate layers before yielding to semantic abstraction in the final layers. By connecting internal encoding mechanics with downstream performance, these findings provide a basis for effective feature selection and lightweight decoder design. The source code is available at https://github.com/Zhou-Weichen/Geosubprobe.