ECCV 2026 论文列表

Adapting 2D foundation models for 3D spatial understanding faces a critical dilemma. Fine-tuning 2D models to learn 3D geometry causes the catastrophic forgetting of native 2D knowledge. Conversely, freezing the 2D model restricts 3D spatial perception. To resolve this issue, we propose DINOv3D, a joint optimization framework built upon DINOv3 that employs a homologous teacher-student architecture to establish a regularized integration between the 2D and 3D understanding. To preserve original 2D priors, DINOv3D distills 2D knowledge into a 3D Gaussian regularization field, which aligns the rendered features with the reference visual features from a frozen teacher model. Meanwhile, a student model processes long-context multi-view inputs through parameter-efficient fine-tuning. This step injects 3D spatial consistency priors to 2D model while using the regularization field to mitigate the forgetting of 2D knowledge. To enrich the hierarchical representation of 3D spatial understanding, the student model predicts additional semantic and instance Gaussian features. We then apply a ray-depth-semantic alignment mechanism, which uses 3D depth-ray priors to enforce multiview consistency across 2D semantic rendering. Extensive experiments demonstrate that, despite updating only 10% of the parameters of a 1B-parameter foundation model VGGT, DINOv3D achieves state-of-theart results in comprehensive 3D scene understanding across challenging novel view synthesis, depth estimation, and open-vocabulary semantic segmentation. Furthermore, DINOv3D consistently enhances the generalizability of the DINOv3 backbone on 2D linear probing benchmarks.

As Vision-Language Models (VLMs) become increasingly so-phisticated and widely used, it becomes more and more crucial to un-derstand their decision-making process. Traditional explainability meth-ods, designed for classification tasks, struggle with modern autoregressiveVLMs due to their complex token-by-token generation process and in-tricate interactions between visual and textual modalities. We presentDEX-AR (Dynamic Explainability for AutoRegressive models), a novelexplainability method designed to address these challenges by generat-ing both per-token and sequence-level 2D heatmaps highlighting imageregions crucial for the model’s textual responses. The proposed methodoffers to interpret autoregressive VLMs, accounting for the varying im-portance of layers and generated tokens—by computing layer-wise gradi-ents with respect to attention maps during the token-by-token generationprocess. DEX-AR introduces two key innovations: a dynamic head filter-ing mechanism that identifies attention heads focused on visual informa-tion, and a sequence-level filtering approach that aggregates per-tokenexplanations while distinguishing between visually-grounded and purelylinguistic tokens. Our evaluation on ImageNet, VQAv2, and PascalVOC,shows a consistent improvement in both perturbation-based metrics, us-ing a novel normalized perplexity measure, as well as segmentation-basedmetrics. Code is available at https://walidbousselham.com/DEX-AR

Fine-tuning Multimodal Large Language Models (MLLMs)on specialized tasks often leads to catastrophic forgetting of their gen-eral capabilities. Existing model merging methods to combat this areoften heuristic or use sub-optimal objectives. We propose Curvature-Guided Mixing (CGM), a theoretically grounded framework that mergespre-trained and fine-tuned models. CGM formulates a joint optimiza-tion objective and uses a second-order (Hessian) approximation of theloss landscapes to analytically derive an optimal, closed-form “soft mix-ing” ratio. This ratio intelligently blends parameters based on their rel-ative task-specific curvatures. We also introduce CGM† , a robust “hardmixing” variant that performs sparse parameter selection guided by anovel, curvature-aware score. Experiments on LLaVA-1.5 and Qwen-2.5VL across multiple downstream tasks show that CGM and CGM†consistently improve the trade-off between task specialization and gen-eral knowledge retention over existing methods. Code is available atgithub.com/zzsyjl/CGM-ECCV-2026.

Cross-scene hyperspectral image (HSI) classification facespersistent challenges due to domain shifts in spectral signatures, spatialstructures, and semantic compositions. Existing domain generalizationmethods typically align either features or predictions, but they overlookhow local context (e.g., boundary mixing and class co-occurrence pat-terns) changes across scenes. As a result, one-sided alignment often yieldspartial adaptation: feature-invariant models may still produce unstableoutputs, while output-aligned models can rely on domain-specific short-cuts. We propose Context-Aware Joint Alignment (CAJA), a pixel-faithful framework that preserves the single-label objective while inject-ing explicit context into training. CAJA combines Context-Aware Su-pervision (CAS) to improve boundary-aware supervision using neigh-borhood context and homogeneity, and Context-Conditioned JointAlignment (CCJA) to enforce context-conditioned alignment in bothfeature and prediction spaces across domains. By coupling supervisionand joint alignment under shared context, CAJA mitigates the partial-alignment limitation of one-sided methods. Extensive experiments onthree cross-scene HSI benchmarks show that CAJA consistently improvesgeneralization, especially at class boundaries where context shift is mostsevere, establishing a simple, principled, and deployment-friendly solu-tion for cross-scene HSI domain generalization.

Appearance-based gaze estimation typically relies on facenormalization to reduce appearance variability, but this requires costlyand error-prone landmark detection and head pose estimation. Whilecrop-based alternatives have been explored, their geometric propertiesand performance trade-offs relative to normalization remain underex-plored. In this work, we provide the first systematic comparison of nor-malization versus crop-based gaze estimation. To enable fair compari-son, we formalize the crop-based approach through Constrained Rota-tion Optimization (CROp), making its geometric transformation explicitand comparable to normalization. We further adopt multi-task learningto recover head pose information lost in cropping. Through extensiveexperiments across various datasets, head pose distributions, and pre-processing conditions, we identify the conditions under which each ap-proach excels. CROp shows advantages under extreme poses and noisydetection, while normalization benefits from landmark-based refinementin moderate conditions. Our analysis provides practical guidelines forchoosing preprocessing strategies in real-world gaze estimation systems.

Streaming feed-forward 3D reconstruction enables real-timejoint estimation of scene geometry and camera poses from RGB images.However, without explicit dynamic reasoning, streaming models can beaffected by moving objects, causing artifacts and drift. In this work, wepropose RayMap3R, a training-free streaming framework for dynamicscene reconstruction. We observe that RayMap-based predictions exhibita static-scene bias, providing an internal cue for dynamic identification.Based on this observation, we construct a dual-branch inference schemethat identifies dynamic regions by contrasting RayMap and image pre-dictions, suppressing their interference during memory updates. We fur-ther introduce reset metric alignment and state-aware smoothing to pre-serve metric consistency and stabilize predicted trajectories. Our methodachieves state-of-the-art performance among streaming approaches ondynamic scene reconstruction across multiple benchmarks. The projectpage and code are available at https://raymap3r.github.io/.

Current instruction-based image retrieval systems are powerful but limited to single-turn interactions, failing to capture the iterative nature of complex, real-world visual searches. To overcome this limitation, we introduce Contextual Composed Image Retrieval (CoCo-IR), a novel task that enables users to progressively refine search results through interactions. We address this new task by proposing a new model based on a Large Multimodal Model (LMM) that functions as a context-aware reasoner for CoCo-IR. Our model interprets the entire interaction history to generate Transformable Image Embeddings (TIE) that evolve across turns. To fuel the model training without expensive human annotations, we develop a fully autonomous, scalable data engine that leverages LMMs to generate high-quality contextual retrieval data, and uses model-guided verification to mine challenging hard negatives. Extensive experiments demonstrate that our approach establishes new state-of-the-art performance: We achieve 39.4 mAP@5 on the challenging single-turn benchmark CIRCO; furthermore, on our new CoCo-IR benchmark, our model maintains robust performance with 44.1 R@1 on 4-turn dialogues, dramatically outperforming existing methods (28.2 4-turn R@1) that fail to handle multi-turn context. Project page: https://CoCo-IR.github.io.

We present BWAFDA, a dual-branch no-reference image quality assessment model that replaces uniform averaging with detail-aware, block-wise fusion. The context encoder branch produces multi-scale block tokens, and a lightweight detail branch extracts high-frequency cues. These streams are combined late. First, block tokens are aggregated via block-wise attention across scales, guided by a soft multi-scale prior that stabilizes cross-scale interactions and preserves localized artifacts. The resulting descriptors are then fused per scale using learned weights. Subsequently, the detail cues are injected to refine distortion-sensitive evidence. Finally, a class-token-guided head performs “where-to-trust-what” selection over blocks to produce the quality score, without the need for region labels. Comprehensive evaluations on seven public NR-IQA benchmarks show state-of-the-art (SOTA) performance. Across multiple controlled ablations, we observe consistent drops in correlation compared to the full model, indicating that region-aware masking, contextual features, and multi-scale aggregation contribute jointly rather than redundantly. Overall, detail-aware, block-wise fusion delivers SOTA accuracy while preserving data efficiency and transferability by aligning localized artifact cues with global context.

In human visual perception, uppercase lettering serves as anatural salience cue that captures attention within lowercase text. Inthis paper, we present a systematic empirical characterization study re-vealing that Large Language Models (LLMs) exhibit an analogous prop-erty: letter casing modulates internal attention allocation. Through anal-ysis across 13 models—nine LLMs and four Vision-Language Models(VLMs)—with diverse tokenization schemes, we show that formattingtarget information in alternating or uppercase against a lowercase con-text concentrates attention on those textual spans. In text this e!ectis universal, holding across every evaluated non-reasoning model. Weframe it as a previously under-explored latent property of pretrainedtransformers rather than a prescriptive method. Our investigation re-veals a central attention–performance divergence: while this “casing ef-fect” robustly shifts attention, its impact on downstream accuracy isnon-trivial—increased concentration does not inherently improve taskaccuracy and, in high-entropy contexts like alternating case, can degradeit. We further identify a boundary condition: the deliberative “thinking”phase in reasoning models acts as a semantic bu!er that mitigates ty-pographic sensitivity in text. Extending the study to VLMs, we find thee!ect transfers partially: the same prompt-side casing reorganizes cross-modal attention along two coupled axes—predominantly a macroscopicdisengagement from the image toward the text prompt, and secondar-ily a concentration of the residual visual attention on the target region.By isolating casing as a zero-shot mechanism for attention steering thatrequires no model access or fine-tuning, we provide a new foundationalunderstanding of how pretraining internalizes typographic emphasis.

Recent advancements have explored agentic zero-shot 3D un-derstanding by reformulating it as video keyframe understanding withMultimodal Large Language Models (MLLMs). However, existing meth-ods face an intrinsic bottleneck due to the finite observation perspectivesinherent in videos and the implicit perception of 3D scenes. In this pa-per, we propose a collaborative multi-agent framework that assigns aPlanning Agent to handle high-level viewpoint planning and supplementnovel perspectives, and a Perception Agent to explicitly summarize the3D scene into a structured holistic cognitive map. Specifically, PlanningAgent first analyzes this cognitive map to determine query-relevant view-points and supplements missing critical perspectives to ensure compre-hensive observation. Subsequently, Perception Agent documents object-level attributes from these views by assigning consistent instance identi-fiers across viewpoints, thereby integrating fragmented observations intothe holistic cognitive map. In parallel, it provides feedback to filter outmismatched candidate objects and guide subsequent viewpoint planning.Through this closed-loop iterative process, two agents collaboratively fig-ure out candidates until Perception Agent determines that sufficient in-formation has been captured to complete the task. Extensive experimentsdemonstrate that our method achieves state-of-the-art performance on6 benchmarks, with improvements of 11.1% Acc@0.5 on ScanRefer, 14.6BLEU-1 on 3D-assisted dialog, and 2.1 EM on SQA3D. Project Page:https://zhangbo135.github.io/agentic-collaborative-cognition/

We present MeshFM, an efficient feedforward framework for extracting rich features from 3D inputs. Our method distills 2D features from visual foundation models into 3D. We train a feedforward network to directly predict 3D features without requiring optimization during inference. The approach utilizes a two-stage training strategy. First, we optimize a feature field in 3D using only 2D feature supervision. Second, we train a network to regress this feature field. The entire procedure requires no 3D annotation, instead relying on the powerful information in 2D foundation models. We demonstrate that our learned features can be immediately applied to downstream tasks, including part segmentation, dense correspondence, and mesh deformation. Extensive experiments show that MeshFM, trained solely with 2D supervision, performs on par with methods trained explicitly with 3D supervision, even without task-specific fine-tuning. Moreover, our model is trained to be robust to extreme rotations of the input objects. Project page: https://threedle.github.io/MeshFM/

Multi-tool agents for instruction-guided image editing re-main brittle under open-loop pipelines, where early routing errors propa-gate unchecked through the entire workflow. We propose VLTR (Vision-Language Tool Reasoning), a training-free framework that reformulatesediting as closed-loop tool reasoning over a directed acyclic graph (DAG)of atomic primitives. A generative decomposer first converts each natural-language instruction into executable primitives. A Bayes-UCB routerthen adaptively selects tools by combining contextual statistics withsemantic priors, balancing exploration and exploitation online. Aftereach execution, a heteroscedastic verifier produces a calibrated quality–uncertainty signal (q, σ 2 ) through inverse-variance fusion of three com-plementary assessment tiers. A verifier-guided scheduler uses this jointsignal to retry, reroute, or replan only the failed subgraph while preserv-ing validated ancestors, turning full-pipeline restarts into efficient localrepair. On PIE-Bench++, VLTR achieves the best overall VLM rankingamong all evaluated baselines. On MagicBrush, it delivers a 4.8× run-time speedup over the strongest multi-tool competitor with no qualityloss. Pilot evaluations on GEdit-Bench and RISEBench further confirmgeneralization to challenging edits.

We present VIGS-SLAM, a visual-inertial 3D Gaussian Splat-ting SLAM system that achieves robust real-time tracking and high-fidelity reconstruction. Although recent 3DGS-based SLAM methodsachieve dense and photorealistic mapping, their purely visual design de-grades under challenging conditions such as motion blur, low texture,and exposure variations. Our method tightly couples visual and inertialcues within a unified optimization framework, jointly optimizing cam-era poses, depths, and IMU states. It features robust IMU initialization,time-varying bias modeling, and loop closure with consistent Gaussianupdates. Experiments on five challenging datasets demonstrate our su-periority over state-of-the-art methods. The code will be made public.

Transformer-based video diffusion models rely on 3D attention over spatial and temporal tokens, which incurs quadratic time and memory complexity and makes end-to-end training for ultra-highresolution videos prohibitively expensive. To overcome this bottleneck, we propose a pure image adaptation framework that upgrades a video Diffusion Transformer pre-trained at its native scale to synthesize higherresolution videos. Unfortunately, naively fine-tuning with high-resolution images alone often introduces noticeable noise due to the image–video modality gap. To address this, we decouple the learning objective to separately handle modality alignment and spatial extrapolation. At the core of our approach is Relay LoRA, a two-stage adaptation strategy. In the first stage, the video diffusion model is adapted to the image domain using low-resolution images to bridge the modality gap. In the second stage, the model is further adapted with high-resolution images to acquire spatial extrapolation capability. During inference, only the high-resolution adaptation is retained to preserve the video generation modality while enabling high-resolution video synthesis. To enhance fine-grained detail synthesis, we further propose a High Frequency Awareness Training Objective, which explicitly encourages the model to recover high-frequency components from degraded latent representations via a dedicated reconstruction loss. Extensive experiments demonstrate that our method produces ultra-high-resolution videos with rich visual details without requiring any video training data, even outperforming previous state-ofthe-art models trained on high-resolution videos by 0.8 on the VBench benchmark. Code is available here.

Human motion modeling must support generation from text,music, and speech, comprehension through captioning, temporal com-pletion, and multi-person coordination—yet existing methods typicallytackle each scenario with a separate, task-specific system. We arguethat discrete tokenization plus autoregressive (AR) modeling possessesunique compositional advantages for building a unified motion model:all modalities share a single vocabulary and training objective, tasks aredefined by token arrangement rather than architectural changes, andthe same model naturally supports both generation and understand-ing by simply reordering the sequence. However, realizing this potentialhas been blocked by two obstacles. First, current motion tokenizers useroot-relative, non-causal, frame-level encodings that discard world-spaceinformation, break consistency with forward-only AR decoding, and scalepoorly—confining existing discrete AR models to text–motion pairs insingle-human settings. Second, multi-modal motion data with paired textand audio annotations remains scarce. We address both obstacles. Weintroduce VerMoVQ, a causal spatio-temporal VQ-VAE that factorizeseach frame into low-dimensional body tokens while preserving world-space root trajectories, naturally enabling multi-person tokenization viaper-agent concatenation and arbitrary-length encoding through stricttemporal causality. We curate MotionHub, a \sim 380K-clip corpus in unifiedSMPL representation with large-scale multi-modal annotations spanningnine motion tasks. Built on these foundations, we train VersatileMo-tion, a single decoder-only AR Transformer that covers text\leftrightarrow motion,music\leftrightarrow dance, speech

Despite the recent success of multimodal discrete diffusionlanguage models (multimodal dLLMs) in generating text through iter-ative demasking with bidirectional attention, object hallucination re-mains a critical challenge. Existing hallucination mitigation methods aredesigned for autoregressive generation and encounter fundamental mis-matches when applied to multimodal dLLMs, either doubling inferencecost through contrastive decoding or suffering from ambiguous step andposition alignment under bidirectional context. To address this, we pro-pose Unmasking-Time Visual Calibration (UVC), a lightweight, training-free framework that mitigates hallucinations natively within the demask-ing process. UVC extracts multi-granularity contrastive activation shiftvectors offline by comparing model responses to clean and visually de-graded inputs, identifies the most visually informative attention headsvia per-head AUC scoring, and injects the pre-computed calibration sig-nals exclusively at still-masked positions during inference, requiring noadditional forward passes. Extensive experiments on POPE and MMEwith two representative multimodal dLLMs demonstrate that UVC con-sistently reduces hallucinations by a substantial margin while introducingless than one percent inference overhead. Additional evaluation on open-ended captioning with the CHAIR metric further confirms that UVC sub-stantially lowers hallucination rates without affecting generation length,and generalizes effectively across benchmarks without retraining. Codeis available at https://github.com/THU-BPM/UVC.

Accurate motion prediction of surrounding agents and safemotion planning are two closely coupled key tasks for social robot nav-igation in crowded environments. Deploying these systems on resource-constrained edge devices necessitates compact, unified models that canperform both tasks simultaneously. However, within these compact sharedencoders, recent unified models often overlook severe representationalconflicts that arise from the distinct objectives of predicting neighbor be-haviors versus ego-centric safety planning. To address this issue, we firstidentify the Skill Conflict—a phenomenon where overlapping parameterassignments cause distinct tasks to compete for the same weights, pre-venting the model from fully specializing in individual skills. To resolvethis, we propose a novel model-merging-based framework, Disjoint Pa-rameter Training (DPT). DPT mitigates performance degradationcaused by Skill Conflict through distributed parameter learning, whichseparates the key parameter regions of each task while preserving theircore capabilities prior to merging. In addition, we observe that sparsemerging, which selectively integrates only the most influential param-eters for each task rather than combining all task-specific parameters,yields optimal performance by preventing interference among adjacentfeatures and concentrating representational capacity. DPT can be ap-plied in parallel with a variety of merging methods. Evaluated on stan-dard crowd navigation benchmarks (JRDB and JTA), our frameworkdemonstrates superior performance, validating its versatility and effec-tiveness for safe, resource-efficient robot navigation. The project page isavailable at: https://dpt2026.github.io/

Style transfer must match a target style while preservingcontent semantics. DiT-based diffusion models often suffer from con-tent–style entanglement, leading to reference-content leakage and unsta-ble generation. We present UniCSG, a unified framework for content-constrained, style-driven generation in both text-guided and reference-guided settings. UniCSG employs staged training: (i) a latent-space se-mantic disentanglement stage that combines low-frequency preprocess-ing with conditioning corruption to encourage content–style separation,and (ii) a latent-space frequency-aware detail reconstruction stage thatrefines details via multi-scale frequency supervision. We further incorpo-rate pixel-space reward learning to align latent objectives with percep-tual quality after decoding. Experiments demonstrate improved contentfaithfulness, style alignment, and robustness in both settings.

Pretraining 3D encoders through alignment with ContrastiveLanguage–Image Pre-training (CLIP) has emerged as a promising direc-tion to learn generalizable representations for 3D scene understanding. Inthis paper, we propose UniScene3D, a transformer-based framework thatlearns unified 3D scene representations from multi-view RGB–Pointmapby leveraging the priors of a pretrained 2D foundation model. For robustRGB-Pointmap representation learning, we introduce novel cross-viewgeometric alignment and grounded view alignment to enforce geome-try and semantic consistency across views. Extensive low-shot and task-specific fine-tuning across viewpoint grounding, scene retrieval, sceneclassification, and 3D visual question answering achieves state-of-the-artperformance. These results establish UniScene3D as an effective frame-work for unified 3D scene understanding.