ECCV 2026 Papers

The rapid advancement of diffusion models has enabled thegeneration of high-fidelity images from textual prompts, yet achievingprecise, disentangled control over specific attributes remains a significantchallenge. A fundamental limitation arises because visual differences be-tween images are often far more descriptive and nuanced than what canbe captured through human-crafted text descriptions, which frequentlyfail to convey fine-grained semantic details. To address this, we intro-duce ViDiT (Visual Direction Transfer for Diffusion), a frameworkthat expands the editing vocabulary by capturing latent semantics di-rectly from image-edit pairs. ViDiT learns the underlying transformationby optimizing a single, global, and continuous editing direction from asmall set of “before-and-after” examples. This optimization process trans-fers visual changes into the diffusion model’s conditioning space, allowingfor detailed edits that text alone cannot easily describe. ViDiT operateson a “Learn Once” principle, which completely eliminates the need formodel fine-tuning or expensive per-image optimization during inference.Once learned, these continuous directions enable “Edit Anywhere” capa-bilities, allowing users to apply highly disentangled manipulations, suchas changes in facial features, animal attributes, or artistic styles, to anyimage in a zero-shot manner with granular control over the edit inten-sity. Quantitative and qualitative evaluations demonstrate that ViDiToutperforms existing text-based editing methods in maintaining inputfaithfulness while achieving precise, scalable attribute control.

We tackle fine-grained indoor localization from natural lan-guage: given a free-form description of one’s surroundings, estimate theobserver’s 2D position and heading within a known 3D environment.Language queries are lightweight, privacy-preserving, and need no cam-era – yet prior work stops at coarse scene retrieval and cannot resolve anintra-scene pose. We close this gap with LangLoc, a three-stage pipelinethat (i) retrieves the correct scene via a dual-branch GATv2 encoderwith CLIP semantic features, surpassing the previous best by 8 per-centage points in Top-1 recall; (ii) estimates position and heading byscoring a dense floor grid through ray-cast object visibility, reaching amedian error of 0.95 m; and (iii) resolves residual ambiguity through aBayesian dialog module that asks targeted yes/no questions and updatesa pose posterior until the location is pinpointed. To support this task wecontribute a benchmark of 13,000+ pose-indexed natural-language de-scriptions over 1,300+ indoor 3D scans. Code and data will be released.Project page: https://rzninvo.github.io/Lang-Loc/.

Vision-language models (VLMs) achieve remarkable multi-modal comprehension, yet their massive parameter counts impede de-ployment on resource-limited hardware. Traditional pruning techniquesrely on manual heuristics that frequently disrupt delicate cross-modalsynergies, leading to substantial performance declines. To overcome this,we introduce KATANA (Knowledge-Aligned Topology-Aware NeuralAgents), a reinforcement learning framework that autonomously discoversexecutable pruning algorithms. Guided by a synthesized multi-objectivereward signal and safely evaluated within an isolated sandbox, an LLM-driven agent iteratively evolves innovative compression strategies. Ourflagship discovered algorithm, KIRI (Kernel-Integrated ReconstructionIterator), utilizes a cubic sparsity scheduler alongside a novel Dual-NormActivation (DNA) importance metric that dynamically fuses weight mag-nitudes, structural regularizations, and data-dependent activation profiles.Extensive evaluations across four diverse VLM architectures (LLaVA-1.5,BLIP-2, Qwen2.5, and Llama-3.2-Vision) on MSCOCO, Flickr30k, andNoCaps benchmarks demonstrate KATANA’s superiority. At an aggres-sive 70% sparsity, KIRI delivers a measured 2.8× inference speedup whileconsistently outperforming 16 state-of-the-art baselines, including recentadaptive and hybrid methods such as SCOPE and GSOP. KATANA thusintroduces an automated and extensible framework for VLM compressionthat preserves vision-language coherence for practical deployment.

Generating realistic human avatars in complex motions—suchas clothing dynamics—requires modeling of global and local deformationswhich remains challenging in monocular settings. We address this prob-lem by leveraging neural Jacobian fields (NJFs) for representing semi-rigid deformations. We train self-supervised neural networks for predict-ing Jacobian matrices that give the pose-dependent deformations, bysolving a Poisson equation. However, monocular input presents severaldifficulties such as self-occluded regions and invisible surfaces. To addressthese issues, we introduce three key components: a constrained Poissonsolver, signed distance-based Jacobian regularization, and a deformation-guided residual flow loss, which together suppress boundary artifacts, re-cover frequently occluded regions such as armpits and thighs, and enforcetemporal consistency during motion. Experiments on benchmark and in-the-wild videos demonstrate that our method generates temporally sta-ble and geometrically coherent avatars, outperforming state-of-the-artapproaches.

Modern object detectors systematically fail on concealed, camouflaged, or strategically hidden objects, not due to insufficient capacity, but because they reason solely from visual appearance with no mechanism to consider where objects might be hidden. We propose IACD (Iterative Adversarial Collaborative Detection), a dual-perspective framework that pairs a frozen pretrained YOLOv11m (Seeker) with a lightweight trainable module (Hider) that reasons inversely: given the environment, where would a target be hidden? The Hider analyzes environmental concealment suitability across texture, edge, and semantic dimensions, then identifies the Seeker’s blind spots by contrasting suitability against detection coverage. The resulting blind spot maps residually amplify feature responses via attention gates before a second detection pass. The two branches interact across multiple rounds, with the Hider’s blind spot estimates informing the attention gates that guide the second detection pass. Crucially, the Hider is trained under weak supervision derived solely from the detector’s own false negatives, requiring no manual concealment annotations. We frame this as detector blind-region recovery—recovering a frozen detector’s false negatives—rather than as standard camouflaged object detection. Experiments on the Poppy illegal crop detection dataset and the VisDrone aerial surveillance benchmark demonstrate consistent improvements over the strong YOLOv11m baseline, while introducing only ∼9M trainable parameters atop the frozen detector.

Hallucinations in vision–language models (VLMs) are com-monly treated as semantic errors, yet they often arise from partial or am-biguous visual evidence. Prior work mainly focuses on detecting or sup-pressing hallucinations at generation time, leaving the subsequent reason-ing stage largely unexplored. In this work, we study Post-HallucinationReasoning (PHR), the stage in which hallucinated semantics enter themodel’s inference context and influence downstream predictions. To sys-tematically investigate PHR, we introduce the HIVE (HallucinationInference and Verification Engine), an evaluation infrastructure that en-ables controlled comparisons between faithful and hallucinated captions.Across nine tasks and nine models, we observe structured modality-dependent patterns: hallucinated captions often improve accuracy onvision–language tasks, while text-only tasks exhibit limited or unstableeffects. Further analyses show that hallucinated cues broaden semanticcoverage and reshape reasoning dynamics while preserving stable infer-ence. These findings highlight that hallucinated semantics may influencedownstream reasoning once they enter the model’s inference context. Un-derstanding this post-hallucination stage is important for improving thereliability and interpretability of multimodal reasoning systems.

In this paper, we present a novel geometry-aware style trans-fer framework for 3D Gaussian splatting (3DGS) that simultaneouslytransfers appearance attributes and geometric structures. Unlike priorworks that primarily focus on color-based stylization and often over-look structural adaptation, our method explicitly incorporates geometryadaptation through a decoupled optimization scheme that alternatelyupdates color and geometry parameters. This strategy alleviates poten-tial interference between color and geometry updates, leading to stableand consistent scene-level geometry transformation. The decoupled opti-mization is enabled by the proposed geometry-aware contrastive featurematching (GCFM). GCFM integrates RGB, depth, and edge cues intoa contrastive objective and is employed in both optimization phases toeffectively transfer structural characteristics from style images to Gaus-sian primitives. Extensive experiments show that our approach achievessuperior performance in both qualitative fidelity and quantitative met-rics, significantly outperforming existing 3DGS-based stylization meth-ods. Our code is available at https://github.com/oweixx/gast.

Monocular depth foundation models have demonstrated re-markable generalization capabilities across diverse environments. How-ever, they continue to struggle with metric depth estimation in diverseenvironments. This limitation stems from the inherent scale ambiguity ofsingle-view inference, leading to misaligned scale predictions even whenthe relative geometry is accurate. Conversely, recent multi-view founda-tion models leverage cross-view cues to learn robust scene-level geometryand consistent scale. Yet, these benefits typically vanish during single-image inference, as the absence of explicit geometric constraints causesperformance to degrade. To bridge this gap, we propose a novel frame-work that transfers the scale-aware geometric priors of multi-view modelsinto monocular depth foundation models. Specifically, we introduce anEpipolar Distillation (EpiDistill), an approach utilizing Rectified StereoTokens, which enables the single-view prediction model to retain epipolarattention patterns and maintain geometric consistency without requiringmulti-view inputs at inference. Experimental results demonstrate thatour method significantly improves zero-shot metric depth estimation,particularly on challenging datasets like ETH3D and DIODE where scalealignment is critical. Furthermore, our approach is model-agnostic, con-sistently boosting the performance of state-of-the-art ViT-based models,including UniDepthV2 and DepthPro. Project Link

Embodied Vision-Language Models (VLMs) have demon-strated impressive performance and generalization in robotics, partic-ularly within Vision-Language-Action frameworks. However, a signifi-cant gap remains between the high-level semantic focus of standardtext-guided pre-training paradigms and the low-level spatial and phys-ical knowledge critical for execution in embodied environments. In thispaper, we introduce GEM, a Generative-supervised Embodied vision-language Model designed to bridge this divide. We propose integratinga depth map generation task directly into the VLM pre-training phase.By training this generative objective jointly with the main model, weobserve substantial improvements in embodied intelligence, significantlyenhancing both semantic understanding and physical operation capa-bilities. To support this paradigm, we curate and release GEM-4M, acomprehensive large-scale dataset featuring a mixture of grounding, rea-soning, and planning data paired with high-quality depth supervision.Extensive experiments demonstrate that GEM achieves state-of-the-artresults across diverse embodied benchmarks. Furthermore, our deployedaction model, GEM-VLA, exhibits vastly superior task execution abilitiesin both simulation environments and real-world evaluations. Code, mod-els, and datasets are available at https://zhaorw02.github.io/GEM/.

Open-vocabulary semantic segmentation (OVSS) makes seg-mentation driven by text descriptions, enabling generalization to novelclasses. However, fine-grained OVSS in remote sensing is challenged byvisual similarity and semantic ambiguity, with existing methods lackingexplicit fine-grained visual-linguistic alignment and relying on a singleforward pass. To address these issues, we propose SANO3, a training-free framework that leverages refined textual and spatial guidance toboost the performance of SAM 3. Leveraging the visual self-similaritypriors of vision foundation models, we extract discriminative visual prim-itives and align them with text through visual-conditioned relation-awarealignment strategy. Next, we enhance text representations with hybridprototype injection for deep visual-linguistic fusion. To refine bound-aries, we treat spatial prompts generation as an iterative optimizationprocess, gradually selecting the most informative points via a greedy en-tropy reduction strategy. Further, we introduce FG-OVSSRS Bench, thefirst fine-grained remote sensing OVSS benchmark, encompassing 8 di-verse datasets across different sensors and domains. Evaluated on thesedatasets, SANO3 outperforms existing approaches and achieves state-of-the-art performance.

Foundation Models (FMs) are increasingly integrated into remote sensing (RS)pipelines for applications such as environmental monitoring, disaster assessment, and land-usemapping. These models include unimodal vision encoders trained in a single data modalityand multimodal architectures trained in multiple sensor modalities, such as synthetic aper-ture radar (SAR), multispectral, and hyperspectral imagery, or jointly in image-text pairs invision-language settings. FMs are adapted to diverse perception tasks, such as semantic seg-mentation, image classification, change detection, and visual question answering, dependingon their pretraining objectives and architectural design. However, selecting the most suit-able remote sensing foundation model (RSFM) for a specific task remains challenging dueto scattered documentation, heterogeneous formats, and complex deployment constraints.To address this, we first introduce the RSFM Database (RS-FMD), the first structuredand schema-guided resource covering over 160 RSFMs trained on various data modalities,spanning different spatial, spectral, and temporal resolutions, considering different learn-ing paradigms. Built upon RS-FMD, we further present Remsa (Remote-sensing ModelSelection Agent), a constraint-aware agent that enables automated RSFM selection from nat-ural language queries. Remsa combines structured FM metadata grounding with task-awareorchestration for retrieval, clarification, ranking, and explanation. In detail, it interprets userinput, clarifies missing constraints, ranks models via in-context learning, and provides trans-parent justifications. Our system supports various RS tasks and data modalities, enablingpersonalized, reproducible, and efficient FM selection. To evaluate Remsa, we constructa benchmark of 100 expert-verified RS query scenarios. Each query is evaluated across 4systems and 3 LLM backbones, with the top-3 selected models assessed by domain expertsusing a fixed suitability rubric. This results in 3,000 expert-scored task–system–model con-figurations under our rubric-based expert evaluation protocol. Remsa outperforms multiplebaselines, including naive agent-based method, dense retrieval, and unstructured retrievalaugmented generation approaches, showing its practical utility in real decision-making ap-plications. Remsa operates entirely on publicly available metadata of open source RSFMs,without accessing private or sensitive data. Our code and data are publicly available at:https://github.com/be-chen/REMSA.

Psychiatric diagnosis is moving from subjective clinical ob-servation toward objective multimodal assessment using audio, visual,and textual signals. Federated learning offers a natural framework formulti-center collaboration without centralizing sensitive patient data,but psychiatric data introduce a form of non-IID heterogeneity thatis not captured by standard label or domain skew: patients with thesame diagnosis can exhibit highly polymorphic manifestations, form-ing multi-peak within-class distributions. Existing prototype-based FLmethods usually aggregate client or class features into coarse centroids,which can collapse distinct symptom modes into semantically ambigu-ous prototypes. We propose FedMental, a topology-aware federatedprototype framework for multi-center multimodal psychiatry. FedMen-tal first preserves subsampled sample-level embeddings before prema-ture client-mean aggregation, then applies Topology-Aware Proto-type Refinement (TAPR) on the server to discover class-view la-tent modes through parameter-free clustering. On the client side, Intra-View Mode Alignment (IVMA) softly anchors local representationsto matched mode prototypes, while Cross-View Semantic Contrast(CVSC) promotes view-invariant semantics across modalities and cen-ters. Experiments on the real-world Psycare psychiatric benchmark andtwo standard heterogeneous FL benchmarks show that FedMental consis-tently improves over strong FL baselines, achieving +8.83% over FedAvgon Psycare, +11.42% on Office-Caltech, and +4.41% on Digits.

Existing underwater object tracking (UOT) methods can bebroadly divided into two categories: frame-level trackers and video-leveltrackers. Frame-level trackers process each frame independently, usingenhancement or adaptation techniques to handle underwater distortions.However, this often results in inconsistent feature styles and weakenedtemporal correlations. Video-level trackers attempt to improve temporalconsistency by modeling dependencies across frames through autoregres-sive mechanisms. Despite this, they still struggle with persistent featuredegradation and tracking drift caused by challenging underwater condi-tions. To overcome these limitations, we propose FeatTracker, a feature-level tracking framework that enforces both short- and long-range tem-poral feature consistency, effectively preserving semantic integrity andensuring temporal stability for robust underwater object tracking. Theframework integrates two key innovations: the Short-Range TemporalFeature Consistency module, which combines diffusion-like feature en-hancement with a dynamic memory pool to mitigate degradation andpreserve local temporal coherence; and the Long-Range Temporal Fea-ture Consistency module, which leverages wavelet decomposition to sepa-rate stable structural components from transient details within historicaltokens, effectively preventing the loss of trajectory details. Extensive ex-periments demonstrate that FeatTracker achieves state-of-the-art perfor-mance across four UOT benchmarks. Our code and models are availableat https://github.com/fishgfish/FeatTracker.

Recent multimodal large language models (MLLMs) havesubstantially advanced video understanding, yet long-form video QAremains challenging under fixed input token budgets, where uniform sam-pling can be inefficient for evidence localization. We propose ReQuest, an uncertainty-driven, question-adaptive keyframe selection pipelinethat aligns question intent with relevant video content through selec-tive computation. ReQuest integrates (i) a lightweight question-awareselector distilled from MLLM-generated supervision, (ii) Re-thinkingRouting that triggers additional inference only when the model is uncer-tain with a length-adaptive criterion, and (iii) uncertainty-guided adap-tive non-maximum suppression that selects temporally diverse frameswhile adjusting spacing based on question difficulty. As a plug-and-play method, ReQuest improves long-video QA without modifyingor fine-tuning the underlying MLLM. Experiments on Video-MME,MLVU, and LongVideoBench demonstrate consistent accuracy gainswith competitive computational cost, with particularly strong improve-ments in medium and long video regimes. The code is available athttps://geppa.github.io/ReQuest

Unified Multimodal Models (UMMs) have emerged as apromising paradigm that integrates multimodal understanding and gen-eration within a unified modeling framework. However, current gener-ative training paradigms suffer from inherent limitations. We presentSemantically-Grounded Supervision (SeGroS), a fine-tuning frameworkdesigned to resolve the granularity mismatch and supervisory redundancyin UMMs. At its core, we propose a novel visual grounding map toconstruct two complementary supervision signals. First, we formulatesemantic Visual Hints to compensate for the sparsity of text prompts.Second, we generate a semantically-grounded Corrupted Input to explic-itly enhance the supervision of masking-based UMMs by restricting theText-to-Image loss to core text-aligned regions. Extensive evaluations onGenEval, DPGBench, and CompBench demonstrate that SeGroS signif-icantly improves generation fidelity and cross-modal alignment acrossvarious UMM architectures.Project page: https://segros-project.github.io/

We introduce EgoSAT, the first comprehensive benchmarkfor egocentric video reasoning in streaming settings, designed to evaluatethe capabilities of modern vision–language models (VLMs). The bench-mark targets streaming interaction understanding, where video framesarrive sequentially and models must continuously interpret evolving vi-sual context. EgoSAT unifies several previously distinct tasks within asingle streaming framework. In this formulation, queries about completedevents correspond to retrospective reasoning, queries about ongoing ac-tivities require online understanding, and queries about future actionsinvolve prospective anticipation. This unified setting requires models toreason about the past, present, and future while operating under theconstraint that only previously observed frames are available. EgoSATcontains 1,997 unique videos spanning 165 hours of egocentric footageand around 4,800 high-quality question–answer pairs, carefully designedto probe reasoning across varying temporal contexts. Using this bench-mark, we evaluate a diverse set of both open-weight and closed-weightVLMs, providing a systematic assessment of their ability for streaminginteraction understanding. By distinguishing answerability and conduct-ing diagnostics on confidence of models, we find existing models not onlystruggle with prospective and retrospective modeling, but also exhibit se-vere mis-calibration: confidence often fails to track inherent answerabil-ity, leading to dangerous “confidently wrong” behaviors. Project page:https://leiyj23.github.io/EgoSAT/

The computational cost of training a vision-language model(VLM) can be reduced by sampling the training data. Previous work onefficient VLM pre-training has pointed to the importance of semanticdata balance, adjusting the distribution of topics in the data to improveVLM accuracy. However, existing efficient pre-training approaches maydisproportionately remove rare concepts from the training corpus. As aresult, long-tail concepts remain insufficiently represented in the trainingdata and are not effectively captured during training. In this work, weintroduce a dynamic cluster-based sampling approach (DynamiCS) thatdownsamples large clusters of data and upsamples small ones. We firstdemonstrate the advantage of our cluster-scaling approach, which main-tains the relative order of semantic clusters in the data and emphasizesthe long-tail. This approach contrasts with current work, which focusesonly on flattening the semantic distribution of the data. Then, we showthe importance of dynamic sampling, which applies sampling at eachepoch to improve cross-epoch data diversity and make upsampling prac-tical. Our experiments show that DynamiCS reduces the computationalcost of VLM training and provides a performance advantage for long-tail concepts. Code available at https://github.com/MingliangLiang3/DynamiCS.

Reward models are central to post-training and test-timeoptimization for visual generative models, yet existing approaches typi-cally rely on either large-scale pairwise preference annotations (hundredsof thousands to millions of labeled pairs) or heavily engineered multi-metric scoring pipelines, both of which are costly and labor-intensive. Wepresent DisRM, a discriminative reward modeling framework that avoidsthe need for pairwise annotations by training a lightweight discriminatorto distinguish model-generated outputs from a small set of preferred rep-resentative samples, called Preference Proxy Data (PPD). As the rewardmodel is repeatedly re-trained to distinguish outputs from the updatedgenerator from fixed PPD samples, DisRM naturally supports iterative,multi-round co-refinement with the generator. Across evaluations of im-age quality, safety alignment, and video generation, DisRM achieves com-petitive and often better performance than methods trained with up to1 million annotated preference pairs, while using only 500 unlabeled tar-get samples in image-quality setting, and generalizes across Best-of-Nselection, SFT, and DPO.

This paper presents DeRA, a novel 1D video tokenizer thatdecouples the spatial-temporal representation learning in video tokeniza-tion to achieve better training efficiency and performance. Specifically,DeRA maintains a compact 1D latent space while factorizing video en-coding into appearance and motion streams, which are aligned withpretrained vision foundation models to capture the spatial semanticsand temporal dynamics in videos separately. To address the gradientconflicts introduced by the heterogeneous supervision, we further pro-pose the Symmetric Alignment-Conflict Projection (SACP) module thatproactively reformulates gradients by suppressing the components alongconflicting directions. Extensive experiments demonstrate that DeRAoutperforms LARP, the previous state-of-the-art video tokenizer by 25%on UCF-101 in terms of rFVD. Moreover, using DeRA for autoregressivevideo generation, we also achieve new state-of-the-art results on bothUCF-101 class-conditional generation and K600 frame prediction.