ECCV 2026 Papers

Parsing images with high precision for matting and segmentation is sig-nificantly more challenging than conventional dense prediction, as it requires accurateestimation of fine-grained details. Existing methods either lack semantic awareness orproduce suboptimal predictions; even interactive matting approaches rely on manualverification and repetitive checking, making fully automatic matting still unattainable.In this work, we propose Kiroshi , an agentic perception system for high-accuracyimage parsing. We train an Action Model with iterative refinement and mine pairedtrajectories by sampling grid prompts from residual maps, where each step yields apositive and a negative transition under the same intermediate prediction based onquantitative quality gains. These within-context preference pairs form a reliable su-pervision signal for post-training to align the MLLM policy toward more effectivegrid decisions. We also contribute a new High-Fidelity Referring Matting and Seg-mentation (HiFiRefMS ) benchmark to evaluate the performance of different mod-els. Experimental results demonstrate that our method surpasses state-of-the-art ap-proaches both quantitatively and qualitatively, and extensive ablation studies furthervalidate the effectiveness and superiority of our agentic design. Project will be releasedat https://github.com/haipengzhou856/Kiroshi.

Image restoration is fundamentally constrained by the tradeoff between distortion and perception: minimizing pixel-wise error yields over-smoothed results, whereas optimizing for perceptual realism often introduces structural deviations. Recent approaches attempt to balance this tradeoff via posterior sampling or multi-stage generative pipelines, yet remain computationally expensive and architecturally complex. To overcome these limitations, we propose PCFlow (Perceptually Consistent Flow Matching), a unified framework that directly parameterizes a continuous transport from degraded observations to clean targets, jointly optimizing distortion and perceptual quality. While its latent consistency flow objective drives stable and efficient few-step inference, a Latent Consistency Perceptual Loss (LCPL) imposes semantic constraints directly on the guiding velocity field, steering the dynamics toward visually sharp data manifolds. Furthermore, recognizing the inherent conflict between structural and perceptual consistencies, we integrate a conflict-free gradient projection strategy to stabilize the multi-objective optimization landscape. Combined with lightweight, convolution-only backbone, PCFlow achieves competitive performance across diverse restoration tasks at a fraction of traditional computational costs.

Face anti-spoofing (FAS) is critical for secure deploymentof face recognition in high-stakes applications. However, existing multi-modal FAS methods still generalize poorly to unseen domains. We arguethat a key reason is that they directly align noisy multimodal featuresin which spoof-relevant cues are entangled with modality- and domain-specific variations, so both useful and nuisance patterns are forced tomatch across domains. Motivated by this, we design Multimodal Denoisingand Alignment (MMDA), a CLIP-based framework that (i) purifiesmultimodal features, (ii) softly guides them into a semantic space, and(iii) preserves the pre-trained geometry during task adaptation. To de-couple spoof cues from these variations before alignment, the Modality-Domain Joint Differential Attention (MD2A) module contrasts same-domain cross-sample features to down-weight patterns shared withineach domain/modality, which helps reduce domain- and modality-relatedbiases while better preserving discriminative spoof information. To avoiddestructive, over-rigid alignment, the Representation Space Soft (RS2)strategy aligns visual features to real/spoof subspaces spanned by mul-tiple prompts in the CLIP space, providing a soft semantic pull insteadof collapsing features onto a single text embedding. To prevent task-specific fine-tuning from destroying CLIP’s generalizable structure, theU-shaped Dual Space Adaptation (U-DSA) module introduces deepyet parameter-efficient adaptation and remaps adapted features backto shallower, more domain-invariant layers. Extensive experiments onWMCA, CeFA, PADISI, and SURF under complete-modality, missing-modality, and limited-source protocols show that MMDA consistentlysurpasses state-of-the-art methods, reducing average HTER by up to9.63% and improving AUC by up to 5.98% in cross-domain evaluation.

Vision-language models can produce confident answers on visually ambiguous inputs, resulting in biased predictions. Common entropybased methods, such as Semantic Entropy (SE), rely on output diversity. Yet our analysis shows that overconfident visual embeddings suppress output diversity under stochastic decoding, causing SE to underestimate uncertainty in such cases. Recent methods instead probe output diversity through input perturbations, including textual paraphrasing or joint text-image perturbations, and show improved performance. We study these approaches and reveals that the resulting variability is often dominated by textual changes rather than visual evidence, causing uncertainty estimates to reflect prompt sensitivity rather than visual ambiguity. We therefore propose Visual Semantic Entropy (VSE), which perturbs only the image to probe nearby visual variations while keeping the text query fixed. VSE measures uncertainty by clustering generated answers into semantic prototypes and computing the massweighted dispersion among them. Extensive evaluation across five modern vision-language models and five diverse VQA benchmarks demonstrates that VSE effectively captures visual ambiguity, establishing a new state-of-the-art for VLM uncertainty estimation. Code is available: https://github.com/tadeephuy/visual-semantic-entropy

Large-scale dyadic interactive audio-visual dialogue (DIAD)datasets provide fundamental data resources for developing humanoidinteractive virtual agents and digital humans. However, collecting suchdata is time-consuming, expensive, and ethically sensitive. To addressthis, we propose CHAT, a new dyadic interactive audio-visual dialoguegeneration (DIADG) framework that generates diverse, paired, and mu-tually responsive speech-face dialogue clips from a single textual prompt.CHAT unifies large language models and talking face models with inter-active audio and facial behaviour refinement modules, enabling the gen-eration of aligned dyadic dialogue clips with diverse contents and facialidentities. Experiments show that CHAT outperforms existing relatedmethods designed for similar tasks under both objective and subjectiveevaluations. Moreover, our synthesised CHAT-AVD-50k dataset servesas effective pre-training data for downstream interactive head genera-tion, consistently improving PerFRDiff and ReactDiff on REACT 2024.CHAT offers a scalable alternative to the costly and ethically sensitivecollection of real dyadic interaction data.

DNNs are highly susceptible to noisy labels, often leadingto unstable training and severe performance degradation. Graph-basedmethods have emerged as a promising solution, utilizing sample rela-tionships to refine labels through neighborhood aggregation. However,most existing approaches rely on dense connectivity (e.g., KNN graphs),which tends to aggregate noisy cues and cause oversmoothing. Whilesparse graphs offer a potential remedy, current sparse topologies oftensuffer from blind error propagation between samples, particularly forhard samples near class boundaries. To address these challenges, we pro-pose Maximum Spanning Tree (MST) Guided Confidence and SparseGraph Network (MST-GCSN). We introduce the MST as a robust sparsebackbone to filter out redundant local noise while preserving a globalmanifold skeleton. To rectify the indiscriminate mutual influence betweennodes, a Confidence-Gated Propagation (CGP) mechanism is designedto adaptively regulate information flow based on node reliability, ensur-ing that only high-confidence semantic signals are propagated. Buildingon this optimized structure, a Progressive Label Prediction (PLP) mod-ule integrates local information and global anchor-guided cues to itera-tively correct labels. Extensive experiments on synthetic and real-worldnoisy datasets demonstrate that MST-GCSN significantly enhances ro-bustness.

Supervised learning for image segmentation typically requires spatially aligned image and label sets. When images and labels originate from different sources, the pairing may be misaligned, which can significantly deteriorate the performance of the learned models. This is especially common in remote sensing, when aerial or satellite images are co-registered with labels from another source (e.g., OpenStreetMap). In this work, we propose a novel approach for training on misaligned labels, where we simultaneously learn the label alignment. Our align and segment (AnS) approach builds on the spatial transformer module to transform the misaligned labels using an affine transformation to provide a better learning target for a canonical semantic segmentation network. We prevent shortcut learning of misaligned labels in these semantic segmentation networks through a self-supervised regularization loss and show that it is complementary to data augmentation, especially for systematically misaligned training data. A decisive characteristic of our AnS approach is that it learns without requiring any “golden” labels. We experimentally show on both synthetic and real-world data from different cities that our approach enables high-quality building segmentation and precise label-image alignment at the same time. Code and derived datasets are available at https://github.com/venkanna37/align-and-segment.

Visual fall detection models are usually trained on small,staged datasets. Their real-world utility remains unclear; such data lacksdiversity and evaluation protocols differ from paper to paper. We proposeOmniFall, a unified benchmark of 15k videos (80 hours) with frame-levelannotations in a single 16-class taxonomy. It spans three domains: OF-Staged unifies eight staged datasets with cross-subject and cross-viewsplits; OF-Synthetic adds 12k videos (17 h) with controlled demographicand environmental diversity; and OF-In-the-Wild provides a test-onlyset of genuine accident videos. We evaluate fine-tuned models as wellas much larger zero-shot multimodal LLMs. On in-the-wild fall events,both do comparably well. The clinically critical fallen state is wherethey part: zero-shot models keep confusing fallen with lying, whereasmodels fine-tuned on synthetic data with explicit fallen-state scenes dosubstantially better. We release the unified annotations, the syntheticdata, and the in-the-wild test set to foster the development of fall andfallen-state detectors for uncontrolled environments.Dataset: https://hf.co/datasets/simplexsigil2/omnifall

We investigate the problem of identifying objects that havebeen added, removed, or moved between a pair of captures (imagesor videos) of the same scene at different times. Accurately identifyingverifiable changes is extremely challenging – some objects may appear tobe missing because they are occluded or out of frame, while others mayappear different due to large viewpoint changes. To study this problem, weintroduce the SceneDiff Benchmark, the first multiview change detectiondataset for scenes captured along different camera trajectories, comprising350 diverse video pairs with dense object instance-level annotations. Wealso introduce the SceneDiff algorithm, a training-free approach thatsolves for image poses, segments images into objects, and compares themusing semantic and geometric features. By building on pretrained models,SceneDiff generalizes across domains without retraining and naturallyimproves as the underlying models advance. Experiments on multiviewand two-view benchmarks demonstrate that our method outperformsexisting approaches by large margins (51.6% and 30.6% relative APimprovements). Project page: https://yuqunw.github.io/SceneDiff

Online 3D reconstruction models perform poorly on longvideos. This happens because regressing poses relative to a fixed first-frameanchor forces extrapolation far beyond the training distribution. Smalldrifts accumulate and amplify into significant geometric collapse. However,we observe that per-frame depth remains stable throughout this failure.The backbone’s local geometry remains intact; only the global pose headbreaks down. Motivated by this decoupling, we introduce Scal3R. Thisapproach reformulates online reconstruction as multi-reference relativepose querying. We use lightweight learnable tokens, which make upabout ∼1% of the parameters, and inject them into a completely frozenbackbone via asymmetric attention. This setup queries poses relative tomultiple past keyframes. An online pose-graph optimization system withloop closure suppresses long-range drift. Scal3R reaches convergence in8 hours on a single GPU. It reduces the average ATE by over 60% onKITTI compared to the online baseline. It also achieves state-of-the-artperformance across Virtual KITTI, Sintel, TUM-Dynamic, ScanNet, and7-Scenes. Project page: https://linjohnss.github.io/scal3r/

Image-to-image (I2I) translation has achieved strong resultsin tasks like human relighting and driving scene translation using la-tent diffusion models (LDMs). However, compact LDMs often strug-gle to preserve fine-grained structures because the encoder compresseshigh-resolution inputs into a spatially downsampled latent space. Toaddress this issue, we propose a simple saliency-guided warp–unwarpframework that reallocates spatial representation toward salient regionsbefore encoding, enabling better preservation of structural details with-out increasing latent resolution. The warped image is processed by theoriginal diffusion model and then mapped back via an inverse warp.In addition, we propose a simple and efficient outpainting-based syn-thetic data generation pipeline to produce high-quality paired data forimage relighting. Our method is model-agnostic, requires no architec-tural modification, and introduces negligible computational overhead.Experiments on human relighting, driving scene relighting, and transla-tion demonstrate improved structural preservation, lighting faithfulness,and image quality, with our framework extending naturally to video viaframe-by-frame application with good temporal stability. Project Web-page: https://shenzheng2000.github.io/WarpI2I.github.io/

Visual object tracking relies on modeling cross-frame dynam-ics, and recent approaches employ learnable temporal tokens to encodehistorical information. However, our analysis reveals that these tokensare often marginalized by dominant spatial features during joint opti-mization. This imbalance leads to shortcut learning where the networkover-relies on current-frame appearance, causing temporal representationcollapse. To address this, we propose DASTrack, a framework featur-ing Decoupled Auxiliary Supervision (DAS). By masking the templatesduring an auxiliary forward pass, DAS compels the network to localizetargets using only temporal tokens and thus encode robust cross-framepriors. This mechanism is applied strictly during training and discardedat inference to preserve the original architecture and computational ef-ficiency. DASTrack yields consistent and interpretable average perfor-mance gains across multiple backbones. Our findings demonstrate thatthe primary bottleneck in temporal tracking stems from biased learn-ing dynamics rather than architectural design, offering a novel trainingparadigm for eliciting latent temporal capabilities. The code and modelswill be released at: https://github.com/ZhangDailing8/DASTrack.

In-context learning (ICL) enables large vision-language models (LVLMs) to perform tasks by following patterns in context examples, yet its potential to amplify societal biases remains underexplored. We systematically investigate how ICL influences gender bias in LVLMs through VL-BICLE, an evaluation framework comprising six ICL settings, three tasks, and four datasets. Our experiments on six LVLMs reveal that gendered ICL demonstrations act as a directional force, shifting model bias toward the demonstrated gender through a cross-gender mechanism that disproportionately degrades performance on the opposite gender. This effect appears in image captioning and pronoun prediction but not in visual question answering, indicating that gendered ICL influences bias only when the task output involves gendered language. Similarity-based retrieval methods inherit the training pool’s gender imbalance and offer no debiasing advantage, while standard quality metrics remain blind to these bias shifts. To mitigate this bias, we replace real in-context images with synthetic ones from stable diffusion models while keeping captions unchanged. This simple intervention reduces gender bias without degrading caption quality. The code can be found at: https://github.com/mathfather/Gender-Bias-in-VL-ICL.

We present DRS-VPT, a feed-forward transformer archi-tecture for foundational image-to-scan registration. Given query imagesand a reference 3D point cloud, the model predicts the scan pose andpoint map alongside the poses and point maps of each camera, all ex-pressed in the first camera’s frame. It additionally predicts a coarse-to-fine pyramid of per-point and per-pixel features for direct reprojectivealignment of the scan to the first image. This formulation unifies down-stream tasks such as camera–LiDAR calibration in autonomous driv-ing and indoor camera-to-map relocalization. A single DRS-VPT modelachieves state-of-the-art performance for image-to-LiDAR registration inautonomous driving, competitive indoor relocalization without trainingmap-specific weights, and strong zero-shot transfer to unseen environ-ments. We also show qualitatively that the model learns complex scan-to-image projection properties such as occlusion of back-facing points.

Metric 3D object detection is a core capability for embodied agents, yet most reliable systems lean on depth sensors, trading away cost, power, and integration simplicity. This motivates monocular 3D detection, which avoids additional constraints, yet it faces a major obstacle: from a single image, depth, and especially absolute scale, are underconstrained. As a result, the prevailing pattern of detecting in 2D and then predicting 3D attributes is often brittle, since modest range errors can dominate 3D localization, and the learned scale prior can fail when cameras, motion, or environments undergo domain shifts. To address this, we propose Map-Det3D, an online multi-view 3D object detection model that brings detection directly into a 3D space reconstructed from RGB. We map a short temporal window into multiple views and repurpose a feed-forward metric 3D reconstruction model as our geometric backbone while tuning its object-aware capabilities. Building on this representation, Map-Det3D directly predicts boxes in metric 3D space, without the widely used 2Dto-3D lifting. Experiments across different benchmarks show that this design supports strong online performance and robust transfer without adaptation, suggesting that training reconstruction priors for detection is a practical route to stable metric 3D detection from monocular video. Code and models are available at royyang0714.github.io/Map-Det3D.

We propose a feed-forward method for dense Signed DistanceField (SDF) regression from unstructured image collections in less thanthree seconds, without camera calibration or post-hoc fusion. Our keyinsight is that the intermediate feature space of pretrained multi-viewfeed-forward geometry transformers (FFGT) already encodes a powerfuljoint world representation; yet, existing pipelines discard it, routing fea-tures through per-view prediction heads before assembling 3D geometrypost-hoc, which discards valuable completeness information and accu-mulates inaccuracies. We instead perform 3D extraction directly fromFFGT features via learned volumetric extraction: voxelized canonicalembeddings that progressively absorb multi-view geometry informationthrough interleaved cross- and self-attention into a structured volumet-ric latent grid. A simple convolutional decoder then maps this grid to adense SDF. We additionally propose a scalable, validity-aware supervi-sion scheme directly using SDFs derived from depth maps or 3D assets,tackling practical issues like non-watertight meshes. Our approach yieldscomplete and well-defined distance values across sparse- and dense-viewsettings and demonstrates geometrically plausible completions.

Monocular depth estimation foundation models, such as the Depth Anything series, have achieved remarkable performance across diverse domains. However, they still suffer from critical failures under adverse weather conditions, such as fog, rain, snow, or at night. To address this, we present Weather-Conditioned Depth Anything (DA-W), a framework that explicitly disentangles style from content for weatherrobust depth estimation. Specifically, we introduce a Style Filter trained on a curated mix of real and synthetic degradation datasets to extract content-independent, degradation-aware weather embeddings. This style embedding is then injected into the Depth Anything backbone using a parameter-efficient, zero-initialized adapter. Such a lightweight modulation allows a single unified model to robustly adapt to diverse conditions—including fog, rain, snow, and low-light—while avoiding catastrophic forgetting of its core generalization abilities in normal conditions. We train the adapter using a pseudo-label distillation and alignment strategy. Our comprehensive experiments demonstrate that our proposed DA-W achieves state-of-the-art robust depth estimation, improving AbsRel by an average of 3.7% on our curated weather benchmarks, while matching or slightly outperforming performance on standard clean benchmarks.

Safety alignment of text-to-image (T2I) diffusion models aimsto suppress harmful generations while preserving utility on benign prompts.Recent methods often appear to deliver high safety with high utility, butthis conclusion rests largely on coarse global utility metrics (e.g., FID,CLIPScore) that are insensitive to fine-grained semantic correctness, cre-ating an illusion of high utility. We show that when utility is measuredwith structured evaluation, this illusion breaks: on TIFA (Text-to-ImageFaithfulness evaluation with Question Answering), safety-aligned modelssuffer substantial drops in semantic fidelity, including failures in objectcounts, attributes, and relationships. To diagnose the source of this gap,we analyze the text-encoder prompt embedding space and uncover se-mantic collapse, a contraction of embedding spread coupled with distor-tion of inter-prompt similarity structure, which strongly correlates withstructured utility loss. Guided by this insight, we propose Structure-Aware Geometric Regularization (SAGE)1 , a safety alignment objec-tive that explicitly preserves embedding spread and inter-prompt rela-tional structure during adaptation. Our method restores structured util-ity (TIFA +5.0% over prior state-of-the-art) while maintaining strongsafety performance and competitive coarse-grained utility scores.

Deep research agents integrate fragmented evidence throughmulti-step tool use. BrowseComp offers a text-only testbed for such agents,but existing multimodal benchmarks rarely require both weak visual cuescomposition and BrowseComp-style multi-hop verification. Geolocationis a natural testbed because answers depend on combining multipleambiguous visual cues and validating them with open-web evidence.Thus, we introduce GeoBrowse, a geolocation benchmark that combinesvisual reasoning with knowledge-intensive multi-hop queries. Level 1 testsextracting and composing fragmented visual cues, and Level 2 increasesquery difficulty by injecting long-tail knowledge and obfuscating keyentities. To support evaluation, we provide an agentic workflow GATEwith five think-with-image tools and four knowledge-intensive tools, andrelease expert-annotated stepwise traces grounded in verifiable evidencefor trajectory-level analysis. Experiments show that GATE outperformsdirect inference and open-source agents, indicating that no-tool, search-only or image-only setups are insufficient. Gains come from coherent,level-specific tool-use plans rather than more tool calls, as they morereliably reach annotated key evidence steps and make fewer errors whenintegrating into the final decision. The GeoBrowse benchmark and codesare provided in https://github.com/ornamentt/GeoBrowse.