ECCV 2026 Papers

Compositions of frozen foundation models offer a trainingfree route to instance segmentation, yet a clear gap remains between current systems and their empirical upper bounds. We introduce progressive oracle replacement, a diagnostic procedure that decomposes this gap into stage-level components, and find that the scoring stage accounts for most of it. Across diverse encoders and mask generators, we trace this gap to Quality Misalignment: a mismatch between semantic matching confidence and mask geometric fidelity that yields ranking errors when the two signals are used independently. This pattern persists across all tested configurations, suggesting that it is intrinsic to decoupled scoring. We therefore propose Semantic-Geometric Quality Alignment (SGQA), which replaces decoupled scoring with a single quality score formed by the geometric mean of the two signals and adds no hyperparameters to the scoring formula. Under a fixed backbone of DINOv2 ViT-L paired with SAM2 Hiera-L, SGQA outperforms existing training-free methods and matches or exceeds fine-tuning-based methods with comparable or smaller backbones.

Monocular depth estimation remains challenging for trans-parent objects, where refraction and transmission are difficult to modeland break the appearance assumptions used by depth networks. As a re-sult, state-of-the-art estimators often produce unstable or incorrect depthpredictions for transparent materials. We propose SeeClear, a novelframework that converts transparent objects into diffusion-generated opaqueimages, enabling stable monocular depth estimation for transparent ob-jects. Given an input image, we first localize transparent regions andtransform their refractive appearance into geometrically consistent opaqueappearances using a diffusion-based generative opacification module. Theprocessed image is then fed into an off-the-shelf monocular depth esti-mator without retraining or architectural changes. To train the opaci-fication model, we construct SeeClear-396k, a synthetic dataset con-taining 396k rendered images across 66k paired transparent-opaque con-figurations. Experiments on both synthetic and real-world datasets showthat SeeClear significantly improves depth estimation for transparentobjects.

State-of-the-art post-hoc out-of-distribution detection meth-ods rely on intermediate layer activation editing. However, they exhibitinconsistent performance across datasets and models. We show that thisinstability is driven by differences in the activation distributions, andidentify a failure mode of scaling-based methods that arises when penul-timate layer activations are not rectified. Motivated by this analysis,we propose RAS, a hyperparameter-free post-hoc method that replacessorted activation magnitudes with a fixed in-distribution reference profile.Our simple plug-and-play method shows strong and consistent perfor-mance across datasets and architectures without assumptions on thepenultimate layer activation function, and without requiring any hyperpa-rameter tuning, while empirically preserving in-distribution classificationaccuracy. We further analyze what drives the improvement, showing thatboth inhibiting and exciting activation shifts independently contribute tobetter out-of-distribution discrimination1 .

Although recent trajectory prediction and end-to-end au-tonomous driving methods improve robustness in urban environments,they still lack meaningful controllability. Existing benchmarks either pro-vide no persona-conditioned annotations or support only a single urgencyspectrum (i.e., emergency, normal, relaxed), which cannot distinguishpersonas that share the same urgency level but require different drivingdynamics. To address this, we propose (i) the Persona-Conditioned Tra-jectory (PCT) dataset, which decomposes driving personas along twoaxes—Temporal Urgency and Ride Comfort—and combines three levelsof each to form a grid of nine personas, each paired with natural-languagedescriptions and trajectories, and (ii) PersonaDrive, a framework thatcan learn driving personas from language and can generate persona-specific trajectories. PersonaDrive incorporates Persona-Conditioned An-chor Transform (PCAT), which hierarchically reshapes anchors alongboth axes, and Persona-Conditioned Multi-Modal Fusion (PCMF) forBEV-level persona fusion. Training is supervised by a Hierarchical GuideLoss enforcing axis-aligned physical orderings and an Axis-DecomposedDiversity Loss preventing diagonal mode collapse. Experimental resultsshow that PersonaDrive consistently improves over the compared base-lines across multi-dimensional scenarios. The code and PCT dataset areavailable at https://github.com/VisualAIKHU/PersonaDrive.

Modern smartphones capture Live Photos—short video bursts sur-rounding a still image—offering a dynamic and engaging photographic experi-ence. However, the cover photo and video components are generated by two dis-tinct imaging pipelines: the photo stream undergoes full computational photog-raphy processing, while the video stream is constrained by real-time efficiencyand heavy compression. This intrinsic separation produces a substantial qual-ity gap in resolution, color fidelity, and dynamic range between the cover photoand video frames. When users reselect an alternative frame from the video toreplace an imperfect cover, the chosen frame often suffers from severe degrada-tion, making direct replacement visually unsatisfactory. Restoring such framesrequires simultaneous enhancement of spatial detail and color appearance, a taskconsiderably more challenging than ordinary super-resolution or color enhance-ment. To address this, we define the Live Photo Cover Frame Reselection andEnhancement (LPRE) task, which leverages the intrinsic cues available withineach Live Photo: the high-quality cover image as a structural and color reference,the user-reselected low-quality frame as the reconstruction target and several ad-jacent video frames providing temporal cues. Building upon this formulation, weconstruct Live2K, a real-world dataset of 2,042 Live Photos, and develop a unifiedone-stage baseline that integrates multi-frame fusion, guided color enhancementand super-resolution—establishing the first benchmark for Live Photo enhance-ment research.

Omnidirectional stereo matching (OSM) estimates 360◦ depthfrom multi-view fisheye images, where inherent fisheye distortion causessparse matching cues in seam regions. However, existing methods rely onspatially uniform update policies, degrading performance in these chal-lenging regions. To address this limitation, we propose PDF-Omni, ageometry-aware OSM framework built on a Poincaré Dual Disk distor-tion field that models spatially varying geometric uncertainty across seamregions. We introduce a distortion-informed GRU with distortion-awareattention to selectively expand the receptive field in high-uncertaintyregions while preserving efficiency in low-uncertainty ones. We furtherpresent a local entropy cumulative loss to supervise high-entropy costdistributions in ambiguous regions. Experimental results demonstratestate-of-the-art performance with consistent gains especially in seam re-gions. The code is available at https://github.com/ynskyang/PDF-Omni

Distilled autoregressive di!usion models facilitate real-timeshort video synthesis but su!er from severe error accumulation duringlong-sequence generation. While existing Test-Time Optimization (TTO)methods prove e!ective for images or short clips, we identify that they failto mitigate drift in extended sequences due to unstable reward landscapesand the hypersensitivity of distilled parameters. To overcome these limi-tations, we introduce Test-Time Correction (TTC), a training-free alter-native. Specifically, TTC utilizes the initial frame as a stable referenceanchor to calibrate intermediate stochastic states along the sampling tra-jectory. Extensive experiments demonstrate that our method seamlesslyintegrates with various distilled models, extending generation lengthswith a slight overhead while matching the quality of resource-intensivetraining-based methods on 30-second benchmarks.

Bird’s-eye view (BEV) perception fuses multi-camera imagesinto a unified top-down representation for autonomous driving. Despiterecent progress, state-of-the-art methods remain confined to closed-setscenarios, making them vulnerable to unpredictable real-world environ-ments. In this work, we introduce open-vocabulary BEV segmentation(OVBS), which leverages vision-language models (VLMs) to recognizecategories beyond the training set while maintaining precise BEV percep-tion and real-time efficiency. A key challenge in OVBS lies in the 3D geo-metric inconsistency inherent in the ill-posed lifting of 2D VLM semanticsinto BEV. To address this, we propose OVBEVSeg, a geometry-awareOVBS framework that enhances efficient Gaussian splatting (GS)-basedunprojection by leveraging robust 3D geometric constraints across threeprogressive stages: (1) 2D-to-BEV pseudo-labeling via reliable 3D pro-jection for OV generalization; (2) joint 2D–BEV per-scene optimizationwith BEV structural constraints for 3D geometric consistency; and (3)3D geometric distillation for online efficiency. On the nuScenes dataset,OVBEVSeg achieves state-of-the-art performance, outperforming closed-set methods by 15.3 mIoU on unseen categories. Remarkably, even withno novel-class ground-truth labels, it remains competitive with self- andsemi-supervised baselines trained with up to 40% of ground-truth anno-tations. Furthermore, it achieves 2.5× faster inference with only 0.22×the memory consumption of projection-based methods.

We tackle the problem of sparse novel view synthesis (NVS)using video diffusion models; given K (≈ 5) posed images of a scene,we predict the view from a target camera pose. Many prior approachesleverage generative image priors encoded via diffusion models. However,models trained on single images lack multi-view knowledge, and we in-stead argue that video models contain implicit multi-view knowledge andare therefore easier to adapt for NVS. Our key insight is to formulatesparse NVS as a low-frame-rate video completion task. However, onechallenge is that sparse NVS is defined over an unordered set of inputimages, often too sparse to admit a meaningful order, so the modelsshould be invariant to permutations of that input image set. To thisend, we present FrameCrafter, which adapts video models (naturallytrained with coherent frame orderings) to permutation-invariant NVSthrough simple but effective architectural modifications, including per-frame latent encoding, query-centered camera conditioning and removalof temporal positional embeddings. Our results suggest that video modelscan be easily modified to “forget” about time with minimal supervision,producing state-of-the-art performance on sparse-view NVS benchmarks.

Visual navigation policies are widely regarded as a critical re-search direction, as they emulate human navigation behavior by leverag-ing egocentric visual observations. However, unlike LiDAR point cloudsor depth maps, visual observations do not provide explicit geometricinformation for navigation, especially in cluttered or dynamic environ-ments, which motivates the need for learning-based models and large-scale data. To this end, we propose to leverage Vision-Language-Action(VLA) models to learn diverse navigation capabilities from synthetic ex-pert data and to alleviate the sim-to-real gap by co-training on large-scalereal-world Visual Question Answering (VQA) data. Specifically, we de-velop MM-Nav, a 7B multi-view VLA model featuring custom-designedarchitectures and enabling a 7 Hz inference speed with 360◦ observation.For large-scale navigation data, we collect a total of 1.5 million expertdemonstrations from three reinforcement learning (RL) experts, eachtrained with privileged information in a challenging, tailor-made envi-ronment and specialized in one of three navigation capabilities: reaching,squeezing, and avoiding. We then iteratively train MM-Nav on these data,dynamically balancing the training data ratio across the three capabil-ities based on their respective performance. Through extensive exper-iments in synthetic and real-world environments, we demonstrate thatour model achieves strong performance and generalization on differentbenchmarks. MM-Nav obtains a success rate of 88.1% on the InternVLA-N1 System-1 point-goal navigation benchmark. Moreover, we find thatour student VLA model outperforms the RL teachers, demonstrating thesynergistic effect of integrating multiple capabilities. Extensive real-worldexperiments further confirm the effectiveness of our method.

Video-to-video diffusion models achieve impressive singleturn editing performance, but practical editing workflows are inherently iterative. When edits are applied sequentially, existing models treat each turn independently, often causing previously generated regions to drift or be overwritten. We identify this failure mode as the problem of cross-turn consistency in multi-turn video editing. We introduce Memory-V2V, a memory-augmented framework that treats prior edits as structured constraints for subsequent generations. Memory-V2V maintains an external memory of previous outputs, retrieves task-relevant edits, and integrates them through relevance-aware tokenization and adaptive compression. These technical ingredients enable scalable conditioning without linear growth in computation. We demonstrate Memory-V2V on iterative video novel view synthesis and text-guided long video editing. Memory-V2V substantially enhances cross-turn consistency while maintaining visual quality, outperforming strong baselines with modest overhead.

Audio-driven avatar interaction demands real-time, stream-ing, and in_x001C_nite-length generation_x0016_capabilities fundamentally at oddswith the sequential denoising and long-horizon drift of current di_x001B_usionmodels. We present Live Avatar, an algorithm-system co-designed frame-work that addresses both challenges for a 14-billion-parameter di_x001B_usionmodel. On the algorithm side, a two-stage pipeline distills a pre-trained bidirectional model into a causal, few-step streaming one whoseKV cache stores noisy rather than clean history. This lossy condition-ing, counter-intuitively, suppresses error accumulation and enables mod-els trained on short clips to generalize stably beyond 10,000 seconds.On the system side, this noisy conditioning also breaks the sequen-tial sampling bottleneck, allowing Timestep-forcing Pipeline Parallelism(TPP) to assign each GPU a _x001C_xed denoising timestep, converting the se-quential di_x001B_usion chain into an asynchronous spatial pipeline that simul-taneously boosts throughput and improves temporal consistency. LiveAvatar achieves 45 FPS with a TTFF of 1.21 s on 5 H100 GPUs, andto our knowledge is the _x001C_rst to enable practical real-time streaming ofa 14B di_x001B_usion model for in_x001C_nite-length avatar generation. Our projectpage is at https://liveavatar.github.io/.

Large foundation models trained on extensive datasets demon-strate strong zero-shot capabilities in various domains. Knowledge dis-tillation has become an established tool for transferring knowledge fromfoundation models to small student networks when data and model sizeare constrained. However, the efficacy of distillation is often hampered bylimited training data coverage. This can result in a covariate shift betweentraining and test data, which in turn can lead the student to exploitspurious features or even shortcut learning. We address this problemby introducing a novel diffusion-based data augmentation strategy thatgenerates images by maximizing the disagreement between the teacherand the student, effectively creating challenging samples that the studentstruggles with, thus mitigating the problem of covariate shift. Experi-ments demonstrate that, compared to state-of-the-art diffusion-based dataaugmentation baselines, our approach is best or second-best in samplemean accuracy and improves the worst group and mean group accuracyon CelebA-HQ, SpuCo Birds and BAR as well as the spurious score onSpurious ImageNet under covariate shift.

The quadratic time and memory complexity of the attention mechanism in modern Transformer-based video generators makes endto-end training for ultra-high-resolution videos prohibitively expensive. Motivated by this limitation, we introduce a training-free approach that leverages video Diffusion Transformers pre-trained at their native scale to synthesize higher-resolution videos without any additional training or adaptation. At the core of our method lies an inward sliding-window attention mechanism, which originates from a key observation: maintaining each query token’s training-scale receptive field is crucial for preserving visual fidelity and detail. However, naive local window attention, unfortunately, often leads to repetitive content and exhibits a lack of global coherence in the generated results. To overcome this challenge, we devise a dual-path pipeline that backs up window attention with a novel cross-attention override strategy, enabling the semantic content produced by local attention to be guided by another branch with a full receptive field and, therefore, ensuring holistic consistency. Furthermore, to improve efficiency, we incorporate a cross-attention caching strategy for this branch to avoid the frequent computation of full 3D attention. Extensive experiments demonstrate that our method delivers ultra-highresolution videos with fine-grained visual details and high efficiency in a training-free paradigm. Meanwhile, it achieves superior performance on VBench, even compared to training-based alternatives, with competitive or improved efficiency.

State Space Models (SSMs) have surfaced as a promisingarchitecture in Video Frame Interpolation (VFI), as they can capturelong-range dependencies with linear computational complexity. However,their predefined scanning order limits their effectiveness in modelingthe dynamic motion trajectories inherent in VFI problems. To tacklethis challenge, we propose Motion-Guided Mamba for Video FrameInterpolation (MGMVFI), an adaptation of the selective state spacemodel tailored explicitly for VFI. MGMVFI introduces Motion-GuidedSerialization (MGS), which leverages optical flow to define a motion-adaptive 1D input order for the SSM. This aligns the causal state up-dates with semantically related tokens, enabling motion-consistent fea-ture propagation, particularly for large and dynamic motions. Addition-ally, to mitigate the unreliable feature representations caused by inaccu-rate optical flow estimates, we introduce contextual synthesis that uti-lizes the surrounding spatial context for robust inter-frame feature syn-thesis. These components are seamlessly integrated within our tailoredMamba architecture, which also employs a lightweight refinement blockto enhance local detail reconstruction at a reduced computational cost.Extensive experiments on standard VFI benchmarks demonstrate thatMGMVFI achieves state-of-the-art performance, particularly on complexand dynamic motions, thereby establishing a new direction for sequencemodeling in video interpolation.

3D dog pose estimation is hindered by two limitations in ex-isting benchmarks: distributional bias toward canonical viewpoints, sta-ble poses, and narrow morphologies, and alignment degradation betweenimages and 3D annotations. We instead synthesize a large-scale bal-anced benchmark (4.2M images with exact 3D ground truth) by decou-pling pose, shape, and texture into independent libraries (70K retargetedposes, 500 breed shapes, 1,000 appearance maps) and rendering directlyto guarantee pixel-perfect alignment and broad view coverage. However,this balanced distribution exposes a challenge previously masked by bi-ased data: non-canonical combinations of pose, shape, and viewpointcreate multimodal 3D ambiguities that deterministic regression cannotresolve. We therefore propose a flow-based probabilistic framework thatmodels the conditional distribution of 3D poses given an image, pro-ducing multiple plausible hypotheses under ambiguity while convergingto precise estimates for clear views. Experiments on StanfordExtra, An-imal3D, and our synthetic benchmark demonstrate that our method,trained solely on synthetic data, outperforms state-of-the-art approaches(79.9 vs. 33.2 PCK on the synthetic benchmark). Dataset available here.

Decomposing outgoing surface radiance into material and illumination during inverse rendering is essential for applications such as relighting and augmented reality, yet it is severely ill-posed since multiple combinations can result in the same observed colour. Capturing an object under multiple lighting conditions usually helps resolve this ambiguity as it constrains the optimization towards correct solutions. In this work, we explore the potential of reconstructing rigidly moving objects—which provides observations of diverse light-surface interactions—to resolve the material-lighting ambiguity in inverse rendering. For this purpose, we introduce a relightable approach that marries object tracking and reconstruction with inverse rendering for general rigidly moving objects. Our experimental analysis on synthetic data demonstrates that motion can be an advantage for disentangling material and lighting: the reconstructed material is significantly more accurate when the object is observed under rigid motion than when it is static. Moreover, results on RGB videos of real hand-held objects show that our pipeline preserves this advantage even under noisy real-world conditions.

In industrial environments, new product categories arrive se-quentially, requiring continual anomaly detection without access to pastdata. Normalizing Flows (NFs) provide exact density estimation but suf-fer from catastrophic forgetting as parameter updates across tasks dis-tort the density manifold. While parameter isolation can prevent inter-ference, it must preserve the strict invertibility and Jacobian validityof NFs. To satisfy these requirements, we exploit the inherent propertythat affine coupling layers maintain transformation validity regardless ofsubnet parameterization. Based on this, we propose DeCoFlow, which de-composes subnets into a frozen universal base and task-specific low-rankadapters to isolate updates. We further introduce Task-Specific Align-ment, Auxiliary Coupling Layers, and Tail-Aware Loss to compensatefor frozen-base rigidity. DeCoFlow achieves state-of-the-art image-levelAUROCs of 98.40% on MVTec-AD and 93.00% on VisA, while maintain-ing parameter-level zero forgetting (0.00% FM under correct routing)with only 2.27M parameters per task.

Generalized Category Discovery (GCD) is an intriguing open-world problem that has garnered increasing attention: given partially la-belled data, the goal is to correctly recognize known classes while discov-ering coherent novel categories from unlabelled samples. Recent GCDmethods typically adapt foundation models by jointly optimizing su-pervised classification and unsupervised discovery objectives on mixedlabelled–unlabelled data. While effective, this coupled training can en-tangle closed-set recognition and open-set discovery, leading to objectiveconflict and biased predictions, and may disturb the semantic geome-try of pretrained representations under limited labels and noisy pseudo-labels. We propose CloSeR, a simple plug-and-play framework that in-jects Closed-Set Relational knowledge into GCD training. CloSeR firstbuilds a domain-adapted closed-set teacher by tuning lightweight block-wise adapters on labelled known-class data while keeping the founda-tion model backbone frozen, thereby preserving pretrained priors at lowtraining cost. It then transfers the teacher’s knowledge to downstreamGCD via Unified Relational Distillation (URD), which distills comple-mentary global sample-to-prototype relations to anchor known-class se-mantics and local sample-to-sample relations to preserve neighborhoodstructure, using separate feature pathways to reduce optimization in-terference. CloSeR is head-agnostic and readily integrates with bothparametric and non-parametric GCD methods. Extensive experimentswith DINO and DINOv2 backbones on six benchmarks (CIFAR-10/100,ImageNet-100, CUB, Stanford-Cars, and FGVC-Aircraft) show consis-tent gains over GCD baselines, achieving state-of-the-art performance.Project page: https://visual-ai.github.io/closer/