ECCV 2026 Papers

Interactive video-to-video applications require real-time gen-eration while maintaining long-range temporal consistency. However, re-cent methods achieve speed by restricting attention to a fixed-lengthcausal window. This limits long-range consistency whenever relevant his-tory falls outside the context window. To address this limitation, we in-troduce vid2vid-long, a correspondence-based approach grounded in asimple observation. Video-to-video translation provides an explicit cross-frame reference: the regions that match in the input video should matchin the output video too. Concretely, we compute patch-level correspon-dences in the input, guiding retrieval of previously generated outputs inthe corresponding regions, and adding them to the context window. Ourexperiments demonstrate that vid2vid-long consistently improves long-range temporal consistency across recent video models while preservinggeneration quality and real-time performance.

Chemical reaction diagram parsing aims to automaticallyconvert reactions from images into machine-readable formats. Currentleading approaches employ end-to-end generative models to directly de-code reaction diagrams into structured coordinate sequences, performingobject localization and relationship modeling in a unified process. How-ever, two critical bottlenecks remain: severe data scarcity due to thehigh cost of expert annotation, and output instability caused by thecomplex sequential dependencies between chemical reactions, where mi-nor prefix errors can lead to cascading parsing failures. In this paper, wepropose RobustRDP, a Robust Reaction Diagram Parser built on amultimodal large language model (MLLM). Its superior performance pri-marily stems from two advancements: Synthetic-to-Real Data Scal-ing: We develop a layout-driven synthesizer and an efficient annotationplatform to create a large-scale training set, along with a more compre-hensive evaluation benchmark. Robustness-Oriented Training: Wepropose a three-stage progressive training strategy (Pretraining, Multi-Task SFT, DPO). The SFT stage employs two auxiliary tasks includ-ing region-guided reaction parsing and prefix-perturbed reaction parsingto mitigate sequential dependencies between reactions and enhance themodel’s robustness against prior errors. The DPO stage further stabilizesoutputs in failure-prone scenarios. Extensive experiments demonstratethe superior performance of RobustRDP, establishing a solid foundationfor automated reaction diagram parsing. Code and model weights areavailable at https://github.com/jaydetang/RobustRDP.

Skeleton-based action recognition classifies human actionsfrom sequences of joint coordinates and is widely studied using graph con-volutional networks (GCNs). GCNs typically rely on repeated graph nodefeature aggregation via one-hop message passing to capture long-rangedependencies across entire joint sequences, while progressively refiningrepresentations from multiple perspectives through block-wise learnedtopology. However, this repeated aggregation can often degrade repre-sentations, making the node features in the final embedding increasinglyhomogenized and causing the final embedding to fail to capture local orfine-grained representations formed in early blocks. To address these is-sues, we propose a novel topology-conditioned block-progressive memoryfor skeleton GCNs. Along the GCN block axis, memory stores comple-mentary block-wise representations and fuses the accumulated memorystate into the final embedding, thereby mitigating representation degra-dation caused by repeated aggregation. At each block, a sample-specificmemory state is updated by minimizing a topology-conditioned recon-struction loss, where the reconstruction error quantifies how well theprevious memory explains the current block representation while captur-ing newly emerging, discriminative cues. We support two inference-timemodes: memory-free, which uses memory only during training, and anoptional memory-enabled mode, which also applies the same memoryupdate rule at test time for modest gains. Both variants achieve state-of-the-art results on large-scale benchmarks, including NTU RGB+D,NTU RGB+D 120, and Kinetics-Skeleton.

Due to the sparsity of LiDAR measurements, RGB-assisteddense depth reconstruction is widely adopted to provide structural pri-ors for autonomous driving. However, the inherent sensitivity of RGBimaging to illumination conditions makes dense depth reconstruction un-der low-light scenarios still a critical challenge. Specifically, under long-exposure imaging, motion blur in low-light RGB frames significantlydegrades the accuracy of depth reconstruction. To address this issue,we exploit the high-temporal-resolution motion cues captured by eventcameras and propose Event-guided Restoration and Upsampling Network(ERU-Net), a unified framework that tightly couples event-guided featurerestoration with depth completion. The Event-guided Feature Restora-tion (EFR) module combines implicit neural representation (INR) andself-recursive optimization to remove motion blur from long-exposure in-puts and recover artifact-free features. These features serve as structuralpriors for the Restoration-Aware Depth Upsampling (RDU) module, en-abling accurate completion of sparse LiDAR measurements with finegeometric details. Extensive experiments on challenging synthetic andreal-world captured datasets demonstrate that ERU-Net significantlyoutperforms state-of-the-art approaches, producing accurate and tem-porally consistent dense depth sequences in severe low-light conditions.Our code will be available at https://github.com/qzm777/Night-RIDE-ECCV2026.

RGB–Thermal (RGB-T) salient object detection leveragescomplementary cues from visible and thermal modalities to improve ro-bustness in challenging environments. However, in real-world scenarios,the reliability of each modality is inherently unstable: RGB images de-grade under low illumination, motion blur, and noise, while thermal im-agery often suffers from contrast compression and sensor artifacts. Suchdegradation introduces unreliable perceptual evidence that can misleadcross-modal fusion and significantly deteriorate detection performance.To address this challenge, we propose RA-SOD, a reliability-aware RGB-T salient object detection framework that explicitly models modality re-liability and integrates it into feature learning and cross-modal fusion.First, we introduce a reliability-conditioned representation that adap-tively compensates degraded modality features while preserving struc-tural cues. Second, an uncertainty-guided dual-stream refinement strat-egy progressively corrects cross-modal representations while suppress-ing unreliable evidence. Finally, we propose a pixel-wise modality com-petition mechanism that dynamically selects modality cues accordingto spatial reliability for fine-grained fusion. Extensive experiments onfour benchmarks (VT821, VT1000, VT5000, and VT-IMAG) demon-strate that RA-SOD achieves state-of-the-art performance and exhibitsstrong robustness under severe modality degradation. Code and modelsare available at https://github.com/zaoxienian/RA-SOD.

Cooperative perception through Vehicle-to-Everything (V2X) communication offers significant potential for enhancing vehicle perception by mitigating occlusions and expanding the field of view. However, past research has predominantly focused on improving accuracy metrics without addressing the crucial system-level considerations of efficiency, latency, and real-world deployability. Noticeably, most existing systems rely on full-precision models, which incur high computational and transmission costs, making them impractical for real-time operation in resource-constrained environments. In this paper, we introduce QuantV2X, the first fully quantized multi-agent system designed specifically for efficient and scalable deployment of multi-modal, multi-agent V2X cooperative perception. QuantV2X introduces a unified end-to-end quantization strategy across both neural network models and transmitted message representations that simultaneously reduces computational load and transmission bandwidth. Remarkably, despite operating under low-bit constraints, QuantV2X achieves accuracy comparable to full-precision systems. More importantly, when evaluated under deployment-oriented metrics, QuantV2X reduces system-level latency by 3.2× and achieves a +9.5 improvement in mAP30 over full-precision baselines. Furthermore, QuantV2X scales more effectively, enabling larger and more capable models to fit within strict memory budgets. These results highlight the viability of a fully quantized multi-agent intermediate fusion system for real-world deployment. The system will be publicly released to promote research in this field: https://github.com/ucla-mobility/QuantV2X.

Images composed of 2D pixel arrays are the standard inputto computer vision algorithms, yet many underlying computations can bedistributed across pixels. Transmitting raw, redundant, and noisy pixeldata off the sensor remains inefficient, motivating a shift toward focal-plane sensor-processors that perform a significant part of the computa-tion directly within each pixel. We envision pixels synthesizing higher-level signals locally, reducing downstream load, and providing richer in-puts for higher-level vision tasks.We propose a fully parallelizable form of visual odometry and depthestimation across pixels, where sensor-processors exchange informationthrough Gaussian Belief Propagation (GBP) to achieve consensus aboutcamera motion and infer depth from per-pixel photometric observationsand a surface normal prior. To maintain geometric stability during op-timization, we introduce a keyframe-like anchoring mechanism that reg-ulates the effective baseline between frames, enabling consistent mo-tion and depth updates. Our method is evaluated on realistic datasets,demonstrating the feasibility of GBP-based pixel-level distributed odom-etry and depth estimation with keyframe anchoring on-sensor. ProjectPage: https://www.shinjeongkim.com/pixvod/

Developing physically aware video generation models remainsa significant challenge due to the difficulty in capturing diverse physicalphenomena, such as thermal dynamics, mechanics, and optics. In thiswork, we introduce PhysRAG, a novel pipeline that enhances physicalawareness in video generation through Retrieval-Augmented Generation(RAG). To address the issue of limited high-quality data, we design atwo-stage data filtering pipeline based on the WISA-80K dataset, result-ing in a curated set of 7K high-quality videos for training. Furthermore,we construct a physical video database and develop a mechanism to injectphysical knowledge into a video diffusion model using learnable queries.Our method achieves state-of-the-art performance in both visual qualityand physical rule compliance, surpassing existing models in benchmarkssuch as PhyGenBench and VBench. We conduct extensive ablation stud-ies to validate the effectiveness of our key components, including the datafiltering pipeline, RAG mechanism, and method for physical informationextraction. To facilitate future research, our code, data, and models areprepared for release at https://github.com/sediment1024/PhysRAG.

Skeleton-based action recognition has achieved remarkablesuccess by exploiting joint coordinates and their topological connec-tions, yet prevailing methods overwhelmingly assume complete and cleanskeleton inputs. In real-world deployments, such as egocentric vision,crowded surveillance, wearable devices, or edge robotics, limited field-of-view (FoV) frequently causes substantial joint visibility dropout, leadingto severe performance degradation that existing models are largely un-prepared to handle. To bridge this critical yet underexplored gap, weintroduce PartialVisGraph, a novel hypergraph framework tailored forrobust skeleton action recognition under constrained FoV. We first con-struct highly expressive hypergraphs by introducing learnable virtualhyperedges that form a soft incidence matrix, capturing flexible high-order dependencies beyond conventional pairwise graphs. We then pro-pose the Single-Head Sample-Adaptive Transformer, which adaptivelyaggregates joint features onto hyperedges while explicitly incorporatinga visibility prior. This prior selectively gates information flow, preventingoccluded or out-of-view joints from corrupting reliable feature propaga-tion. We further establish rigorous evaluation protocols with realisticFoV simulation benchmarks on NTU RGB+D 60 and 120. Extensive ex-periments demonstrate that PartialVisGraph consistently achieves state-of-the-art accuracy under partial visibility, with gains of up to 68.8% onsubsets with severe FoV restrictions compared to recent strong base-lines, while remaining superior on full-visibility settings. Our approachoffers a principled and practical pathway toward deployable skeleton-based action understanding in unconstrained environments. Resourcesrelated to the constrained FoV setting used in this work are available at:https://github.com/yaa1haa1/PartialVisGraph.

This study focuses on Text-to-Sounding-Video (T2SV) gen-eration, which aims to generate a video with synchronized audio fromtext, with both modalities aligned to the text conditions. Despite progressin joint audio-video training, two critical challenges remain: (1) text con-ditioning is a bottleneck—shared captions (TV = TA ) trigger modalinterference, while a gap persists between dense training captions andconcise inference user prompts, and (2) the optimal fusion mechanismfor cross-modal feature interaction remains unclear. To address the firstchallenge, we first propose the Cross-Referential Rewriter (CRR) cap-tion framework, a dual-agent pipeline where a Semantic Checker extractsgrounded Semantic Anchors and a Cross-Modal Rewriter generates dis-entangled caption pairs (TV and TA ), eliminating modal interference and1 →Equal contribution. Work done during an internship at Apple.2 †Project lead.3 ‡Corresponding author.bridging the training-inference gap via prompt expansion. For the sec-ond, we introduce BridgeDiT, a dual-tower di!usion transformer thatemploys Dual Cross-Attention (DCA) as a bidirectional bridge betweenvideo and audio streams, which we show through systematic compari-son to be the optimal fusion strategy for the dual-tower paradigm. Ex-tensive experiments on three benchmarks, supported by human evalu-ations, demonstrate state-of-the-art results on most metrics. Compre-hensive ablation studies further validate each component and o!er keyinsights for future T2SV systems. The codes and models are available athttps://bridgedit-t2sv.github.io/.

A faithful 3D world representation should account for layeredgeometry, where a single camera ray may contain multiple visible and ge-ometrically valid surfaces. Monocular depth estimation, however, reducesthis structure to one scalar depth per pixel. Transparent scenes makethis ambiguity measurable: the same ray can pass through foregroundglass and observe the background, turning the supervised target into aconvention of annotation, data, and training rather than a scene-intrinsictruth. A learned predictor exposes this convention as its depth-layerpreference. We introduce MultiDepth-3k (MD-3k), a sparse two-layer or-dinal benchmark for measuring depth-layer preference and multi-layerspatial relationship accuracy (ML-SRA). On MD-3k, leading depth foun-dation models exhibit diverse layer preferences under standard RGBinput, showing that the same layered geometry can be resolved differentlyacross models. We further find that Laplacian Visual Prompting (LVP),a training-free spectral input transformation, can substantially changethe reported layer for certain frozen models. The strongest RGB/LVPpair, DAv2-L, reaches 75.5% ML-SRA. These results suggest that depthfoundation models may express complementary geometric hypotheses thatstandard RGB inference leaves unexpressed. We invite the community torethink depth supervision and evaluation through an ambiguity-awarelens, where multiple valid 3D interpretations are treated as geometricstructure to be measured, preserved, and expressed.

Recovering metric 3D geometry from monocular images is a fundamental computer vision task, yet current methods remain heavily fragmented by fixed camera model assumptions and inflexible input schemes. We present OmniPoint, a unified framework designed to generalize metric reconstruction across diverse imaging sensors, including pinhole, fisheye, and equirectangular projections, while accommodating varying geometric priors. To overcome projection rigidity, OmniPoint abandons conventional planar depth regression. It instead adopts a decoupled ray and distance representation alongside a decoupled training objective, explicitly separating the camera projection model from the scene structure. To address the severe scarcity of training data for alternative cameras, we introduce a bidirectional augmentation strategy that explicitly bridges labeled perspective data and unlabeled omnidirectional domains in 3D space. Furthermore, to seamlessly integrate optional inputs like camera intrinsics or sparse depth without destabilizing the network through feature distribution shifts, we propose a robust information injection mechanism. This mechanism utilizes learnable input state embeddings to resolve architectural ambiguity and applies vectorized Gaussian smoothing to densify irregular measurements. Extensive experiments demonstrate that OmniPoint achieves state-of-the-art zeroshot performance across multiple benchmarks, establishing a robust new standard for unified monocular 3D reconstruction.

Test-Time Adaptation (TTA) has emerged as an effectivesolution for adapting Vision Transformers (ViT) to distribution shiftswithout additional training data. However, existing TTA methods oftenincur substantial computational overhead, limiting their applicability inresource-constrained real-world scenarios. To reduce inference cost, plug-and-play token aggregation methods merge redundant tokens in ViTsto lower computation demands. Albeit efficient, it suffers from signif-icant performance degradation when directly integrated with existingTTA methods. We formalize this problem as Efficient Test-Time Adap-tation (ETTA), seeking to preserve the adaptation capability of TTAwhile reducing inference latency. In this paper, we start by providingan analysis showing that token aggregation inherently leads to informa-tion loss, which cannot be fully mitigated by conventional norm-tuning-based TTA methods. Guided by this insight, we propose to NeutralizeToken Aggregation via Information Augmentation (NAVIA). Specif-ically, we propose an information augmentation mechanism that em-ploys an input-level [CLS] embedding augmentation to compensate fordomain-level information loss, and a feature-level [CLS] bias augmen-tation for fine-grained layerwise information compensation. Theoreticalanalysis provides a principled motivation that augmenting a global infor-mation carrier before aggregation can improve the learnable-informationupper bound, with entropy minimization serving as a practical surro-gate objective. Extensive experiments across various out-of-distributionbenchmarks and model backbones demonstrate that NAVIA outperformsall competing methods. Notably, NAVIA improves performance by upto 3.1% while achieving 14% to 26% wall-clock speedup over the TTAstate-of-the-art, effectively addressing the ETTA challenge. We provideour official implementation at https://github.com/Bostoncake/NAVIA.

Spatial reasoning is foundational for Vision-Language Mod-els (VLMs), particularly when deployed as Vision-Language-Action (VLA)agents in physical environments. However, existing benchmarks predomi-nantly focus on elementary, single-hop relations, neglecting the multi-hopcompositional reasoning and precise visual grounding essential for real-world scenarios. To address this, we introduce MultihopSpatial, o!er-ing three key contributions: (1) A comprehensive benchmark designedfor multi-hop and compositional spatial reasoning, featuring 1- to 3-hopcomplex queries across diverse spatial perspectives. (2) Acc@50IoU,a complementary metric that simultaneously evaluates reasoning andvisual grounding by requiring both answer selection and precise bound-ing box prediction—capabilities vital for robust VLA deployment. (3)MultihopSpatial-Train, a dedicated large-scale training corpus to fos-ter spatial intelligence. Extensive evaluation of 37 state-of-the-art VLMsyields eight key insights, revealing that compositional spatial reasoningremains a formidable challenge. Finally, we demonstrate that reinforce-ment learning post-training on our corpus enhances both intrinsic VLMspatial reasoning and downstream embodied manipulation performance.

Knowledge distillation (KD) aims to transfer useful informa-tion from a large-scale model (teacher) to a lightweight model (student).Classical KD focuses on leveraging the teacher’s predictions as soft labelsto regularize student training. However, the exact match of predictionsin Kullback-Leibler (KL) divergence could be somewhat in conflict withthe classification objective, given that the distribution discrepancies be-tween teacher-generated predictions and ground-truth annotations tendto be fairly severe. In this paper, we rethink the role of teacher predic-tions from a Mixture-of-Experts (MoE) perspective and transfer knowl-edge by introducing teacher predictions as latent variables to reformulatethe classification objective. This MoE strategy results in breaking downthe vanilla classification task into a mixture of easier subtasks with theteacher classifier as a gating function to weigh the importance of sub-tasks. Each subtask is efficiently conquered by distinct experts that areeffectively implemented by resorting to multi-level teacher outputs. Wefurther develop a theoretical framework to formulate our method, termedMoE-KD, as an Expectation-Maximization (EM) algorithm and provideproof of the convergence. Extensive experiments manifest that MoE-KDoutperforms advanced knowledge distillers on mainstream benchmarks.

Vision and language models (VLMs) hold immense promiseto transform biomedical imaging workflows, from detecting lesions inchest X-rays to profiling cellular features in microscopy. Realizing this po-tential, however, requires robust and fine-grained visual perception. Mod-els need to correctly interpret subtle features in images, and they mustdo so across diverse biomedical modalities, scales, and contexts. Never-theless, current benchmarks remain limited. To address these gaps, weintroduce the Massive Multimodal Biomedical Understanding (MMBU)benchmark. It is the largest biomedical vision and language benchmarkto date, covering 35 submodalities with rich structured metadata. Itincludes both open and closed versions of ungrounded classification,grounded classification, and object detection, enabling systematic evalu-ation of model performance across biological scales, clinical settings, andimaging modalities. Evaluating 15 open-weight and 2 frontier VLMs, wefind that while medical adaptation provides measurable gains for somemodels, the high accuracy often reported on established benchmarks canmask deficiencies in visual perception and domain generalization.

Retrieval-Augmented Generation (RAG) mitigates halluci-nations in Multimodal Large Language Models (MLLMs), yet existingsystems struggle with complex cross-modal reasoning. Flat vector retrievaloften ignores structural dependencies, while current graph-based methodsrely on costly “translation-to-text” pipelines that discard fine-grainedvisual information. To address these limitations, we propose MG2 -RAG,a lightweight Multi-Granularity Graph RAG framework that jointlyimproves graph construction, modality fusion, and cross-modal retrieval.MG2 -RAG constructs a hierarchical multimodal knowledge graph bycombining lightweight textual parsing with entity-driven visual ground-ing, enabling textual entities and visual regions to be fused into unifiedmultimodal nodes that preserve atomic evidence. Building on this repre-sentation, we introduce a multi-granularity graph retrieval mechanismthat aggregates dense similarities and propagates relevance across thegraph to support structured multi-hop reasoning. Extensive experimentsacross four representative multimodal tasks (i.e., retrieval, knowledge-based VQA, reasoning, and classification) demonstrate that MG2 -RAGconsistently achieves state-of-the-art performance while reducing graphconstruction overhead with an average 43.3× speedup and 23.9× costreduction compared with advanced graph-based frameworks. The sourcecode is publicly available at https://github.com/Daboolu/MG2-RAG.

High-precision medical diagnosis relies not only on staticimaging features but also on the implicit diagnostic memory experts in-stantly invoke during image interpretation. We pinpoint a fundamentalcognitive misalignment in medical VLMs caused by discrete tokeniza-tion, leading to quantization loss, long-range information dissipation,and missing case-adaptive expertise. To bridge this gap, we proposeMedSynapse-V, a framework for latent diagnostic memory evolution thatsimulates the experiential invocation of clinicians by dynamically synthe-sizing implicit diagnostic memories within the model’s hidden stream.Specifically, it begins with a Meta Query for Prior Memorizationmechanism, where learnable probes retrieve structured priors from ananatomical prior encoder to generate condensed implicit memories. Toensure clinical fidelity, we introduce Causal Counterfactual Refine-ment (CCR) which leverages reinforcement learning and counterfactualrewards derived from region-level feature masking to quantify the causalcontribution of each memory, thereby pruning redundancies and aligninglatent representations with diagnostic logic. This evolutionary processculminates in Intrinsic Memory Transition (IMT), a privileged-autonomous dual-branch paradigm that internalizes teacher-branch di-agnostic patterns into the student-branch via full-vocabulary divergencealignment. Comprehensive empirical evaluations across multiple datasetsdemonstrate that MedSynapse-V, by transferring external expertise intoendogenous parameters, significantly outperforms existing state-of-the-art methods, particularly Chain-of-Thought (CoT) paradigms, in diag-nostic accuracy and multi-dataset generalization without compromisingthe inference efficiency of standard VLMs.

3D Gaussian Splatting (3DGS) has established itself as apremier technique for high-fidelity radiance field rendering. However, itsapplication in SLAM tasks is frequently hindered by the explosive growthof redundant Gaussian primitives, which imposes prohibitive memoryand rendering overhead. Existing compaction strategies predominantlyrely on heuristic pruning based on importance scores; such methods oftenfail to provide global fidelity guarantees and can disrupt the optimizationcontinuity required for online tracking. To address these challenges, weintroduce Manifold-Aware Spectral Compaction (MASC), a frameworkthat reformulates map reduction as a continuous signal reconstructiontask mapped onto a Riemannian manifold. We first construct a topology-aware graph representation using Symmetrized Kullback-Leibler diver-gence to precisely capture the statistical connectivity between primi-tives. To ensure real-time efficiency, we implement an incremental Nys-tröm spectral embedding strategy to project the Gaussian map onto alow-dimensional spectral subspace. We further derive a mathematicallyrigorous, closed-form solution for primitive aggregation leveraging Bures-Wasserstein barycenters, which theoretically ensures the conservation oflocal radiance field moments. Extensive evaluations on the Replica andTUM RGB-D datasets demonstrate that MASC achieves a 2.8× reduc-tion in GPU memory and a 2.4× speedup in rendering throughput.Remarkably, our method maintains or even enhances tracking accuracywhile eliminating approximately 60% of redundant primitives, achiev-ing an ATE of 0.98 cm on the TUM dataset. These results substantiatethat MASC provides an efficient and principled plug-and-play pathwayfor scalable, long-term neural rendering SLAM.