paperlist.ai

AI conference papers

AI conference paper library

Search papers from 30+ top conferences, including NeurIPS, ICLR, ICML, CVPR, ACL and AAAI. Filter by keyword, then open any paper for its abstract, authors and links.

ECCV 2026

Low-Level Dataset Distillation for Medical Image Enhancement

Fengzhi Xu, Ziyuan Yang, Mengyu Sun, Joey Tianyi Zhou, Yi Zhang

Medical image enhancement is clinically important, but ex-isting methods often rely on large-scale datasets to learn complex image-to-image mappings. Such reliance leads to high training and storage costsand hinders practical deployment.Dataset distillation (DD) provides a promising solution by synthesizingcompact datasets that preserve the training behavior of the original data.However, existing DD methods mainly focus on high-level tasks, wheremultiple samples share a common semantic label and the distilled datacan compress shared representations. In contrast, low-level medical im-age enhancement involves dense image-to-image mappings, where eachsample corresponds to a unique pixel-level target. This fundamental dif-ference makes low-level DD significantly more challenging and inherently

ECCV 2026

Learning Transferable Dynamics Priors from Action to World Modeling

Ze Huang, Zhang Jiahui, Hairuo Liu, Chenxi Zhang, Ran Cheng, Li Zhang

We study action-conditioned world modeling as a scalableway to learn transferable dynamics priors for robot learning. By pre-training a model to predict how actions drive visual scene evolution,the resulting world model captures reusable interaction dynamics be-yond appearance-level video generation. Concretely, we pretrain a multi-view interactive base diffusion world model, A2World, on large-scalerobot manipulation data with real action annotations. We validate thelearned dynamics priors from two complementary perspectives. First,we adapt A2World into a task- or scene-specialized real-world simula-tor, A2World-sim, whose long-horizon rollouts support simulator-basedpolicy evaluation and scalable what-if analysis by replacing real-robotrollouts with world model rollouts. Second, starting from the same pre-trained weights, we adapt A2World into a video-action joint predictionmodel, A2World-policy, that predicts actions under visual and instruc-tion conditioning. Experiments across simulation benchmarks and real-robot settings demonstrate that action-conditioned world model pre-training yields transferable dynamics priors that benefit both simulator-centric and policy-centric robot learning.

ECCV 2026

Learning Probabilistic Embeddings for Unsupervised Action Segmentation

Shuai Li, Duc Vu, Juergen Gall

This paper concerns the problem of unsupervised temporalaction segmentation for long, untrimmed videos. Recent successful ap-proaches follow a joint representation learning and clustering paradigm,where optimal transport (OT) is adopted to produce pseudo labels forlearning frame representations. These approaches alternate between es-timating pseudo labels using OT and optimizing the parameters withgradient descent during training, where OT is used for obtaining the fi-nal temporal action segmentation. A major limitation of these works isthat they learn a deterministic embedding for frame representations. Theiterative procedure between learning deterministic embeddings based onpseudo labels and estimating pseudo labels from the learned embeddingcan thus get quickly stuck in a local optimum. As an alternative, we thuspropose to learn a probabilistic embedding for frame representations. Theembeddings are modeled by Gaussian distributions and we sample fromthe distributions before estimating the pseudo labels. We evaluate ourapproach on several challenging temporal action segmentation datasetsand achieve results comparable to, and in some cases, better than thestate of the art. Compared to baselines with deterministic embeddings,our approach improves MoF up to 20.7% and F1-score up to 19.0%. Ourcode is available at https://github.com/derkbreeze/PEOT.

ECCV 2026

HO-Flow: Generalizable Hand-Object Interaction Generation with Latent Flow Matching

Zerui Chen, Rolandos Alexandros Potamias, Shizhe Chen, Jiankang Deng, Cordelia Schmid, Stefanos Zafeiriou

Generating realistic 3D hand-object interactions (HOI) is afundamental challenge in computer vision and robotics, requiring bothtemporal coherence and high-fidelity physical plausibility. Existing meth-ods remain limited in their ability to learn expressive motion repre-sentations for generation and perform temporal reasoning. In this pa-per, we present HO-Flow, a framework for synthesizing realistic hand-object motion sequences from texts and canonical 3D objects. HO-Flowfirst employs an interaction-aware variational autoencoder to encode se-quences of hand and object motions into a unified latent manifold byincorporating hand and object kinematics, enabling the representationto capture rich interaction dynamics. It then leverages a masked flowmatching model that combines auto-regressive temporal reasoning withcontinuous latent generation, improving temporal coherence. To furtherenhance generalization, HO-Flow predicts object motions relative to theinitial frame, enabling effective pre-training on large-scale synthetic data.Experiments on the GRAB, OakInk, and DexYCB benchmarks demon-strate that HO-Flow achieves state-of-the-art performance in both phys-ical plausibility and motion diversity for interaction motion synthesis.

ECCV 2026

HERO: Enhancing Multimodal Faithfulness via Dynamic Entropy-Aware Reinforcement Learning

Xiongfeng Yang, Qingan Zhang, Yuheng Zhang, Kun-Yu Lin, Jian-Fang Hu, Dongmei Jiang, Weishi Zheng

Chain-of-Thought fine-tuning enhances the reasoning capabilities of Large Vision-Language Models, but can paradoxically exacerbate hallucinations, revealing a reasoning-faithfulness trade-off. We identify a key cause of this phenomenon as the “Confidence Trap”: under visual ambiguity or degradation, models may produce low-entropy, high-confidence errors instead of expressing appropriate uncertainty. This failure mode challenges conventional mitigation strategies that rely on uncertainty cues or contrastive decoding signals, since confident hallucinations can remain highly stable under visual perturbations. To address this problem, we propose Hallucination-Entropy Regulated Optimization (HERO), an on-policy reinforcement learning framework for mitigating confident hallucinations. HERO introduces a dynamic entropy-aware objective that assigns larger optimization weights to low-entropy erroneous outputs, and further improves training efficiency through variance-gated sample selection that focuses updates on informative hard negatives. Experiments on POPE, THRONE, and AMBER show that HERO achieves strong overall hallucination mitigation, substantially improves truthfulness in critical low-entropy regions, and maintains competitive general reasoning capabilities. These results suggest that explicitly targeting confidence-misaligned errors is an effective direction for improving multimodal faithfulness in CoT-style LVLMs.

ECCV 2026

Learning Consistency in Reward Modeling for Multi-Modal Reasoning

Chen Li, Bolin Ni, Boxin Zhang, Ke Ye, Jinnian Zhang, Houwen Peng, Han Hu, Nanning Zheng

Reliable reward system is essential for reinforcement learn-ing in multi-modal reasoning, yet existing methods face key limitations:preference-based models often misjudge correctness on complex tasks,while rule-based approaches struggle with semantically equivalent an-swers and therefore restrict the scale of usable RL data. To address thesechallenges, we propose Consistency Reward Models (CRM), which assesshow well a candidate response aligns semantically and mathematicallywith ground truth. Our model produces a continuous reward derived fromthe logits associated with “Consistent” vs. “Inconsistent”, enabling moreinformative and stable optimization signals than binary rewards duringRL. By design, CRM compares answers in text space, which our ablationshows is both sufficient and more accurate for consistency judgment. Wealso find that incorporating Chain-of-Thought (CoT) reasoning improvesthe reliability of these consistency assessments. As a judging model, itachieves an 77.8% win rate over VLMEvalKit, demonstrating strongerjudging accuracy. When used for RL, the approach enables a 7B pol-icy model to attain state-of-the-art performance on MathVista (75.2%)and MathVision (30.6%), outperforming rule-based systems, while sub-stantially increasing the scale of RL-usable data. The method furthergeneralizes to OCR, MMMU and open tasks and supports stable RLtraining up to 72B model. These results show consistency reward modelsoffer a more scalable and robust foundation for multi-modal reasoningRL.

ECCV 2026

SiGMA: Sign-Guided Merging and Adaptation framework for Multimodal Continual Instruction Tuning

Keonhee Park, Gunhee Kim

Multimodal Continual Instruction Tuning (MCIT) is crucial for adapting Multimodal Large Language Models (MLLMs) to evolving a sequence of downstream tasks. Prior methods mostly utilize Mixtureof-Experts or expansion–merge approach, primarily focusing on catastrophic forgetting, yet they still suffer from negative interference during inference, where newly learned updates overwrite useful prior knowledge and degrade overall performance. To address this, we propose SiGMA (Sign-Guided Merging and Adaptation), a simple yet effective framework that mitigates negative interference with two components: signguided adaptive tuning during training and sign-guided merging at inference. Sign-guided adaptive tuning reduces collisions with past knowledge and learns the current task with minimal drift, mitigating severe forgetting. Sign-guided merging further improves consolidation by selectively scaling salient parameters to preserve and amplify useful taskspecific knowledge. Experiments on UCIT and DCL benchmarks show that SiGMA significantly reduces negative interference and outperforms state-of-the-art MCIT methods. Our code is available at SiGMA.

ECCV 2026

GIDE: Unlocking Diffusion LLMs for Precise Training-Free Image Editing

Zifeng Zhu, Jiaming Han, Jiaxiang Zhao, Minnan Luo, Xiangyu Yue

While Diffusion Large Language Models (DLLMs) have demon-strated remarkable capabilities in multi-modal generation, performingprecise, training-free image editing remains an open challenge. Unlikecontinuous diffusion models, the discrete tokenization inherent in DLLMshinders the application of standard noise inversion techniques, often lead-ing to structural degradation during editing. In this paper, we introduceGIDE (Grounded Inversion for DLLM Image Editing), a unified frame-work designed to bridge this gap. GIDE incorporates a novel DiscreteNoise Inversion mechanism that accurately captures latent noise pat-terns within the discrete token space, ensuring high-fidelity reconstruc-tion. We then decompose the editing pipeline into grounding, inver-sion, and refinement stages. This design enables GIDE supportingvarious editing instructions (text, point and box) and operations whilestrictly preserving the unedited background. Furthermore, to overcomethe limitations of existing single-step evaluation protocols, we introduceGIDE-Bench, a rigorous benchmark comprising 805 compositional edit-ing scenarios guided by diverse multi-modal inputs. Extensive exper-iments on GIDE-Bench demonstrate that GIDE significantly outper-forms prior training-free methods, improving Semantic Correctness by51.83% and Perceptual Quality by 50.39%. Additional evaluations onImgEdit-Bench confirm its broad applicability, demonstrating consistentgains over trained baselines and yielding photorealistic consistency onpar with leading models.3 .

ECCV 2026

Geometric Gradient Rectification for Safe Open-Set Semi-Supervised Learning

Jiahe Chen, Qian Shao, Qiyuan Chen, Jiaying He, Jintai Chen, Hongxia Xu, Jian Wu

Open-set semi-supervised learning aims to leverage unlabeleddata that may contain out-of-distribution outliers while maintainingperformance on in-distribution classes. Existing methods mainly followtwo paradigms: filtering suspicious samples or incorporating unlabeledobjectives with soft weighting. We argue that both face a common trade-off: aggressive filtering can discard informative but hard ID samples,whereas utilization can introduce auxiliary gradients that conflict withsupervised learning when pseudo labels are wrong. We therefore shiftthe focus from sample selection to gradient-level control. We proposeGeometric Gradient Rectification (GGR), a plug-in framework that usesthe supervised gradient as an anchor and projects conflicting auxiliarygradients onto an admissible region in gradient space. This makes theapplied auxiliary update first-order non-opposing within the rectifiedcoordinate block while preserving orthogonal components that may stillcarry useful representation signals. We further extend GGR with subspace-aware rectification to stabilize the anchor under noisy mini-batch gradients.Experiments on CIFAR and ImageNet benchmarks show that GGRimproves representative OSSL baselines in most settings and yields gainsin both closed-set generalization and open-set robustness. Code will beavailable at https://github.com/JiaheChen2002/GGR.

ECCV 2026

ZAP: Zero-Shot Assembly Planning with Large Language Models

Linpeng Peng, Yanbo Wang, Chuanjie Lv, Wencan Jiang, Liming Xu, Jianbiao Mei, Xinyue Yao, Yong Liu

Achieving general-purpose robotic assembly is a longstand-ing goal in AI. Early data-driven methods, particularly those based onReinforcement Learning (RL), made progress but are fundamentally lim-ited by poor sample efficiency and struggle to generalize to unseen ob-jects in zero-shot scenarios. To overcome these issues, more structuredapproaches have emerged. Classical planners like ASAP explicitly enforcegeometric feasibility but require precise, pre-existing geometric models,which can hinder deployment under real-world perception noise. Con-versely, recent Vision-Language Model (VLM)-based systems like Man-ual2Skill excel at semantic understanding but are critically dependenton pre-authored manuals, restricting their autonomy in unstructuredenvironments. To address both planning-time model dependency andmanual dependency, we introduce ZAP, a novel framework for zero-shotVLM-guided assembly planning with geometry-based simulation verifi-cation. ZAP constructs an internal structural representation from per-ception without relying on task-specific manuals, pre-authored assemblysequences, or hard-coded assembly logic. Our framework features twocore modules: a VLM-Parser that analyzes segmentation-based multi-view RGB-D observations to infer an “implicit manual” and compile itinto structured part dossiers, and a VLM-Planner that performs Chain-of-Thought (CoT) reasoning over these dossiers to produce physicallyplausible, robot-executable assembly sequences. We validate candidateplans using a geometry-based digital-twin verifier and further exam-ine representative executions on a robotic arm. Our experiments showthat ZAP remains effective in manual-free scenarios where methods likeManual2Skill are less directly applicable. Furthermore, by reasoning di-rectly from visual perception, ZAP can be more flexible than geometry-dependent classical planners like ASAP in the evaluated settings. Theframework generates and executes plans for complex items such as multi-part LEGO models and household furniture in our evaluation setting,providing evidence toward more autonomous assembly without task-specific manuals.

ECCV 2026

Gaussians on Fire: High-Frequency Reconstruction of Flames

Jakob Nazarenus, Dominik Michels, Wojciech Palubicki, Simin Kou, Fang-Lue Zhang, Soren Pirk, Reinhard Koch

We propose a method to reconstruct dynamic fire in 3D froma limited set of camera views with a Gaussian-based spatiotemporal rep-resentation. Capturing and reconstructing fire and its dynamics is highlychallenging due to its volatile nature, transparent quality, and multitudeof high-frequency features. Despite these challenges, we aim to recon-struct fire from only three views, which consequently requires solvingfor under-constrained geometry. We solve this by separating the staticbackground from the dynamic fire region by combining dense multi-viewstereo images with monocular depth priors. The fire is initialized as a 3Dflow field, obtained by fusing per-view dense optical flow projections. Tocapture the high-frequency features of fire, each 3D Gaussian encodesa lifetime and linear velocity to match the dense optical flow. To en-sure sub-frame temporal alignment across cameras, we employ a customhardware synchronization pattern – allowing us to reconstruct fire withaffordable commodity hardware. Our quantitative and qualitative vali-dations across numerous reconstruction experiments demonstrate robustperformance for diverse and challenging real and simulated fire scenarios.

ECCV 2026

From Open Loop to Closed Loop: A Test-Time Iterative Optimization Framework for Reference-Consistent Image Generation

Baixuan Zhao, Xinyu Zhang, Shuaicheng Liu, Xiongkuo Min, Guangtao Zhai, Xiaohong Liu

While controllable image generation has made significantstrides by incorporating visual reference conditions, existing methodspredominantly operate as open-loop systems. They inject control sig-nals in a strictly feed-forward manner, failing to guarantee strict fidelityto the reference due to the absence of active feedback and error cor-rection mechanisms. To address this fundamental limitation, we pro-pose a novel test-time iterative optimization framework that reformu-lates reference-consistent generation as a closed-loop dynamic trackingproblem. By treating the pre-trained generative model as a control plant,our framework employs a sensor-controller architecture driven by a mod-ified Proportional-Integral-Derivative (PID) algorithm. This mechanismiteratively optimizes the latent control signals at test time based on thesensed discrepancy between the generated output and the reference tar-get. Notably, this approach is entirely training-free, model-agnostic, andintegrates seamlessly around existing diffusion pipelines. Extensive eval-uations across ID-preserving, pose-controlled, and depth-controlled gen-eration tasks validate the universality of our method. Empirical resultsdemonstrate improvements over computation-matched open-loop base-lines, achieving relative performance gains of up to 25.36% for facialsimilarity, alongside spatial error reductions of up to 27.71% for posealignment and 28.50% for depth consistency. More broadly, this workoffers a new conceptual perspective: it demonstrates that controllablegeneration can be effectively managed as a dynamic feedback system,bringing the rigorous principles of classical control theory into the op-timization of generative models. Code is available at https://github.com/zzdrill/From-Open-Loop-to-Closed-Loop.

ECCV 2026

Towards in-the-wild Egocentric 3D Hand-Object Pose Estimation

Siddhant Bansal, Zhifan Zhu, Shashank Tripathi, Jiahe Zhao, Michael Black, Dima Damen

Estimating accurate 3D hand–object pose from in-the-wildegocentric RGB remains challenging due to severe occlusions and am-biguous contact. Existing learning-based methods often struggle to gen-eralise to in-the-wild scenes and are limited by the scarcity of super-vision. We address these issues with two contributions. First, we in-troduce EPIC-Contact, an in-the-wild egocentric dataset of 2.3K clips(62.3K frames) with dense, bijective 3D hand–object contact correspon-dences and posed meshes. Second, we propose HOPformer, an end-to-end transformer that jointly predicts bi-manual hand and object posein a single forward pass. A cross-attention decoder conditions objectfeatures on hand priors, producing robust pose estimation. We testHOPformer on the in-lab 3D dataset, ARCTIC, as well as our newlyintroduced EPIC-Contact dataset. HOPformer reaches 82.4% successrate on ARCTIC (+6.2 pts over current SOTA). On EPIC-Contact,it nearly doubles the success rate while reducing contact deviation by75%. EPIC-Contact, HOPformer code and checkpoints are released:https://sid2697.github.io/epic-contact.

ECCV 2026

Free-Range Gaussians: Non-Grid-Aligned Generative 3D Gaussian Reconstruction

Akhmedkhan Shabanov, Peter Hedman, Ethan Weber, Zhengqin Li, Denys Rozumnyi, Gael Le Lan, Naina Dhingra, Lei Luo, Andrea Vedaldi, Christian Richardt, Andrea Tagliasacchi, Bo Zhu, Numair Khan

We present Free-Range Gaussians, a multi-view reconstruc-tion method that predicts non-pixel, non-voxel-aligned 3D Gaussiansfrom as few as four images. This is done through flow matching overGaussian parameters. Our generative formulation of reconstruction allowsthe model to be supervised with non-grid-aligned 3D data, and enables itto synthesize plausible content in unobserved regions. Thus, it improveson prior methods that produce highly redundant grid-aligned Gaussians,and suffer from holes or blurry conditional means in unobserved regions.To handle the number of Gaussians needed for high-quality results, weintroduce a hierarchical patching scheme to group spatially related Gaus-sians into joint transformer tokens, halving the sequence length whilepreserving structure. We further propose a timestep-weighted renderingloss during training, and photometric gradient guidance and classifier-freeguidance at inference to improve fidelity. Experiments on Objaverse andGoogle Scanned Objects show consistent improvements over pixel andvoxel-aligned methods while using significantly fewer Gaussians, withlarge gains when input views leave parts of the object unobserved.

ECCV 2026

Frames2Residual: Spatiotemporal Decoupling for Self-Supervised Video Denoising

Mingjie Ji, Zhan Shi, Kailai Zhou, Zixuan Fu, Xun Cao

Self-supervised video denoising methods typically extend image-based frameworks into the temporal dimension, yet they often struggleto integrate inter-frame temporal consistency with intra-frame spatialspecificity. Existing Video Blind-Spot Networks (BSNs) require noise in-dependence by masking the center pixel, this constraint prevents the useof spatial evidence for texture recovery, thereby severing spatiotempo-ral correlations and causing texture loss. To address this, we proposeFrames2Residual (F2R), a spatiotemporal decoupling framework thatexplicitly divides self-supervised training into two distinct stages: blindtemporal consistency modeling and non-blind spatial texture recovery.In Stage 1, a blind temporal estimator learns inter-frame consistencyusing a frame-wise blind strategy, producing a temporally consistent an-chor. In Stage 2, a non-blind spatial refiner leverages this anchor to safelyreintroduce the center frame and recover intra-frame high-frequency spa-tial residuals while preserving temporal stability. Extensive experimentsdemonstrate that our decoupling strategy allows F2R to outperform ex-isting self-supervised methods on both sRGB and raw video benchmarks.Code is available at https://github.com/m1NGGi/F2R.

ECCV 2026

FlexiBrain: Resolution-Agnostic Voxel-Level Encoding for Native fMRI

Mo wang, Wenhao Ye, Junfeng Xia, Minghao Xu, Hongkai Wen, Quanying Liu

The success of large-scale deep learning models in neuro-science is fundamentally constrained by severe data heterogeneity. NativefMRI data aggregated from diverse sources exhibit substantial variationin both spatial and temporal resolutions. Consequently, most existingframeworks rely on lengthy, rigid preprocessing pipelines that enforceuniformity across datasets. This practice introduces two critical limita-tions: (1) potential degradation of subject-speci_x001C_c anatomical informa-tion; (2) signi_x001C_cant computational overhead, often requiring hours of pro-cessing per subject. Here, we propose FlexiBrain, a resolution-agnosticvoxel-level encoding framework for native fMRI based on Mamba-JEPA.FlexiBrain de_x001C_nes patch sizes in real-world physical units and employs adynamic patch resizing, thereby bypassing destructive spatial standard-ization while enabling direct ingestion of data in native space. We instan-tiate the framework using an e_x001E_cient Mamba-JEPA backbone to modelhigh-dimensional 4D fMRI signals. Across _x001C_ve diverse downstream neuro-science tasks, FlexiBrain consistently outperforms recent state-of-the-artmethods, achieving gains of up to 12 percentage points without exter-nal data augmentation. Importantly, FlexiBrain functions as a seamlessplug-in module, substantially reducing preprocessing costs and accelerat-ing the development of robust voxel-level fMRI foundation models. Codeis available at https://github.com/OneMore1/FlexiBrain.

ECCV 2026

EvoWorld: A World-Model-Centric Framework for Continuous Self-Evolution of Modular Embodied Skills

Boshi Zhang, Sen Cui, Baohua Yin, Youyi Kou, Junyu Wu, Zuo Pu, Tao Xue, Zhikang Chen, Shanshan Wei, Min Zhang, Miao Liu, Changshui Zhang, Zhang Tao

Current progress in physical intelligence largely relies onscaling monolithic Vision-Language-Action (VLA) models, yet real-worldpolicy data remain fragmented across scenes and tasks. This mismatchlimits transfer, exacerbates catastrophic forgetting, and impedes contin-ual improvement. A modular design that shares dynamics while special-izing skills is therefore a promising paradigm.We introduce EvoWorld (EvoW), a world-model-centric frameworkfor skill orchestration and iterative self-evolution. In EvoW, VLAs forman expandable library of pluggable experts. A high-level router selectsexperts conditioned on scene and task, while an action-conditioned videoworld model provides a shared dynamics prior for rollout-based planning.The world model provides counterfactual rollouts to score candidate ex-perts, while selected experts execute in the grounded scene to generatetrajectories for verification. A vision-language evaluator delivers seman-tic scoring and diagnostic tags, enabling targeted updates to the worldmodel, router memory, or specific experts rather than global retraining.This closes an automated loop that jointly improves grounding, routing,and skill refinement without manual task engineering. Experiments showthat EvoW enables automated task-to-policy synthesis with competi-tive success rates and consistent iterative gains in the evaluated settings,while producing valid and diverse trajectories that support evaluationand skill refinement.

ECCV 2026

EventVGGT: Exploring Cross-Modal Distillation for Consistent Event-based Depth Estimation

Yinrui Ren, Jinjing Zhu, Kanghao Chen, Zhuoxiao Li, Jing OU, Zidong Cao, Tongyan Hua, Peilun Shi, Yingchun Fu, Wufan Zhao, Hui Xiong

Event cameras offer superior sensitivity to high-speed motionand extreme lighting, making event-based monocular depth estimation apromising approach for robust 3D perception in challenging conditions.However, progress is severely hindered by the scarcity of dense depthannotations. While recent annotation-free approaches mitigate this bydistilling knowledge from Vision Foundation Models (VFMs), a criticallimitation persists: they process event streams as independent frames. Byneglecting the inherent temporal continuity of event data, these meth-ods fail to leverage the rich temporal priors encoded in VFMs, ultimatelyyielding temporally inconsistent and less accurate depth predictions. Toaddress this, we introduce EventVGGT, a novel framework that ex-plicitly models the event stream as a coherent video sequence. To thebest of our knowledge, we are the first to distill spatio-temporal andmulti-view geometric priors from the Visual Geometry Grounded Trans-former (VGGT) into the event domain. We achieve this via a comprehen-sive tri-level distillation strategy: (i) Cross-Modal Feature Mixture(CMFM) bridges the modality gap at the output level by fusing RGBand event features to generate auxiliary depth predictions; (ii) Spatio-Temporal Feature Distillation (STFD) distills VGGT’s powerfulspatio-temporal representations at the feature level; and (iii) Tempo-ral Consistency Distillation (TCD) enforces cross-frame coherenceat the temporal level by aligning inter-frame depth changes. Extensiveexperiments demonstrate that EventVGGT consistently outperforms ex-isting methods—reducing the absolute mean depth error at 30m by over53% on EventScape (from 2.30 to 1.06)—while exhibiting robust zero-shot generalization on the unseen DENSE and MVSEC datasets. Thecode is available at https://github.com/yinruiRen/EventVGGT.

ECCV 2026

EmbodiedVAE: Disentangled Video VAE for Efficient and Controllable Embodied Manipulation

Jiayi Luo, Hanxin Zhu, Chen Gao, Jiankun Wang, Cong Wang, Tianyu He, Jianxin Li, Zhibo Chen

Latent diffusion models (LDMs) have recently significantlyadvanced embodied learning in constructing powerful embodied manipu-lation world models. However, despite the remarkable performance, exist-ing LDMs predominantly rely on Variational Autoencoders (VAEs) opti-mized for natural scenes while failing to account for the unique character-istics of embodied manipulation scenarios, yielding latent representationsthat are neither compact nor controllable, thereby hindering efficienttraining of LDMs and precise robotic control. To solve this problem, wepresent EmbodiedVAE, a novel video VAE that provides compact yetcontrollable latent representations tailored for the robotic manipulationworld models. Specifically, EmbodiedVAE adopts a dual-encoder, single-decoder architecture with an asymmetric spatio-temporal compressionmodule, which automatically disentangles the robot arm’s motion frombackground environment, resulting in overall compactness while provid-ing explicit embodied latent to support fine-grained action control. Tofurther preserve the temporal consistency of learned robotic motion la-tent, we introduce an optimal-transport-based consistency module thatexplicitly enforces motion fidelity and inter-frame coherence. Extensiveexperiments demonstrate that our proposed EmbodiedVAE achieves su-perior reconstruction quality with high compression rate, while enablingmore precise action control in robotic manipulation scenarios with anaverage of 2dB PSNR improvement over state-of-the-art video VAEs.Code is available at: https://github.com/Mutual-Luo/EmbodiedVAE

ECCV 2026

Dynamic-Robust Photometric–Semantic Reconstruction for Open-Vocabulary 3D Scene Understanding

Boyu Cai, Li Yang, Yan Xu, Wei Liu, Nian Liu, Sikui Zhang, Yan Wang, Chunfeng Yuan, Weiming Hu

The integration of novel view synthesis (NVS) and open-vocabulary segmentation (OVS) has recently yielded powerful feed-forward3D foundation models. However, their inherent reliance on static-sceneassumptions leads to severe misalignment of spatial features in uncon-strained dynamic environments. To bridge this critical gap, we proposeSPAR, a novel joint semantic-geometric encoding architecture that ex-plicitly isolates transient dynamic noise prior to latent space aggregation.Furthermore, we introduce a dynamic-region-aware end-to-end trainingparadigm that structurally couples motion estimation with multi-viewvisual and semantic learning. This unified approach enables the networkto inherently resolve motion conflicts and distill multi-view consistent,temporally stable scene representations from dynamic inputs. Extensiveexperiments on the challenging D-RE10K benchmark demonstrate thatSPAR achieves state-of-the-art performance. Our end-to-end approachachieves exceptional novel view synthesis quality, yielding a PSNR of22.15 dB and 23.33 dB given only 3 and 4 input views respectively. De-spite being trained in a self-supervised manner, our model achieves anmIoU of 88.5% for motion mask prediction. Furthermore, our analysisreveals a strong inter-task synergy between photometric scene recon-struction and semantic understanding, where semantic synthesis learningconsistently enhances photometric fidelity in novel view rendering. Codewill be available at https://github.com/dmucby/SPAR.

Keep browsing

More papers from top AI conferences

30+ top conferences, by conference, year, and topic.