ECCV 2026 论文列表

Multimodal Continual Instruction Tuning (MCIT) is crucial for adapting Multimodal Large Language Models (MLLMs) to evolving a sequence of downstream tasks. Prior methods mostly utilize Mixtureof-Experts or expansion–merge approach, primarily focusing on catastrophic forgetting, yet they still suffer from negative interference during inference, where newly learned updates overwrite useful prior knowledge and degrade overall performance. To address this, we propose SiGMA (Sign-Guided Merging and Adaptation), a simple yet effective framework that mitigates negative interference with two components: signguided adaptive tuning during training and sign-guided merging at inference. Sign-guided adaptive tuning reduces collisions with past knowledge and learns the current task with minimal drift, mitigating severe forgetting. Sign-guided merging further improves consolidation by selectively scaling salient parameters to preserve and amplify useful taskspecific knowledge. Experiments on UCIT and DCL benchmarks show that SiGMA significantly reduces negative interference and outperforms state-of-the-art MCIT methods. Our code is available at SiGMA.

While Diffusion Large Language Models (DLLMs) have demon-strated remarkable capabilities in multi-modal generation, performingprecise, training-free image editing remains an open challenge. Unlikecontinuous diffusion models, the discrete tokenization inherent in DLLMshinders the application of standard noise inversion techniques, often lead-ing to structural degradation during editing. In this paper, we introduceGIDE (Grounded Inversion for DLLM Image Editing), a unified frame-work designed to bridge this gap. GIDE incorporates a novel DiscreteNoise Inversion mechanism that accurately captures latent noise pat-terns within the discrete token space, ensuring high-fidelity reconstruc-tion. We then decompose the editing pipeline into grounding, inver-sion, and refinement stages. This design enables GIDE supportingvarious editing instructions (text, point and box) and operations whilestrictly preserving the unedited background. Furthermore, to overcomethe limitations of existing single-step evaluation protocols, we introduceGIDE-Bench, a rigorous benchmark comprising 805 compositional edit-ing scenarios guided by diverse multi-modal inputs. Extensive exper-iments on GIDE-Bench demonstrate that GIDE significantly outper-forms prior training-free methods, improving Semantic Correctness by51.83% and Perceptual Quality by 50.39%. Additional evaluations onImgEdit-Bench confirm its broad applicability, demonstrating consistentgains over trained baselines and yielding photorealistic consistency onpar with leading models.3 .

Open-set semi-supervised learning aims to leverage unlabeleddata that may contain out-of-distribution outliers while maintainingperformance on in-distribution classes. Existing methods mainly followtwo paradigms: filtering suspicious samples or incorporating unlabeledobjectives with soft weighting. We argue that both face a common trade-off: aggressive filtering can discard informative but hard ID samples,whereas utilization can introduce auxiliary gradients that conflict withsupervised learning when pseudo labels are wrong. We therefore shiftthe focus from sample selection to gradient-level control. We proposeGeometric Gradient Rectification (GGR), a plug-in framework that usesthe supervised gradient as an anchor and projects conflicting auxiliarygradients onto an admissible region in gradient space. This makes theapplied auxiliary update first-order non-opposing within the rectifiedcoordinate block while preserving orthogonal components that may stillcarry useful representation signals. We further extend GGR with subspace-aware rectification to stabilize the anchor under noisy mini-batch gradients.Experiments on CIFAR and ImageNet benchmarks show that GGRimproves representative OSSL baselines in most settings and yields gainsin both closed-set generalization and open-set robustness. Code will beavailable at https://github.com/JiaheChen2002/GGR.

Achieving general-purpose robotic assembly is a longstand-ing goal in AI. Early data-driven methods, particularly those based onReinforcement Learning (RL), made progress but are fundamentally lim-ited by poor sample efficiency and struggle to generalize to unseen ob-jects in zero-shot scenarios. To overcome these issues, more structuredapproaches have emerged. Classical planners like ASAP explicitly enforcegeometric feasibility but require precise, pre-existing geometric models,which can hinder deployment under real-world perception noise. Con-versely, recent Vision-Language Model (VLM)-based systems like Man-ual2Skill excel at semantic understanding but are critically dependenton pre-authored manuals, restricting their autonomy in unstructuredenvironments. To address both planning-time model dependency andmanual dependency, we introduce ZAP, a novel framework for zero-shotVLM-guided assembly planning with geometry-based simulation verifi-cation. ZAP constructs an internal structural representation from per-ception without relying on task-specific manuals, pre-authored assemblysequences, or hard-coded assembly logic. Our framework features twocore modules: a VLM-Parser that analyzes segmentation-based multi-view RGB-D observations to infer an “implicit manual” and compile itinto structured part dossiers, and a VLM-Planner that performs Chain-of-Thought (CoT) reasoning over these dossiers to produce physicallyplausible, robot-executable assembly sequences. We validate candidateplans using a geometry-based digital-twin verifier and further exam-ine representative executions on a robotic arm. Our experiments showthat ZAP remains effective in manual-free scenarios where methods likeManual2Skill are less directly applicable. Furthermore, by reasoning di-rectly from visual perception, ZAP can be more flexible than geometry-dependent classical planners like ASAP in the evaluated settings. Theframework generates and executes plans for complex items such as multi-part LEGO models and household furniture in our evaluation setting,providing evidence toward more autonomous assembly without task-specific manuals.

We propose a method to reconstruct dynamic fire in 3D froma limited set of camera views with a Gaussian-based spatiotemporal rep-resentation. Capturing and reconstructing fire and its dynamics is highlychallenging due to its volatile nature, transparent quality, and multitudeof high-frequency features. Despite these challenges, we aim to recon-struct fire from only three views, which consequently requires solvingfor under-constrained geometry. We solve this by separating the staticbackground from the dynamic fire region by combining dense multi-viewstereo images with monocular depth priors. The fire is initialized as a 3Dflow field, obtained by fusing per-view dense optical flow projections. Tocapture the high-frequency features of fire, each 3D Gaussian encodesa lifetime and linear velocity to match the dense optical flow. To en-sure sub-frame temporal alignment across cameras, we employ a customhardware synchronization pattern – allowing us to reconstruct fire withaffordable commodity hardware. Our quantitative and qualitative vali-dations across numerous reconstruction experiments demonstrate robustperformance for diverse and challenging real and simulated fire scenarios.

While controllable image generation has made significantstrides by incorporating visual reference conditions, existing methodspredominantly operate as open-loop systems. They inject control sig-nals in a strictly feed-forward manner, failing to guarantee strict fidelityto the reference due to the absence of active feedback and error cor-rection mechanisms. To address this fundamental limitation, we pro-pose a novel test-time iterative optimization framework that reformu-lates reference-consistent generation as a closed-loop dynamic trackingproblem. By treating the pre-trained generative model as a control plant,our framework employs a sensor-controller architecture driven by a mod-ified Proportional-Integral-Derivative (PID) algorithm. This mechanismiteratively optimizes the latent control signals at test time based on thesensed discrepancy between the generated output and the reference tar-get. Notably, this approach is entirely training-free, model-agnostic, andintegrates seamlessly around existing diffusion pipelines. Extensive eval-uations across ID-preserving, pose-controlled, and depth-controlled gen-eration tasks validate the universality of our method. Empirical resultsdemonstrate improvements over computation-matched open-loop base-lines, achieving relative performance gains of up to 25.36% for facialsimilarity, alongside spatial error reductions of up to 27.71% for posealignment and 28.50% for depth consistency. More broadly, this workoffers a new conceptual perspective: it demonstrates that controllablegeneration can be effectively managed as a dynamic feedback system,bringing the rigorous principles of classical control theory into the op-timization of generative models. Code is available at https://github.com/zzdrill/From-Open-Loop-to-Closed-Loop.

Estimating accurate 3D hand–object pose from in-the-wildegocentric RGB remains challenging due to severe occlusions and am-biguous contact. Existing learning-based methods often struggle to gen-eralise to in-the-wild scenes and are limited by the scarcity of super-vision. We address these issues with two contributions. First, we in-troduce EPIC-Contact, an in-the-wild egocentric dataset of 2.3K clips(62.3K frames) with dense, bijective 3D hand–object contact correspon-dences and posed meshes. Second, we propose HOPformer, an end-to-end transformer that jointly predicts bi-manual hand and object posein a single forward pass. A cross-attention decoder conditions objectfeatures on hand priors, producing robust pose estimation. We testHOPformer on the in-lab 3D dataset, ARCTIC, as well as our newlyintroduced EPIC-Contact dataset. HOPformer reaches 82.4% successrate on ARCTIC (+6.2 pts over current SOTA). On EPIC-Contact,it nearly doubles the success rate while reducing contact deviation by75%. EPIC-Contact, HOPformer code and checkpoints are released:https://sid2697.github.io/epic-contact.

We present Free-Range Gaussians, a multi-view reconstruc-tion method that predicts non-pixel, non-voxel-aligned 3D Gaussiansfrom as few as four images. This is done through flow matching overGaussian parameters. Our generative formulation of reconstruction allowsthe model to be supervised with non-grid-aligned 3D data, and enables itto synthesize plausible content in unobserved regions. Thus, it improveson prior methods that produce highly redundant grid-aligned Gaussians,and suffer from holes or blurry conditional means in unobserved regions.To handle the number of Gaussians needed for high-quality results, weintroduce a hierarchical patching scheme to group spatially related Gaus-sians into joint transformer tokens, halving the sequence length whilepreserving structure. We further propose a timestep-weighted renderingloss during training, and photometric gradient guidance and classifier-freeguidance at inference to improve fidelity. Experiments on Objaverse andGoogle Scanned Objects show consistent improvements over pixel andvoxel-aligned methods while using significantly fewer Gaussians, withlarge gains when input views leave parts of the object unobserved.

Self-supervised video denoising methods typically extend image-based frameworks into the temporal dimension, yet they often struggleto integrate inter-frame temporal consistency with intra-frame spatialspecificity. Existing Video Blind-Spot Networks (BSNs) require noise in-dependence by masking the center pixel, this constraint prevents the useof spatial evidence for texture recovery, thereby severing spatiotempo-ral correlations and causing texture loss. To address this, we proposeFrames2Residual (F2R), a spatiotemporal decoupling framework thatexplicitly divides self-supervised training into two distinct stages: blindtemporal consistency modeling and non-blind spatial texture recovery.In Stage 1, a blind temporal estimator learns inter-frame consistencyusing a frame-wise blind strategy, producing a temporally consistent an-chor. In Stage 2, a non-blind spatial refiner leverages this anchor to safelyreintroduce the center frame and recover intra-frame high-frequency spa-tial residuals while preserving temporal stability. Extensive experimentsdemonstrate that our decoupling strategy allows F2R to outperform ex-isting self-supervised methods on both sRGB and raw video benchmarks.Code is available at https://github.com/m1NGGi/F2R.

The success of large-scale deep learning models in neuro-science is fundamentally constrained by severe data heterogeneity. NativefMRI data aggregated from diverse sources exhibit substantial variationin both spatial and temporal resolutions. Consequently, most existingframeworks rely on lengthy, rigid preprocessing pipelines that enforceuniformity across datasets. This practice introduces two critical limita-tions: (1) potential degradation of subject-speci_x001C_c anatomical informa-tion; (2) signi_x001C_cant computational overhead, often requiring hours of pro-cessing per subject. Here, we propose FlexiBrain, a resolution-agnosticvoxel-level encoding framework for native fMRI based on Mamba-JEPA.FlexiBrain de_x001C_nes patch sizes in real-world physical units and employs adynamic patch resizing, thereby bypassing destructive spatial standard-ization while enabling direct ingestion of data in native space. We instan-tiate the framework using an e_x001E_cient Mamba-JEPA backbone to modelhigh-dimensional 4D fMRI signals. Across _x001C_ve diverse downstream neuro-science tasks, FlexiBrain consistently outperforms recent state-of-the-artmethods, achieving gains of up to 12 percentage points without exter-nal data augmentation. Importantly, FlexiBrain functions as a seamlessplug-in module, substantially reducing preprocessing costs and accelerat-ing the development of robust voxel-level fMRI foundation models. Codeis available at https://github.com/OneMore1/FlexiBrain.

Current progress in physical intelligence largely relies onscaling monolithic Vision-Language-Action (VLA) models, yet real-worldpolicy data remain fragmented across scenes and tasks. This mismatchlimits transfer, exacerbates catastrophic forgetting, and impedes contin-ual improvement. A modular design that shares dynamics while special-izing skills is therefore a promising paradigm.We introduce EvoWorld (EvoW), a world-model-centric frameworkfor skill orchestration and iterative self-evolution. In EvoW, VLAs forman expandable library of pluggable experts. A high-level router selectsexperts conditioned on scene and task, while an action-conditioned videoworld model provides a shared dynamics prior for rollout-based planning.The world model provides counterfactual rollouts to score candidate ex-perts, while selected experts execute in the grounded scene to generatetrajectories for verification. A vision-language evaluator delivers seman-tic scoring and diagnostic tags, enabling targeted updates to the worldmodel, router memory, or specific experts rather than global retraining.This closes an automated loop that jointly improves grounding, routing,and skill refinement without manual task engineering. Experiments showthat EvoW enables automated task-to-policy synthesis with competi-tive success rates and consistent iterative gains in the evaluated settings,while producing valid and diverse trajectories that support evaluationand skill refinement.

Event cameras offer superior sensitivity to high-speed motionand extreme lighting, making event-based monocular depth estimation apromising approach for robust 3D perception in challenging conditions.However, progress is severely hindered by the scarcity of dense depthannotations. While recent annotation-free approaches mitigate this bydistilling knowledge from Vision Foundation Models (VFMs), a criticallimitation persists: they process event streams as independent frames. Byneglecting the inherent temporal continuity of event data, these meth-ods fail to leverage the rich temporal priors encoded in VFMs, ultimatelyyielding temporally inconsistent and less accurate depth predictions. Toaddress this, we introduce EventVGGT, a novel framework that ex-plicitly models the event stream as a coherent video sequence. To thebest of our knowledge, we are the first to distill spatio-temporal andmulti-view geometric priors from the Visual Geometry Grounded Trans-former (VGGT) into the event domain. We achieve this via a comprehen-sive tri-level distillation strategy: (i) Cross-Modal Feature Mixture(CMFM) bridges the modality gap at the output level by fusing RGBand event features to generate auxiliary depth predictions; (ii) Spatio-Temporal Feature Distillation (STFD) distills VGGT’s powerfulspatio-temporal representations at the feature level; and (iii) Tempo-ral Consistency Distillation (TCD) enforces cross-frame coherenceat the temporal level by aligning inter-frame depth changes. Extensiveexperiments demonstrate that EventVGGT consistently outperforms ex-isting methods—reducing the absolute mean depth error at 30m by over53% on EventScape (from 2.30 to 1.06)—while exhibiting robust zero-shot generalization on the unseen DENSE and MVSEC datasets. Thecode is available at https://github.com/yinruiRen/EventVGGT.

Latent diffusion models (LDMs) have recently significantlyadvanced embodied learning in constructing powerful embodied manipu-lation world models. However, despite the remarkable performance, exist-ing LDMs predominantly rely on Variational Autoencoders (VAEs) opti-mized for natural scenes while failing to account for the unique character-istics of embodied manipulation scenarios, yielding latent representationsthat are neither compact nor controllable, thereby hindering efficienttraining of LDMs and precise robotic control. To solve this problem, wepresent EmbodiedVAE, a novel video VAE that provides compact yetcontrollable latent representations tailored for the robotic manipulationworld models. Specifically, EmbodiedVAE adopts a dual-encoder, single-decoder architecture with an asymmetric spatio-temporal compressionmodule, which automatically disentangles the robot arm’s motion frombackground environment, resulting in overall compactness while provid-ing explicit embodied latent to support fine-grained action control. Tofurther preserve the temporal consistency of learned robotic motion la-tent, we introduce an optimal-transport-based consistency module thatexplicitly enforces motion fidelity and inter-frame coherence. Extensiveexperiments demonstrate that our proposed EmbodiedVAE achieves su-perior reconstruction quality with high compression rate, while enablingmore precise action control in robotic manipulation scenarios with anaverage of 2dB PSNR improvement over state-of-the-art video VAEs.Code is available at: https://github.com/Mutual-Luo/EmbodiedVAE

The integration of novel view synthesis (NVS) and open-vocabulary segmentation (OVS) has recently yielded powerful feed-forward3D foundation models. However, their inherent reliance on static-sceneassumptions leads to severe misalignment of spatial features in uncon-strained dynamic environments. To bridge this critical gap, we proposeSPAR, a novel joint semantic-geometric encoding architecture that ex-plicitly isolates transient dynamic noise prior to latent space aggregation.Furthermore, we introduce a dynamic-region-aware end-to-end trainingparadigm that structurally couples motion estimation with multi-viewvisual and semantic learning. This unified approach enables the networkto inherently resolve motion conflicts and distill multi-view consistent,temporally stable scene representations from dynamic inputs. Extensiveexperiments on the challenging D-RE10K benchmark demonstrate thatSPAR achieves state-of-the-art performance. Our end-to-end approachachieves exceptional novel view synthesis quality, yielding a PSNR of22.15 dB and 23.33 dB given only 3 and 4 input views respectively. De-spite being trained in a self-supervised manner, our model achieves anmIoU of 88.5% for motion mask prediction. Furthermore, our analysisreveals a strong inter-task synergy between photometric scene recon-struction and semantic understanding, where semantic synthesis learningconsistently enhances photometric fidelity in novel view rendering. Codewill be available at https://github.com/dmucby/SPAR.

Multi-Modal LLMs (MLLMs) demonstrate strong visualgrounding capabilities on popular object detection benchmarks likeOdinW-13 and RefCOCO. However, state-of-the-art models still struggleto generalize to out-of-distribution classes, tasks and imaging modalitiesnot typically found in their pre-training. While in-context promptingis a common strategy to improve performance across diverse tasks, wefind that it often yields lower detection accuracy than prompting withclass names alone. This suggests that current MLLMs cannot yet effec-tively leverage few-shot visual examples and rich textual descriptionsfor object detection. Since frontier MLLMs are typically only accessi-ble via APIs, and state-of-the-art open-weights models are prohibitivelyexpensive to fine-tune on consumer-grade hardware, we instead exploreblack-box prompt optimization for few-shot object detection. To this end,we propose Detection Prompt Optimization (DetPO), a gradient-freetest-time optimization approach that refines text-only prompts by max-imizing detection accuracy on few-shot visual training examples whilecalibrating prediction confidence. Our proposed approach yields consistentimprovements across generalist MLLMs on Roboflow20-VL and LVIS,outperforming prior black-box approaches by up to 9.7 mAP. Our codeand optimized prompts are available on our project page.

Video generation models achieve high visual quality but of-ten struggle to generate physics-aware videos. Unlike rigid-body motion,which can be described by explicit trajectories or formulas, complex de-formation dynamics remain challenging to synthesize. We observe that alack of physical reasoning for localizing dynamic areas allows irrelevantregions to dilute the model’s attention, leading to generation failure.In this paper, we propose DeforM, a reasoning-guided image-to-videogeneration framework that directs the model’s focus toward physics-critical regions. To reason and localize these critical regions, we introducea VLM-guided physical reasoning module, DeforM-Reason, to identifytarget objects and generate spatial-temporal masks. For physical guid-ance, we develop two alternative strategies: DeforM-Free for training-freemechanism analysis, and DeforM-Injection as a powerful training-basedgenerator. Experimental results demonstrate that DeforM improves therealism of generated deformation scenarios, outperforming baseline mod-els in both visual quality and physical consistency.

Applying 3D Gaussian Splatting (3DGS) to uncontrolled, in-the-wild environments remains challenging due to transient distractors.Existing approaches typically mask these distractors and suppress theirlosses during training. However, this zero-masking strategy leaves suchregions unsupervised, leading to per-view overfitting and causing severefloaters in novel views. To resolve this, it is essential to re-establish densemulti-view constraints by recovering the missing background informa-tion. In light of this, we propose PriSplat, a novel framework designed topropagate reliable multi-view information to restore these missing regionswith high geometric integrity. Specifically, we repurpose a large-scaleview synthesis prior into a 3D-aware inpainting engine, adapted throughmask-aware fast-weight updates to prevent distractor leakage into thescene memory. To ensure the fidelity of this restoration, we introducea geometry-aware support view selection algorithm based on informa-tion density and spatio-angular constraints. Ultimately, these synergisticadvancements yield 3D-consistent pseudo-ground truth from masked re-gions, establishing the dense supervision necessary to eliminate artifacts.Extensive experiments show that our method outperforms state-of-the-art baselines in both synthesis quality and multi-view consistency. Thecode is available at https://github.com/yun-seo/PriSplat.git.

Indoor scene relighting demands photorealism, precise spa-tial control, and strict multi-view consistency. While diffusion-based im-age editing models enable semantic lighting manipulation via text prompts,enforcing exact 3D light placement often disrupts their generative pri-ors. We propose Lume-Palette, a progressive framework that leveragessemantic lighting priors for spatially controllable multi-view indoor re-lighting. The approach decouples relighting into two stages: (1) illumi-nation distillation, which extracts canonical illumination palettes froma pretrained diffusion model to preserve realistic material–light interac-tions, and (2) illumination casting, which explicitly maps target spatiallighting conditions defined from coarse 3D geometry. To efficiently handledense multi-view and multi-modal inputs, we introduce an asymmetricmulti-view conditioning strategy that selectively injects essential spatialcontext. Experiments on diverse synthetic scenes and real-world scenesdemonstrate that Lume-Palette produces photorealistic, spatially con-trollable, and multi-view consistent relighting results.

Classical mesh texturing techniques blend captured multi-view images directly, which inevitably suffer from baked-in shading andcasted shadows that compromise visual fidelity during relighting. To cir-cumvent this issue, we present a neural texturing framework, namelyDante-w, to enable high-fidelity diffuse albedo texture recovery fromunstructured image collections for large-scale, in-the-wild scenes, whichintegrates seamlessly with traditional 3D reconstruction pipelines. Givena reconstructed mesh and its surface parameterization, our method fusesview-space generative albedo priors into a coherent texture space viaan expressive neural representation, while substantially enhancing fine-grained textural details through physically principled neural rendering.To comprehensively evaluate our method, we curate a benchmark datasetfeaturing diverse, fine-grained textures, comprising both real-world in-the-wild scenes and synthetic objects. Extensive experiments verify theeffectiveness of our approach in reconstructing accurate albedo texturesand boosting relighting fidelity. Project page: dante-wild.github.io.