ECCV 2026 Papers

While Vision-Language Models (VLMs) have empowered em-bodied agents to execute complex household tasks, they struggle toproactively handle dynamically emerging hazards during closed-loop in-teractions. Existing safety approaches often rely on runtime guardrailsto block unsafe actions or induce excessive caution, which severely stallstask progress instead of actively resolving the underlying risks. To breakthis safety–progress trade-off, we introduce the Self-Evolving Just-In-Time Memory framework, which reframes embodied safety from progress-stalling guardrails to proactive hazard mitigation. The framework con-sists of a Risk-Sufficient Topological Belief Graph (RSG) for persistentsafety-relevant state tracking under partial observability, an Agency-Grounded Factual Memory for precise hazard anticipation, and an Expe-rience Memory that injects procedural Meta-Skills to guide executable,progress-preserving mitigation. Furthermore, we propose an automatedTest-Verify-Write loop, allowing agents to continually refine their mit-igation Meta-Skills from execution traces at test time. Experiments onIS-Bench demonstrate that our framework substantially boosts the Safe-Success rate across multiple VLM backbones (e.g., +30.3% on Qwen3-VL-8B), enabling agents to proactively mitigate hazards without stallingtask progress. Code is available at https://github.com/DyMessi/JIT-Memory.

Integrating macro-scale radiology and micro-scale pathologyprovides complementary insights crucial for precision oncology. How-ever, this integration is often hindered by modality discrepancy. The fea-ture embeddings from heterogeneous foundation models reside on poorlyaligned latent manifolds, and naive static fusion may therefore mix unre-lated semantic neighborhoods, leading to topological degeneracy and se-mantic entanglement. To bridge this gap, we propose a novel multimodallearning framework for Synthesizing radiology and pathology EvidencEvia Knowledge-guided FLOW matching. Specifically, we construct ashared feature manifold anchored by learnable clinical prototypes, en-abling the disentanglement and synthesis of shared and modality-specificfeature evidence. We then reformulate multimodal integration as a trans-port problem via knowledge-guided flow matching and seek continuousflows that smoothly transport features onto this semantic substrate. Thisparadigm preserves the intrinsic topology of the feature spaces and re-duces feature collapse. Furthermore, an uncertainty-aware evidential ag-gregator is incorporated to synthesize cross-modal feature evidence forreliable decision-making. Extensive experiments on three cancer datasets(i.e., TCGA Glioma, Gastric Cancer, and Chondrosarcoma) demonstratethe effectiveness of SeekFlow. Notably, SeekFlow consistently achievesstate-of-the-art performance across all benchmarks, outperforming ex-isting multimodal fusion methods in both diagnosis and prognosis tasks.The code is available at https://github.com/HKU-MedAI/SeekFlow.

We introduce ScenarioControl, the first vision-language con-trol mechanism for learned driving scenario generation. Given a textprompt or an input image, ScenarioControl synthesizes diverse, realistic3D scenario rollouts – including map, 3D boxes of reactive actors overtime, pedestrians, driving infrastructure, and ego camera observations.The method generates scenes in a vectorized latent space that representsroad structure and dynamic agents jointly. To connect multimodal con-trol with sparse vectorized scene elements, we propose a cross-global con-trol mechanism that integrates cross-attention with a lightweight global-context branch, enabling fine-grained control over road layout and traf-fic conditions while preserving realism. The method produces tempo-rally consistent scenario rollouts from the perspectives different actorsin the scene, supporting long-horizon continuation of driving scenarios.Extensive experiments validate that the control adherence and fidelityof ScenarioControl compare favorable to all tested methods across allexperiments. To facilitate training and evaluation, we release a datasetwith text annotations aligned to vectorized map structures at the projectwebpage: https://light.princeton.edu/ScenarioControl

Pre-trained Vision-Language Models (VLMs) like CLIP haveproven highly effective as foundation models for various downstream ap-plications. However, prompt learning in VLMs encounters a performance-generalization dilemma: while prompts can be tuned to achieve highaccuracy on seen distributions, this tuning process often undermines theirgeneralizability to unseen data. The limited set of learnable prompts,which contextualize and condition the input to steer it toward the taskwithin the pretrained VLM, tends to overfit the training data, leading toa trade-off between task-specific performance and preserving generaliza-tion. To address this dilemma, we introduce SAMPLe (Sharpness-AwareMinimization Prompt Learning), a plug-in sharpness-aware optimizerthat enhances prompt generalizability by accounting for loss landscapesharpness. Unlike conventional methods, SAMPLe balances explorationand exploitation by satisfying objective function constraints at each step,dynamically adapting to the current optimization state based on the localcurvature and gradient properties. This approach reduces overfitting onseen distributions and improves adaptability to unseen data, preservingthe generalization potential of pre-trained VLM models. We integrateSAMPLe into multiple prompt learning frameworks, including CoOp,CoCoOp, MaPLe, TCP, and Co-Prompt, demonstrating its effectivenessacross diverse methods. Experiments show that SAMPLe elevates promptlearning frameworks and consistently outperforms existing optimizersacross diverse settings, establishing itself as a robust, model-agnosticsolution for prompt learning.

Subject-driven image generation and manipulation are fun-damental for personalized content creation, such as identity-preservingportrait synthesis, multi-character storytelling, and controllable photoediting. However, existing models still struggle with multi-reference set-tings, often showing identity drift and unstable editing quality. A keybottleneck is data: current resources are often limited in scale, diversity,and consistency of subject-level details, especially for unified support ofboth generation and manipulation. To address this gap, we introduceOpenSubject, a video-derived large-scale corpus for subject-driven gen-eration and manipulation. Our pipeline leverages cross-frame identitypriors through four key stages: (i) Video Curation. We apply resolutionand aesthetic filtering to obtain high-quality clips. (ii) Cross-FrameSubject Mining and Pairing. We utilize vision language model (VLM)-based category consensus, local grounding, and diversity-aware pairing toselect image pairs. (iii) Identity-Preserving Reference Image Syn-thesis. We introduce segmentation map-guided outpainting to synthesizeinput images for subject-driven generation and box-guided inpaintingto generate input images for subject-driven manipulation, together withgeometry-aware augmentations and irregular boundary erosion. (iv) Ver-ification and Captioning. We utilize a VLM to validate synthesizedsamples, re-synthesize failed samples based on stage (iii), and then con-struct short and long captions. Furthermore, we introduce a benchmarkcovering both subject-driven generation and subject-driven manipulation,evaluated with a VLM-based judge on identity fidelity, prompt adherence,and consistency. Extensive experiments demonstrate that training onOpenSubject significantly enhances performance in complex scenes.

Current instruction-guided video editing models struggle tosimultaneously balance precise semantic modifications with faithful mo-tion preservation. While existing approaches rely on injecting explicitexternal priors (e.g., VLM features or structural conditions) to mitigatethese issues, this reliance severely bottlenecks model robustness and gen-eralization. To overcome this limitation, we present SAMA (factorizedSemantic Anchoring and Motion Alignment), a framework that factor-ize video editing into semantic anchoring and motion modeling. First, weintroduce Semantic Anchoring which establish a reliable visual anchorby jointly predicting semantic tokens and video latents at sparse an-chor frames, enabling purely instruction-aware structural planning. Sec-ond, Motion Alignment pre-trains the same backbone on motion-centricvideo restoration pretext tasks (cube inpainting, speed perturbation, andtube shuffle), enabling the model to internalize temporal dynamics di-rectly from raw videos. SAMA is optimized with a two-stage pipeline: afactorized pre-training stage that learns inherent semantic-motion rep-resentations without paired video-instruction editing data, followed bysupervised fine-tuning on paired editing data. Remarkably, the factorizedpre-training alone already yields strong zero-shot video editing ability,validating the proposed factorization. SAMA achieves state-of-the-artperformance among open-source models and is competitive with leadingcommercial systems (e.g. Kling-Omni). Code, models, and datasets arereleased in https://cynthiazxy123.github.io/SAMA.

Diffusion Transformers (DiTs) achieve superior image gener-ation quality but suffer from quadratic computational complexity rela-tive to token count. While various token reduction (TR) methods havebeen proposed to mitigate this cost, they overlook the primary objectiveof generative models: minimizing recovery error, which requires reflect-ing output token similarity. Specifically, they rely solely on input tokensimilarity inherited from reduction-only ViT paradigms, leading to a fun-damental misalignment with this objective.To bridge this gap, we propose DiTo, a novel TR paradigm that shiftsthe focus toward output-centric token reduction. Based on the observa-tion that output token similarity is consistently preserved across adjacenttimesteps, DiTo utilizes prior-step similarities as an effective proxy toestablish token correspondences at a Matching timestep, and reuses theresulting correspondences across multiple subsequent Reduction steps.To optimize this interleaved scheduling, we propose Pair Match Ra-tio (PMR)-guided Interval Scheduling to determine the optimal match-ing frequency. Furthermore, to mitigate localized approximation errorsand resulting blocking artifacts caused by repeated reuse, we proposeFrequency-aware Token Matching by incorporating a selection-frequencypenalty. Extensive experiments demonstrate that DiTo consistently out-performs existing TR methods with 1.6–3.9 dB higher PSNR at compa-rable speedups, achieving a superior Pareto frontier.

Region-level retrieval aims to align user-specified image regions with relevantregions or textual descriptions, playing a crucial role in realworld applications such as e-commerce product search and RAG. Although recent Large Multimodal Models (LMMs)have made significant strides in multimodal retrieval, they primarily focus on global-level tasksand struggle to capture effective region-level representations. To bridge this gap, we presentRegRet, an LMM-based Region-level Retrieval framework that enhances the regional rep-resentations without compromising overall global retrieval performance. At its core, RegRetintegrates a Region-Aware Encoder to capture detailed regional features while balancing themwith the global background context. To further enhance the fine-grained understanding anddiscriminability of representations, we design a multi-stage training pipeline that includesdetailed localized captioning and regional contrastive learning tasks. In addition, consideringthe absence of region-level contrastive training data and the limited diversity of evaluationtasks in current benchmarks, we introduce the REGMB benchmark. It comprises 225k con-trastive pairs, covering four multimodal retrieval tasks. Extensive experiments validate theeffectiveness of our approach. RegRet outperforms strong baselines in the zero-shot setting.Further training with contrastive learning leads to an average improvement of more than 20%on both REGMB and public benchmarks, while achieving comparable or better results onglobal-level retrieval tasks. The code and data will be released for future research.

The massive volume and privacy-sensitive nature of visualdata have made federated learning (FL) a preferred paradigm for train-ing vision models across distributed data sources. However, during train-ing, FL repeatedly shares the in-progress model with randomly selectedparticipants. This paper investigates an overlooked yet practical threatarising from this sharing process: leaked intermediate models can be ex-ploited by adversaries to craft adversarial examples that compromise thefinal deployed model. Although directly using an intermediate model,especially one leaked early in training, as a surrogate yields only mod-erate attack gains, it can serve as an anchor for anticipating subsequenttraining dynamics. Based on this insight, we propose Proteus, a modelleakage-induced adversarial attack that leverages a leaked model to iden-tify vulnerabilities that persist throughout training, thereby generatingadversarial examples that remain effective against the final deployedmodel. For the first time, we show that models exposed well before con-vergence can already pose substantial risks to the final model, even if itundergoes hundreds of additional training rounds after leakage. Exten-sive experiments across diverse datasets, neural architectures, and FLconfigurations confirm the severity of this threat. Proteus exploits in-herent model leakage in FL and improves the attack success rate from59.75% when directly using the leaked model to 85.40%, even when leak-age occurs after only 30% of the total training process.

Magnetic resonance imaging comes in various modality con-trasts that provide complementary anatomical and pathological informa-tion. Complete multimodal acquisitions are often unavailable due to timeand protocol constraints. This leads to real-world datasets with missingmodalities, where conventional medical image translation methods aretypically limited to fixed source-target settings or require retraining foreach observed source-target pair. We propose a unified framework thatformulates missing-modality generation as a linear inverse problem un-der a joint distribution and solves it via posterior sampling with a flowmatching model. By learning a joint prior over the complete modalityset, our method can reconstruct arbitrary missing modalities at inferencetime by guiding the sampling trajectory to enforce measurement consis-tency with observed modalities. We further mitigate inter-modality er-ror propagation in multi-target generation by adopting a many-to-onesampling strategy. Experiments on BraTS and IXI datasets show thatour method achieves the best performance over baselines across mostmissing-modality scenarios. In downstream tumor segmentation, synthe-sized images from our method result in higher segmentation performance,indicating better preservation of clinically relevant structures. Our codeis available at github.com/jongdory/PS-MIT.

Flow matching (FM) trains a time-dependent vector field that transports samples from a simple prior to a complex data distribution. However, for high-dimensional images, each training sample supervises only a single trajectory and intermediate point, yielding an extremely sparse and high-variance training signal. This under-constrained supervision can cause flow collapse, where the learned dynamics memorize specific source–target pairings, mapping diverse inputs to overly similar outputs, failing to generalize. We introduce Posterior Augmented Flow Matching (PAFM), a theoretically grounded generalization of FM that replaces single-target supervision with an expectation over an approximate posterior of valid target completions for a given intermediate state and condition. PAFM factorizes this intractable posterior into (i) the likelihood of the intermediate under a hypothesized endpoint and (ii) the prior probability of that endpoint under the condition, and uses an importance sampling scheme to construct a mixture over multiple candidate targets. We prove that PAFM yields an unbiased estimator of the original FM objective while substantially reducing gradient variance during training by aggregating information from many plausible continuation trajectories per intermediate. Finally, we show that PAFM improves over FM by up to 3.4 FID50K across different model scales (SiT-B/2 and SiT-XL/2), different architectures (SiT and MMDiT), and in both class and text conditioned benchmarks (ImageNet and CC12M), with a negligible increase in the compute overhead. Code: https://github.com/gstoica27/PAFM.git.

When should an intelligent assistant speak up without beingasked? Continuous egocentric video offers rich, evolving context that en-ables a new form of assistance: one that is proactive rather than merelyreactive. Yet existing approaches either wait passively for user queries ortreat every detected event as requiring a response, without consideringthe user’s history, current activity, or whether assistance would actuallybe welcome. We reframe proactive assistance as a context-dependent de-cision problem: the agent must not only perceive what is happening,but reason over accumulated temporal context to determine when andwhether to intervene. To this end, we present Vinci2, a proactive ego-centric assistance system that advances the on-device assistant Vincifrom reactive response toward proactivity. On the evaluation side, wepresent EgoServe, the first large-scale benchmark for proactive assistancein continuous egocentric video. EgoServe comprises over 3,000 serviceinstances organized along 4 temporal memory horizons, ranging fromimmediate safety alerts to long-term habit coaching, across 10 servicecategories. On the modeling side, we propose EgoMemo, a training-free,memory-augmented agent that maintains three complementary mem-ory representations: multi-scale temporal summaries, a semantic knowl-edge graph, and visual embedding archives. At each timestep, EgoMemoperforms retrieval-augmented reasoning to determine whether assistanceis warranted and, if so, produces contextually grounded responses. Ex-periments demonstrate that EgoMemo establishes strong baselines onEgoServe while remaining competitive on existing egocentric benchmarks.Our benchmark and code are publicly available at Vinci2.

Realistic and diverse multi-agent driving scenes are crucial for evalu-ating autonomous vehicles, but safety-critical events which are essential for thistask are rare and underrepresented in driving datasets. Data-driven scene genera-tion offers a low-cost alternative by synthesizing complex traffic behaviors fromexisting driving logs. However, existing models often lack controllability or yieldsamples that violate physical or social constraints, limiting their usability. Wepresent OMEGA, an optimization-guided, training-free framework that enforcesstructural consistency and interaction awareness during diffusion-based samplingfrom a scene generation model. OMEGA re-anchors each reverse diffusion stepvia constrained optimization, steering the generation towards physically plausibleand behaviorally coherent trajectories. Building on this framework, we formu-late ego–attacker interactions as a game-theoretic optimization in the distributionspace, approximating Nash equilibria to generate realistic, safety-critical adver-sarial scenarios. Experiments on nuPlan and Waymo show that OMEGA improvesgeneration realism, consistency, and controllability, increasing the ratio of physi-cally and behaviorally valid scenes from 32.35% to 72.27% for free explorationcapabilities, and from 11% to 80% for controllability-focused generation. Our ap-proach can also generate 5× more near-collision frames with a time-to-collisionunder three seconds while maintaining the overall scene realism.

While modern 3D reconstruction excels at modeling objectgeometry and appearance, it largely ignores the rich acoustic cues re-vealed through physical interaction. Object impact sounds convey mate-rial, stiffness, and structural properties that complement vision, yet exist-ing impact sound modeling approaches either rely on expensive physics-based simulation or require large datasets to generalize in a purely data-driven manner. We introduce Audio-Visual Modal Sound Field (AV-MSF), a novel object-level acoustic representation reconstructed frommulti-view images and only a few impact sound recordings. AV-MSFbuilds on 3D Gaussian Splatting integrated with dense 3D visual featureto provide a strong geometry-aware prior, and represents the impactsound field using compact, physically meaningful modal parameters, en-abling robust few-shot reconstruction. Experiments on two real-worlddatasets show that AV-MSF achieves state-of-the-art impact sound ren-dering, outperforming both physics-based and data-driven baselines. Fur-thermore, we demonstrate downstream applications enabled by our rep-resentation, including contact localization and object sound editing.

Material transfer seeks to re-render an object’s surface usinga reference exemplar while preserving its original geometry and identity.Current training-free methods often rely on attention-based injection,which entangles attributes like color, texture, and structural patternsinto a single representation. This entanglement leads to structural in-stability and imprecise results. Additionally, inversion-based editing inflow models is prone to reconstruction-induced trajectory drift, whereaccumulated errors degrade object details. We propose MaterialFlow, atraining-free framework for precise material transfer using pre-trainedflow models. Our approach introduces a trajectory-aware velocity modu-lation mechanism that rebalances the generative flow in an inversion-free manner. This ensures stable and semantically consistent editingdynamics without the need for explicit latent reconstruction. We fur-ther introduce an attribute-aware disentanglement paradigm that sepa-rates reference materials into color, texture, and meso-scale patterns forfine-grained control. Evaluations on the Material Transfer Benchmark(MTB) show that MaterialFlow surpasses state-of-the-art methods intransfer quality, inference efficiency, and controllability. Project page:https://github.com/Sung-Lin/MaterialFlow.

High-resolution multi-contrast MRI is essential for capturingrich anatomical structures and supports more accurate disease diagnosisthrough the reference images from another contrast. It often suffers fromlimited spatial resolution due to acquisition constraints, and requestsexploring arbitrary-scale reconstruction beyond fixed scale factors. How-ever, related research exists two following issues: 1) Implicit neural rep-resentations (INR) as mainstream methods are prone to spectral bias,which limits their ability to recover high-frequency details; 2) Multi-contrast MRI is often used as the effective prior, but lacks the targetednetwork design to further merge INR positional information. To solvethese problems, we propose a Fourier-enhanced implicit framework forarbitrary-scale multi-contrast MRI super-resolution (MArFE). First, theFourier reparameterization module (FRM) is introduced to character-ize frequency features through fixed Fourier basis decomposition. Then,Fourier neural operators can alleviate frequency degradation by the non-local Galerkin-type linear attention (NLGLA) based on kernel integralmechanisms. Additionally, we design a hybrid loss that jointly supervisesboth spatial and frequency domains to improve high-frequency texturereconstruction. Extensive experiments on two public multi-contrast MRIdatasets demonstrate that MArFE outperforms existing methods undervarious scale factors with superior visual performances and quantitativemetrics. These results highlight the clinical potential of MArFE in en-hancing MRI quality across multi-contrast sequences. The project codecan be found at https://github.com/zhiwen-shi/MArFE.

The performance of vision-language models (VLMs) in videounderstanding declines with increasing video duration, as video momentsunrelated to the query confuse their language components. Multimodalretrieval has emerged as a critical component of video understanding,addressing this challenge by localizing key visual evidence. However, ex-isting multimodal retrieval methods suffer from biased relevance estima-tion, limited diversity, and temporal collapse. In this paper, we proposeQSVideo, a unified framework that systematically addresses relevance,diversity, and temporal modeling in video retrieval. We first introducea query-conditioned semantic ranker, QSRanker, which reformulates ar-bitrary questions into retrieval-friendly queries and estimates structuredrelevance along object, action, and location dimensions. Building uponthis, we design QSRetrieval to jointly optimize relevance and diversityfor more informative frame selection. Moreover, we propose temporalalignment strategies tailored for both long and streaming videos to im-prove evidence recall. Extensive experiments on long and streaming videobenchmarks demonstrate that QSVideo greatly enhances video VLM per-formance under strict frame limit constraints. The code is available athttps://github.com/human-analysis/QSVideo.

While Image-to-Video models unlock new creative possibil-ities, they also raise risks of unauthorized adaptation, deepfakes, andscenario manipulation, making robust authentication and traceability so-lutions urgent for image owners. While image watermarking offers a solu-tion, existing methods expose a critical vulnerability: due to the inherentsemantic drift of generated videos, images watermarks survive only in ini-tial frames. This allows attackers to easily evade detection via temporalcropping—discarding early frames to obtain watermark-free derivativevideos. Consequently, in an era of pervasive I2V models, long-term ro-bustness is imperative for image watermarking. Since simply increasingforward embedding strength fails against severe generative divergence,we introduce LoT-Pass, a watermarking framework built on a new in-sight: only one-directional robustness enhancement is hard to achievelong-term robustness. We propose a two-way strategy: (1) expand therobustness distance by training under simulated temporal evolution inI2V generation, and (2) recover unextractable frames by reversing themthrough a temporal inversion module. This shift, from merely resistingdistortions to actively rewinding the video to a watermark-friendly state,enables LoT-Pass to maintain extractability even under temporal drift.Experiments on mainstream open-source and commercial I2V modelsdemonstrate that LoT-Pass achieves stronger long-term robustness whilepreserving imperceptibility, offering a new paradigm for image copyrightprotection in the era of widespread I2V adoption. Code

Recent advancements in open-vocabulary 3D scene under-standing heavily rely on 3D Gaussian Splatting (3DGS) to register vision-language features into 3D space. However, we identify two critical limi-tations in these approaches: the spatial ambiguity arising from unstruc-tured, overlapping Gaussians which necessitates probabilistic feature reg-istration, and the multi-level semantic ambiguity caused by pooling fea-tures over object-level masks, which dilutes fine-grained details. To ad-dress these challenges, we present a novel framework that leverages SparseVoxel Rasterization (SVRaster) as a structured, disjoint geometry repre-sentation. By regularizing SVRaster with monocular depth and normalpriors, we establish a stable geometric foundation. This enables a de-terministic, confidence-aware feature registration process and suppressesthe semantic bleeding artifact common in 3DGS. Furthermore, we resolvemulti-level ambiguity by exploiting the emerging dense alignment proper-ties of the AM-RADIO foundation model, avoiding the computationaloverhead of hierarchical training methods. Our approach achieves state-of-the-art performance on Open Vocabulary Point Cloud Understanding,and highly competitive results on 3D Object Retrieval benchmarks.