ECCV 2026 论文列表

Predicting object dynamics (i.e., world modeling) is a fun-damental challenge for robotic manipulation, and modeling deformableobjects presents a particularly difficult case due to their high-dimensionalstate spaces and complex material properties. While current world mod-els approach this through two distinct paradigms: learning the dynamicsover the 2D pixel space or more explicit 3D geometric space. A systematicunderstanding of their relative strengths and limitations remains elusivedue to the lack of diverse, large-scale real-world data. To address this,we present Deform360, a large-scale visuotactile dataset featuring 198daily-life objects, 1,980 interaction sequences, and over 215 hours of ob-servations from 41 surround-view cameras and bimanual tactile grippersto capture both global motion and contact-induced local deformations.Leveraging a novel markerless visuotactile 3D tracking pipeline to ex-tract dense geometry and motion, we systematically evaluate currentstate-of-the-art world models, comparing 2D video models against 3Dparticle models. Finally, we provide a preliminary demonstration indi-cating the real-world applicability of our dataset by performing robotplanning tasks on deformable objects. Our analysis reveals key insightsinto the trade-offs between structural priors and scalability, providing asolid benchmark for future research in generalizable deformable object-centric world modeling. Project website: https://deform360.lhy.xyz

Understanding 3D objects from images is fundamental to robotics and AR/VR applications. While recent work has made progress in category-level pose estimation, current representations fail to capture the fine-grained semantics needed for reasoning about object parts, functions, and interactions. In this work, we study category-level 3D correspondence in camera space—predicting, from a single image, 3D locations that remain consistent across instances within a category—and show that it can emerge without explicit correspondence supervision by learning a shared morphable object prior. To enable research in this direction, we introduce HouseCorr3D, the first large-scale benchmark for single-view category-level 3D correspondence with 178k images across 50 household object categories, 280 unique instances, and 3D keypoint annotations directly on CAD models. Crucially, HouseCorr3D provides amodal correspondence labels for occluded regions and explicit symmetry annotations, addressing key limitations of existing datasets. We further propose Morpheus, a method that learns morphable category-level shape priors by disentangling canonical shape, deformation, and object pose. Through this shared canonical grounding, semantically meaningful 3D correspondences in camera space emerge implicitly. These emerging 3D correspondences set a new state of the art on HouseCorr3D, demonstrating that semantic 3D object understanding can arise without direct correspondence supervision. Data and code: §/GenIntel/HouseCorr3D.

Shadows encode rich information about scene geometry andillumination, yet existing methods either predict a unified shadow maskor overlook attached shadows entirely. We address this gap by propos-ing a framework for jointly detecting cast and attached shadows throughexplicit physical modeling of light direction and surface geometry un-der a dominant directional-light setting. Our approach is grounded in asimple observation: surfaces facing away from the light source tend tofall into shadow. We exploit the reciprocal relationship between shadowformation and light estimation to construct a closed feedback loop, adual-module architecture in which a shadow detection module and alight estimation module iteratively refine each other. At each pass, up-dated light estimates, together with surface normals, produce partial at-tached shadow maps that guide detection, while improved shadow pre-dictions sharpen light estimation. To support training and evaluation,we introduce a dataset of 1,458 images with manually annotated castand attached shadow masks sourced from three existing benchmarks.Experiments demonstrate that our proposed method outperforms priormethods, with at least a 33% reduction in attached-shadow BER, whilemaintaining strong full-shadow and cast-shadow performance. Projectpage: https://shilin21.github.io/attached_detection/

The next frontier for video generation lies in developing models capable of zero-shot reasoning, where understanding real-world scientific laws is crucial for accurate modeling of physical outcomes under diverse conditions. However, existing video benchmarks are physical commonsense-based, offering limited insight into video models’ scientific reasoning capability. We introduce VideoScience-Bench1, a benchmark for evaluating undergraduate-level scientific understanding in video models. Each prompt encodes a composite scientific scenario that requires understanding and reasoning across multiple scientific concepts to generate the correct phenomenon. The benchmark comprises 200 evaluation items built from 160 expert-curated test cases (160 text-to-video prompts; 40 also include image conditioning) spanning 14 topics and 103 concepts. We conduct expert-annotated evaluations across seven state-of-the-art video models in T2V and I2V settings, along five dimensions: Prompt Consistency, Phenomenon Congruency, Dynamism, Immutability, and Spatio-Temporal Coherence. Using a VLM-as-a-Judge to assess video generations, we observe strong correlation with human assessments. To the best of our knowledge, VideoScience-Bench is the first benchmark to evaluate video models not only as generators but also as reasoners, requiring their generations to demonstrate scientific understanding consistent with expected physical and chemical phenomena.

Visual autoregressive (VAR) models synthesize images in acoarse-to-fine manner by predicting multi-scale token maps, achievingcompetitive text-to-image quality with fewer decoding iterations thanraster-scan autoregression. However, late-scale refinement dominates in-ference time because computation grows quadratically with resolutionand attention becomes increasingly costly. Prior accelerations either fil-ter tokens heuristically, risking severe artifacts, or rely on extra deci-sion models that require per-model fine-tuning with reduced flexibility.We present the first in-depth study of attention behavior across VARgeneration steps and uncover two key properties: (1) attention patternscorresponding layers remain consistent across steps; (2) conditional andunconditional branches progressively converge at later scales. Leveragingthese insights, we introduce a novel training-free, flashattention-friendlyacceleration recipe ARVAR with two primary components: (1) Atten-tion Retrospect (AR) trims tokens at high resolution phase by to-ken saliency from early stage; (2) Branch Halving (BH) drops oneof the conditional/unconditional branches in the late steps. Combined,AR and BH deliver faster and higher-quality autoregressive image gen-eration, achieving 1.56× and 3.01× speedups on HART and Infinityrespectively with negligible performance drop. Compared with existingacceleration methods, ARVAR substantially improves image fidelity.

In real-world deployment, vision-language models often en-counter disturbances such as weather, occlusion, and camera motion.Under such conditions, their understanding and reasoning degrade sub-stantially, revealing a gap between clean, controlled (i.e., unperturbed)evaluation settings and real-world robustness. To address this limitation,we propose ROVA, a novel training framework that improves robustnessby modeling a robustness-aware consistency reward under spatio-temporalcorruptions. ROVA introduces a difficulty-aware online training strategythat prioritizes informative samples based on the model’s evolving capa-bility. Specifically, it continuously re-estimates sample difficulty via self-reflective evaluation, enabling adaptive training with a robustness-awareconsistency reward. We also introduce PVRBench, a new benchmark thatinjects real-world perturbations into embodied video datasets to assessboth accuracy and reasoning quality under realistic disturbances. Weevaluate ROVA and baselines on PVRBench, UrbanVideo, and VisBench,where open-source and proprietary models suffer up to 35% and 28%drops in accuracy and reasoning under realistic perturbations. ROVAeffectively mitigates performance degradation, boosting relative accuracyby at least 24% and reasoning by over 9% compared with baseline models(QWen2.5/3-VL, InternVL2.5, Embodied-R). These gains transfer to cleanstandard benchmarks, yielding consistent improvements.

Affordance prediction, which identifies interaction regionson objects based on language instructions, is critical for embodied AI.Prevailing end-to-end models couple high-level reasoning and low-levelgrounding into a single monolithic pipeline and rely on training over an-notated datasets, which leads to poor generalization on novel objects andunseen environments. In this paper, we move beyond this paradigm byproposing A4-Agent, a training-free agentic framework that decouplesaffordance prediction into a three-stage pipeline. Our framework coordi-nates specialized foundation models at test time: (1) a Dreamer thatemploys generative models to visualize how an interaction would look;(2) a Thinker that utilizes large vision-language models to decide whatobject part to interact with; and (3) a Spotter that orchestrates vi-sion foundation models to precisely locate where the interaction area is.By leveraging the complementary strengths of pre-trained models with-out any task-specific fine-tuning, our zero-shot framework significantlyoutperforms state-of-the-art supervised methods across multiple bench-marks and demonstrates robust generalization to real-world settings.

VLM-Parser object features purpose •••3-View VLM Parse ManualImages Stool AssemblyGeneralizingAssembly VLM-PlannersequenceInferVLMUpdateAssembly LampAttempts sequence Assembly(a) Digital Twins System (b) Parser and Planner (c) Zero-shot AssemblyFig. 1: Illustrating the core concept of ZAP: a VLM-guided agent transforms multi-view RGB-D observations into an assembly plan through internal reasoning, while adigital twin is used for geometry-based feasibility verification.

Dynamic view synthesis has seen significant advances, yet re-constructing scenes from uncalibrated, casual videos remains challengingdue to ambiguity between camera and object motion, inadequate viewcoverage and slow optimization. In this work, we present Minute4D,a novel and efficient approach for high-fidelity 4D scene reconstructionfrom monocular videos using Gaussian Splatting. Our Gaussian initial-ization begins with an efficient geometric recovery leveraging pre-trainedvisual foundation models. To improve reconstruction fidelity, we intro-duce a Segmentation-Tracking Enhancement module that leverages 2Dsemantic priors and 3D point tracking to jointly enhance the geometricaccuracy and completeness of the initial Gaussians. To reduce redun-dancy and accelerate Gaussian optimization, we propose a Loss-GuidedDensity Control strategy that adaptively densifies and prunes Gaussiansbased on multi-view photometric loss. Our method reduces the optimiza-tion time to approximately 40 seconds for a typical 400-frame video,yielding a speed-up of at least 5×. Extensive experiments show thatour method not only enables substantially faster optimization but alsoachieves superior performance across several benchmarks.*: Equal contribution.†: Co-corresponding authors.

Recent text-to-video (T2V) diffusion models rely heavily on auxiliary rewardsignals (e.g., via reward models or DPO) to align generated content with human aesthetics andimprove realism. These signals, however, incur substantial computational overhead, requirecostly human annotations, and often yield limited improvement in fine-grained local details. Inthis paper, we argue that your data manifold is secretly a reward model. By explicitlymodeling the manifold structure of high-quality Supervised Fine-Tuning (SFT) data andencouraging video latents to lie on this manifold, we derive dense, differentiable, and nearlycost-free reward signals that significantly improve video quality, particularly in mitigating low-level distortions. Our modeling builds upon Local Coordinate Coding (LCC), which capturesthe ‘skeleton’ of the manifold. However, directly applying LCC suffers from mean regression,pulling latents toward the geometric mean and losing high-frequency details. We thereforeextend it to Shell Local Coordinate Coding (Shell-LCC), which models the manifold ‘surface’as an isotropic shell to align with the true high-density region. Experiments demonstratethat our approach improves realism, enhances high-frequency details, reduces over-smoothingartifacts, and alleviates motion blur.

Talking-head generation from text typically relies on cas-caded pipelines that synthesize audio before video, leading to error ac-cumulation, latency, and audiovisual misalignment. Existing end-to-endmodels address some of these issues but often lack autoregressive capabil-ities. In this paper, we present VoCa, a unified autoregressive frameworkthat jointly generates speech and talking-head video from text transcriptsand a reference image. VoCa employs a large language model (LLM) tomap text into shared representations, which are concurrently processedby dedicated speech and video decoders to ensure strict synchronization.To bridge the semantic gap between audio-optimized features and hu-man motion, we introduce a speech refiner with a tailored multi-stagetraining strategy. Furthermore, to enable step-by-step autoregressive co-generation, we propose a unified window partitioning mechanism andan associated improved rolling-window denoising strategy. This adaptsa bidirectional diffusion transformer into a causal architecture synchro-nized with the LLM’s autoregressive generation windows. Extensive ex-periments show that VoCa matches or surpasses state-of-the-art baselinesin visual quality, improves audio–lip synchronization, and identity preser-vation. Code and models will be released upon acceptance.

Video-text models adapted from image-text architectures(e.g., CLIP) frequently exhibit temporal blindness—the inability to per-ceive fundamental cues like order, direction, and motion dynamics. Stan-dard datasets mask this limitation by enabling models to exploit staticspatial shortcuts. To systematically evaluate this, we introduce XTE-Bench, a diagnostic probe revealing that even large-scale video-languagemodels struggle with basic temporal reasoning, indicating that param-eter scaling alone is insufficient to resolve this flaw. To address this,we propose Cross-Modal Temporal Edits (XTE), a self-supervisedframework that injects precise temporal supervision. By performing syn-chronized video-text transformations, XTE generates hard temporal neg-atives without manual annotation. We instantiate this with ViTAL-X,a lightweight model that equips frozen image-text backbones with tem-poral awareness while preserving their foundational spatial knowledge.Across six temporal benchmarks, ViTAL-X achieves state-of-the-art per-formance. Utilizing only 0.4B parameters and 1M training clips, ViTAL-X outperforms 7B-parameter models and surpasses baselines trained on600× more data. These results demonstrate that targeted, high-qualitytemporal alignment provides a highly efficient alternative to pure scaling.

Multi-view spatial reasoning remains difficult for currentvision-language models. Even when multiple viewpoints are available,models often underutilize cross-view relations and instead rely on single-image shortcuts, leading to fragile performance on viewpoint transforma-tion and occlusion-sensitive cases. We present ViewFusion, a two-stageframework that explicitly separates cross-view spatial pre-alignment fromquestion answering. In the first stage, the model performs deliberatespatial pre-thinking to infer viewpoint relations and spatial transfor-mations across views, forming an intermediate workspace that goes be-yond a simple re-description. In the second stage, the model conductsquestion-driven reasoning conditioned on this workspace to producethe final prediction. We train ViewFusion with synthetic reasoningsupervision followed by reinforcement learning using GRPO, which im-proves answer correctness while stabilizing the intended two-stage gen-eration behavior. On MMSI-Bench, ViewFusion improves accuracyby 5.3% over Qwen3-VL-4B-Instruct, with the largest gains on exam-ples that require genuine cross-view alignment. Our code is available athttps://github.com/taoxj2001/ViewFusion.

Spatial graphs provide a lightweight and elegant represen-tation of curvilinear anatomical structures such as blood vessels, lungairways, and neuronal networks. Accurately modeling these graphs iscrucial in clinical and (bio-)medical research. However, the high spatialresolution of large networks drastically increases their complexity, result-ing in significant computational challenges. In this work, we aim to tacklethese challenges by proposing VesselTok, a framework that approachesspatially dense graphs from a parametric shape perspective to learn la-tent representations (tokens). VesselTok leverages centerline points witha pseudo radius to effectively encode tubular geometry. Specifically, welearn a novel latent representation conditioned on centerline points to en-code neural implicit representations of vessel-like, tubular structures. Wedemonstrate VesselTok’s performance across diverse anatomies, includ-ing lung airways, lung vessels, and brain vessels, highlighting its ability torobustly encode complex topologies. To prove the effectiveness of Vessel-Tok’s learned latent representations, we show that they (i) generalize tounseen anatomies, (ii) support generative modeling of plausible anatom-ical graphs, and (iii) transfer effectively to downstream inverse problems,such as link prediction.

Recent advances in Large Language Models (LLMs) and Text-to-Image (T2I) models have led to the emergence of Unified Multimodal Models (UMMs), where multimodal understanding and image generation are tightly integrated within a shared architecture. Prior studies suggest that such reciprocity enhances cross-functionality performance through shared representations and joint optimization. However, the safety implications of this tight coupling remain largely unexplored, as existing safety research predominantly analyzes understanding and generation functionality in isolation. In this work, we investigate whether cross-functionality reciprocity itself constitutes a structural source of vulnerability in UMMs. We propose RICE: Reciprocal Interaction–based Cross-functionality Exploitation, a novel attack paradigm that explicitly exploits bidirectional interactions between understanding and generation. Using this framework, we systematically evaluate Generation-toUnderstanding (G→U) and Understanding-to-Generation (U→G) attack pathways, demonstrating that unsafe intermediate signals can propagate across modalities and amplify safety risks. Extensive experiments show high Attack Success Rates (ASR) in both directions, revealing previously overlooked safety weaknesses inherent to UMMs. The code is available at https://github.com/tunantu/UMM-Safety.

Deep learning models have become the standard tool for doc-ument rectification and illumination correction, yet their performance isfundamentally bound by their training data. For nearly a decade, the com-munity has heavily relied on Doc3D, a pioneering but increasingly limiteddocument unwarping dataset in terms of scale and quality. To address thisbottleneck, we introduce SyntheticDoc, a massive, high-quality datasetdesigned to push the boundaries of document unwarping. SyntheticDocis composed of 1,000,000 high-resolution procedurally generated trainingsamples, alongside extensive validation and test sets. Each sample ispaired with rich, pixel-perfect annotations, including UV maps, normalmaps, albedo and shading. To ensure physical accuracy and photorealism,the paper geometries are generated via a physics-based simulator andrendered using a path tracer. To demonstrate the benefit of our dataset,we train a simple baseline model on SyntheticDoc and report on itsperformance in comparison to state-of-the-art methods on both documentunwarping and illumination correction tasks. Our dataset is available athttps://igl.ethz.ch/projects/SyntheticDoc/ and the code used to generateit at https://github.com/tanguymagne/SyntheticDoc.

Despite advances in 3D scene understanding, existing 3DLarge Multimodal Models operate in offline settings, requiring completescene observations or predefined video clips. In this paper, we presentan online 3D vision-language model that enables real-time spatial un-derstanding from streaming video. Our approach adopts an autoregres-sive streaming control modeling based on the LLM’s next-token predic-tion objective to learn when to respond, and employs a lightweight Vi-sual–Spatial Feature Integration (VSFI) module to incrementally injecttemporally aligned geometry priors into the visual stream. To alleviatelong-context decoding overhead, we propose a plug-and-play Geometry-Adaptive Voxel Compression (GAVC) module for efficient visual tokencompression. To address the scarcity of streaming 3D–language data,we further develop a scalable data generation pipeline that curates over1M online spatio-temporal 3D QA pairs and establishes a comprehensivebenchmark spanning 29 tasks. Extensive experiments show that our ap-proach significantly outperforms both proprietary and open-source mod-els across online and offline 3D spatial understanding, reasoning, andgrounding tasks.

Incomplete Multi-View Clustering addresses the problem ofclustering multi-modal data when certain views are missing. Recent end-to-end generative approaches leverage diffusion models to recover miss-ing views via stochastic noise-to-data trajectories. While expressive, suchmechanisms are not explicitly designed for clustering, as they initializefrom cluster-agnostic noise and rely on stochastic denoising dynamics.In this work, we revisit probability path design in end-to-end generativeIMVC. We introduce a flow-matching framework with a linear interpo-lation path between paired view representations, that replaces diffusionwith probability flows between observed and missing views. We providea formal analysis showing that deterministic ODE flows are inherentlybetter aligned with clustering objectives than diffusion-based stochastictrajectories, especially in terms of transport mechanisms that respectclass-conditional data distributions and maintain cluster consistency infinite-step regimes. Building upon this insight, we develop an end-to-end IMVC architecture that integrates straight-path flow-matching viewcompletion with cluster-level and entropy-based alignment to enforcecross-view clustering consistency. Extensive experiments on standardIMVC benchmarks demonstrate that the proposed framework establishesnew state-of-the-art performance.

Storyboarding is a core skill in visual storytelling for film,animation, and games. However, automating this process requires a sys-tem to achieve two properties that current approaches rarely satisfy si-multaneously: inter-shot consistency and explicit editability. While 2Ddiffusion-based generators produce vivid imagery, they often suffer fromidentity drift along with limited geometric control; conversely, traditional3D animation workflows are consistent and editable but require expert-heavy, labor-intensive authoring. We present StoryBlender, a grounded3D storyboard generation framework governed by a Story-centric Re-flection Scheme. At its core, we propose StoryBlender system, whichis built on a three-stage pipeline: (1) Semantic-Spatial Grounding, toconstruct a continuity memory graph to decouple global assets fromshot-specific variables for long-horizon consistency; (2) Canonical AssetMaterialization, to instantiate entities in a unified coordinate space tomaintain visual identity; and (3) Spatial-Temporal Dynamics, to achievelayout design and cinematic evolution through visual metrics. By or-chestrating multiple agents in a hierarchical manner within a verifica-tion loop, StoryBlender iteratively self-corrects spatial hallucinations viaengine-verified feedback. The resulting native 3D scenes support direct,precise editing of cameras and visual assets while preserving unwaver-ing multi-shot continuity. Experiments demonstrate that StoryBlendersignificantly improves consistency and editability over both diffusion-based and 3D-grounded baselines. Code and dataset are available athttps://engineeringai-lab.github.io/StoryBlender.