ECCV 2026 Papers

Talking-head generation from text typically relies on cas-caded pipelines that synthesize audio before video, leading to error ac-cumulation, latency, and audiovisual misalignment. Existing end-to-endmodels address some of these issues but often lack autoregressive capabil-ities. In this paper, we present VoCa, a unified autoregressive frameworkthat jointly generates speech and talking-head video from text transcriptsand a reference image. VoCa employs a large language model (LLM) tomap text into shared representations, which are concurrently processedby dedicated speech and video decoders to ensure strict synchronization.To bridge the semantic gap between audio-optimized features and hu-man motion, we introduce a speech refiner with a tailored multi-stagetraining strategy. Furthermore, to enable step-by-step autoregressive co-generation, we propose a unified window partitioning mechanism andan associated improved rolling-window denoising strategy. This adaptsa bidirectional diffusion transformer into a causal architecture synchro-nized with the LLM’s autoregressive generation windows. Extensive ex-periments show that VoCa matches or surpasses state-of-the-art baselinesin visual quality, improves audio–lip synchronization, and identity preser-vation. Code and models will be released upon acceptance.

Video-text models adapted from image-text architectures(e.g., CLIP) frequently exhibit temporal blindness—the inability to per-ceive fundamental cues like order, direction, and motion dynamics. Stan-dard datasets mask this limitation by enabling models to exploit staticspatial shortcuts. To systematically evaluate this, we introduce XTE-Bench, a diagnostic probe revealing that even large-scale video-languagemodels struggle with basic temporal reasoning, indicating that param-eter scaling alone is insufficient to resolve this flaw. To address this,we propose Cross-Modal Temporal Edits (XTE), a self-supervisedframework that injects precise temporal supervision. By performing syn-chronized video-text transformations, XTE generates hard temporal neg-atives without manual annotation. We instantiate this with ViTAL-X,a lightweight model that equips frozen image-text backbones with tem-poral awareness while preserving their foundational spatial knowledge.Across six temporal benchmarks, ViTAL-X achieves state-of-the-art per-formance. Utilizing only 0.4B parameters and 1M training clips, ViTAL-X outperforms 7B-parameter models and surpasses baselines trained on600× more data. These results demonstrate that targeted, high-qualitytemporal alignment provides a highly efficient alternative to pure scaling.

Multi-view spatial reasoning remains difficult for currentvision-language models. Even when multiple viewpoints are available,models often underutilize cross-view relations and instead rely on single-image shortcuts, leading to fragile performance on viewpoint transforma-tion and occlusion-sensitive cases. We present ViewFusion, a two-stageframework that explicitly separates cross-view spatial pre-alignment fromquestion answering. In the first stage, the model performs deliberatespatial pre-thinking to infer viewpoint relations and spatial transfor-mations across views, forming an intermediate workspace that goes be-yond a simple re-description. In the second stage, the model conductsquestion-driven reasoning conditioned on this workspace to producethe final prediction. We train ViewFusion with synthetic reasoningsupervision followed by reinforcement learning using GRPO, which im-proves answer correctness while stabilizing the intended two-stage gen-eration behavior. On MMSI-Bench, ViewFusion improves accuracyby 5.3% over Qwen3-VL-4B-Instruct, with the largest gains on exam-ples that require genuine cross-view alignment. Our code is available athttps://github.com/taoxj2001/ViewFusion.

Spatial graphs provide a lightweight and elegant represen-tation of curvilinear anatomical structures such as blood vessels, lungairways, and neuronal networks. Accurately modeling these graphs iscrucial in clinical and (bio-)medical research. However, the high spatialresolution of large networks drastically increases their complexity, result-ing in significant computational challenges. In this work, we aim to tacklethese challenges by proposing VesselTok, a framework that approachesspatially dense graphs from a parametric shape perspective to learn la-tent representations (tokens). VesselTok leverages centerline points witha pseudo radius to effectively encode tubular geometry. Specifically, welearn a novel latent representation conditioned on centerline points to en-code neural implicit representations of vessel-like, tubular structures. Wedemonstrate VesselTok’s performance across diverse anatomies, includ-ing lung airways, lung vessels, and brain vessels, highlighting its ability torobustly encode complex topologies. To prove the effectiveness of Vessel-Tok’s learned latent representations, we show that they (i) generalize tounseen anatomies, (ii) support generative modeling of plausible anatom-ical graphs, and (iii) transfer effectively to downstream inverse problems,such as link prediction.

Recent advances in Large Language Models (LLMs) and Text-to-Image (T2I) models have led to the emergence of Unified Multimodal Models (UMMs), where multimodal understanding and image generation are tightly integrated within a shared architecture. Prior studies suggest that such reciprocity enhances cross-functionality performance through shared representations and joint optimization. However, the safety implications of this tight coupling remain largely unexplored, as existing safety research predominantly analyzes understanding and generation functionality in isolation. In this work, we investigate whether cross-functionality reciprocity itself constitutes a structural source of vulnerability in UMMs. We propose RICE: Reciprocal Interaction–based Cross-functionality Exploitation, a novel attack paradigm that explicitly exploits bidirectional interactions between understanding and generation. Using this framework, we systematically evaluate Generation-toUnderstanding (G→U) and Understanding-to-Generation (U→G) attack pathways, demonstrating that unsafe intermediate signals can propagate across modalities and amplify safety risks. Extensive experiments show high Attack Success Rates (ASR) in both directions, revealing previously overlooked safety weaknesses inherent to UMMs. The code is available at https://github.com/tunantu/UMM-Safety.

Deep learning models have become the standard tool for doc-ument rectification and illumination correction, yet their performance isfundamentally bound by their training data. For nearly a decade, the com-munity has heavily relied on Doc3D, a pioneering but increasingly limiteddocument unwarping dataset in terms of scale and quality. To address thisbottleneck, we introduce SyntheticDoc, a massive, high-quality datasetdesigned to push the boundaries of document unwarping. SyntheticDocis composed of 1,000,000 high-resolution procedurally generated trainingsamples, alongside extensive validation and test sets. Each sample ispaired with rich, pixel-perfect annotations, including UV maps, normalmaps, albedo and shading. To ensure physical accuracy and photorealism,the paper geometries are generated via a physics-based simulator andrendered using a path tracer. To demonstrate the benefit of our dataset,we train a simple baseline model on SyntheticDoc and report on itsperformance in comparison to state-of-the-art methods on both documentunwarping and illumination correction tasks. Our dataset is available athttps://igl.ethz.ch/projects/SyntheticDoc/ and the code used to generateit at https://github.com/tanguymagne/SyntheticDoc.

Despite advances in 3D scene understanding, existing 3DLarge Multimodal Models operate in offline settings, requiring completescene observations or predefined video clips. In this paper, we presentan online 3D vision-language model that enables real-time spatial un-derstanding from streaming video. Our approach adopts an autoregres-sive streaming control modeling based on the LLM’s next-token predic-tion objective to learn when to respond, and employs a lightweight Vi-sual–Spatial Feature Integration (VSFI) module to incrementally injecttemporally aligned geometry priors into the visual stream. To alleviatelong-context decoding overhead, we propose a plug-and-play Geometry-Adaptive Voxel Compression (GAVC) module for efficient visual tokencompression. To address the scarcity of streaming 3D–language data,we further develop a scalable data generation pipeline that curates over1M online spatio-temporal 3D QA pairs and establishes a comprehensivebenchmark spanning 29 tasks. Extensive experiments show that our ap-proach significantly outperforms both proprietary and open-source mod-els across online and offline 3D spatial understanding, reasoning, andgrounding tasks.

Incomplete Multi-View Clustering addresses the problem ofclustering multi-modal data when certain views are missing. Recent end-to-end generative approaches leverage diffusion models to recover miss-ing views via stochastic noise-to-data trajectories. While expressive, suchmechanisms are not explicitly designed for clustering, as they initializefrom cluster-agnostic noise and rely on stochastic denoising dynamics.In this work, we revisit probability path design in end-to-end generativeIMVC. We introduce a flow-matching framework with a linear interpo-lation path between paired view representations, that replaces diffusionwith probability flows between observed and missing views. We providea formal analysis showing that deterministic ODE flows are inherentlybetter aligned with clustering objectives than diffusion-based stochastictrajectories, especially in terms of transport mechanisms that respectclass-conditional data distributions and maintain cluster consistency infinite-step regimes. Building upon this insight, we develop an end-to-end IMVC architecture that integrates straight-path flow-matching viewcompletion with cluster-level and entropy-based alignment to enforcecross-view clustering consistency. Extensive experiments on standardIMVC benchmarks demonstrate that the proposed framework establishesnew state-of-the-art performance.

Storyboarding is a core skill in visual storytelling for film,animation, and games. However, automating this process requires a sys-tem to achieve two properties that current approaches rarely satisfy si-multaneously: inter-shot consistency and explicit editability. While 2Ddiffusion-based generators produce vivid imagery, they often suffer fromidentity drift along with limited geometric control; conversely, traditional3D animation workflows are consistent and editable but require expert-heavy, labor-intensive authoring. We present StoryBlender, a grounded3D storyboard generation framework governed by a Story-centric Re-flection Scheme. At its core, we propose StoryBlender system, whichis built on a three-stage pipeline: (1) Semantic-Spatial Grounding, toconstruct a continuity memory graph to decouple global assets fromshot-specific variables for long-horizon consistency; (2) Canonical AssetMaterialization, to instantiate entities in a unified coordinate space tomaintain visual identity; and (3) Spatial-Temporal Dynamics, to achievelayout design and cinematic evolution through visual metrics. By or-chestrating multiple agents in a hierarchical manner within a verifica-tion loop, StoryBlender iteratively self-corrects spatial hallucinations viaengine-verified feedback. The resulting native 3D scenes support direct,precise editing of cameras and visual assets while preserving unwaver-ing multi-shot continuity. Experiments demonstrate that StoryBlendersignificantly improves consistency and editability over both diffusion-based and 3D-grounded baselines. Code and dataset are available athttps://engineeringai-lab.github.io/StoryBlender.

Understanding visual scenes requires not only recognizing objects but also reasoning about their spatial relationships. Unlike general vision-language tasks, spatial reasoning requires integrating multiple inductive biases, such as 2D appearance cues, depth signals, and geometric constraints, whose reliability varies across contexts. This motivates spatial adaptability: the ability to flexibly coordinate reasoning strategies depending on the input. However, most existing approaches rely on a single reasoning pipeline that implicitly learns a fixed spatial prior, limiting their ability to adapt under distribution changes. Multi-agent systems offer a promising alternative by aggregating diverse reasoning trajectories, but prior spatial reasoning attempts primarily employ homogeneous agents, restricting the inductive-bias diversity they can leverage. In this work, we introduce SpatiO, a heterogeneous multi-agent framework for spatial reasoning that coordinates multiple vision-language specialists with complementary inductive biases. To enable effective collaboration, we propose Test-Time Orchestration (TTO), a calibration mechanism that dynamically evaluates and reweights agents based on their observed reliability during inference, without modifying model parameters. Extensive experiments on diverse spatial reasoning benchmarks, including 3DSRBench, STVQA-7k, CV-Bench, and Omni3D-Bench, demonstrate that SpatiO consistently improves spatial reasoning performance over both closed-source and open-source baselines. The project page is available at https://cy-h1329.github.io/spatio/.

Adapting Multimodal Large Language Models (MLLMs) forhour-long videos is bottlenecked by context window limits. Existing effi-ciency heuristics blindly sacrifice fidelity. They frequently discard tran-sient decisive moments, blur fine-grained evidence, and waste represen-tational bandwidth on irrelevant backgrounds. We propose Tempo, anefficient, query-aware framework that compresses long videos for down-stream understanding. Tempo unifies a Small Vision-Language Model(SVLM) to act as a local temporal compressor. It casts visual token re-duction as an early cross-modal distillation process, generating compact,intent-aligned video representations in a single forward pass. To enforcestrict inference budgets without breaking temporal causality, we intro-duce Adaptive Token Allocation (ATA). Exploiting the SVLM’s inher-ent zero-shot relevance prior and empirical semantic front-loading, ATAacts as a training-free, O(1) dynamic router. It allocates dense band-width to query-critical segments while compressing redundancies downto minimal temporal anchors to maintain the global storyline. Exten-sive experiments demonstrate that our compact 6B architecture achieveshighly competitive performance with aggressive dynamic compression(0.5–16 tokens/frame). On the extreme-long LVBench (4101s), Temposcores 52.3 under a strict 8K visual budget, outperforming proprietarymodels like GPT-4o and Gemini 1.5 Pro. Crucially, empirical profiling re-veals that Tempo frequently compresses hour-long videos to token countsbelow theoretical limits, proving that true long-form understanding relieson intent-driven efficiency rather than greedily padded context windows.

Reinforcement learning (RL) has recently emerged as a promis-ing approach for aligning text-to-image generative models with humanpreferences. A key challenge, however, lies in designing effective and in-terpretable rewards. Existing methods often rely on either compositemetrics (e.g., CLIP, OCR, and realism scores) with fixed weights or asingle scalar reward distilled from human preference models, which canlimit interpretability and flexibility. We propose RubricRL, a simple andgeneral framework for rubric-based reward design that offers greater in-terpretability, composability, and user control. Instead of using a black-box scalar signal, RubricRL dynamically constructs a structured rubricfor each prompt—a decomposable checklist of fine-grained visual cri-teria such as object correctness, attribute accuracy, OCR fidelity, andrealism—tailored to the input text. Each criterion is independently eval-uated by a multimodal judge (e.g., o4-mini), and a prompt-adaptiveweighting mechanism emphasizes the most relevant dimensions. This de-sign not only produces interpretable and modular supervision signalsfor policy optimization (e.g., GRPO or DiffusionNFT), but also enablesusers to directly adjust which aspects to reward or penalize. Experimentswith autoregressive and diffusion text-to-image models demonstrate thatRubricRL improves prompt faithfulness, visual detail, and generalizabil-ity, while offering a flexible and extensible foundation for interpretableRL alignment across text-to-image architectures.

Video world models can generate realistic futures from a sin-gle instruction, but they often fail to track the same physical pointsconsistently across time. As a result, the generated videos appear plausi-ble, yet lack the physical grounding required for reliable action execution,such as robot manipulation. We present GEM-4D, a geometry-groundedvideo world model that resolves this limitation by injecting dense 4Dcorrespondence supervision distilled from a pretrained geometry foun-dation model into the video generative backbone during training. Thissupervision enables the video world model to jointly capture appearanceand geometric structure while retaining a single-stream architecture withno additional inference cost. We further introduce an inverse dynam-ics module that converts correspondence-consistent video rollouts intoexecutable robot trajectories, enabling direct deployment in both real-world and simulated manipulation. GEM-4D achieves state-of-the-artperformance on both video prediction and geometric consistency acrossboth simulation and realistic scenarios and improves real-world manipu-lation success from 61% to 81%. Additional results are available at thehttps://gem-4d.github.io/.

Enabling humanoid robots to respond to human speech withsynchronized and semantically meaningful gestures is fundamental tonatural human-robot interaction. However, this task faces three criticalbarriers: the scarcity of semantically rich datasets, the "modality eclipse"where models ignore audio cues in favor of kinematic inertia, and the sim-to-real gap regarding physical safety. We propose RoboGesture, a robot-centric framework that co-designs data, modeling, and control to powera complete interactive human–humanoid system in which the robot lis-tens, responds, and gestures in real time. We first establish the Robo-Gesture dataset featuring over 300 gesture categories and develop anautomated pipeline to synthesize large-scale collision-free, robot-specificaudio-motion pairs. Our architecture features a Hierarchical Semantic-Acoustic Aligner that extracts multi-granular prosodic and semantic cuesdirectly from raw audio tokens. These cues drive a Streaming ConditionalMotion Generator based on a diffusion transformer with conditional flowmatching. To ensure high responsiveness, we introduce Anti-Inertia CFGMasking, which prevents the model from collapsing into repetitive his-torical patterns by compelling it to proactively mine control signals fromthe audio modality. Finally, an MPC-based safety filter ensures real-time, collision-free execution on physical hardware. Experiments on aUnitree G1 humanoid demonstrate that RoboGesture generates safer,more rhythmic, and more semantically appropriate responses comparedto state-of-the-art baselines.

Existing pruning methods for 3D Gaussian splatting (3DGS)suffer from either severe quality degradation or prohibitive computa-tional overhead. In this paper, we propose REFINE, a highly acceler-ated 3DGS pruning framework centered on a novel rendering-free primi-tive importance metric. Our approach leverages an analytically approx-imated, rendering-aware Hessian field to quantify the expected percep-tual error induced by the removal of individual primitives. By modelingthe joint modulation of visibility, projection geometry and the contentadaptive hyperparameter, we entirely bypass costly forward renderingpasses and derive an anisotropic perceptual weight field that serves asa high-fidelity proxy for primitive importance. Extensive experimentsacross multiple benchmark datasets demonstrate that REFINE main-tains highly competitive rendering quality while achieving a 3, 000× re-duction in pruning-related computational complexity, translating to apractical ∼ 20× speedup in device latency compared to state-of-the-artpruning methods.

Pre-trained Vision Foundation Models (VFMs) have becomecentral to modern computer vision due to their powerful semantic rep-resentations and strong generalization ability. However, their patchifiedor pooled outputs are inherently low-resolution, limiting their effective-ness in tasks requiring fine-grained, pixel-level reasoning. Existing fea-ture upsampling approaches either degrade semantic fidelity or rely onVFM-specific retraining and heavy architectures, hindering efficiency andscalability. To address these challenges, we propose RaysUp, an ultra-lightweight, task-agnostic, and VFM-agnostic feature upsampling frame-work that reconstructs high-resolution feature maps at arbitrary resolu-tions. Unlike conventional 2D interpolation or attention-based schemes,RaysUp lifts feature reconstruction into a geometry-aware ray domain.Specifically, we introduce a Spatially Decoupled Guidance Encoder fordirection-aware guidance encoding, an Any-Resolution Cross-Attentionmechanism for resolution-flexible reconstruction, and a novel Ray Posi-tional Encoding (RayPE) that injects implicit 3D geometric priors via6D Plücker ray coordinates. Finally, A Geometry-Aware NeighborhoodAttention module further ensures content-adaptive bilateral aggregationwhile preserving geometric consistency. Extensive experiments across di-verse dense prediction tasks demonstrate that RaysUp achieves state-of-the-art performance while using only 16% of the parameters of AnyUpand delivering approximately 7× faster inference. These results highlighta substantially improved accuracy–efficiency trade-off and establish Ray-sUp as a practical and scalable solution for universal feature upsampling.Code is available at https://github.com/MAP-RaysUp/RaysUp.

The development of 0.1◦ global weather forecasting modelsbased on machine learning (ML) is constrained by the limited availabil-ity of high-resolution data, as decades of reanalysis are only availableat 0.25◦ resolution. While existing approaches fine-tune 0.25◦ forecastmodels on limited 0.1◦ samples, we show that this transfer is hinderedby the irreversible information loss inherent in coarse-resolution fore-casting. Therefore, we propose BaguanHR, a framework that shiftsthe focus from transferring models to transferring data. We first showthat super-resolution (SR) has lower conditional entropy and input am-plification than forecasting, making it a more robust vehicle for reso-lution transfer. By leveraging this advantage through variable-wise SR,we synthesize extensive 0.1◦ data from ERA5. BaguanHR’s performanceon the synthetic-plus-real dataset exceeds both ML-based methods andIFS-HRES, achieving superior performance across over 85% of the leadtimes within 72 hours. Furthermore, our findings highlight a power-lawscaling effect, as a twofold increase in data reduces RMSE by 4.6% for 72-hour forecasting and 4.9% for 120-hour forecasting. Our results demon-strate that scaling high-resolution ML-based forecasting is primarily adata bottleneck, and that variable-wise super-resolution provides a sim-ple yet general solution to unlock long coarse-resolution reanalyses forhigh-resolution training.

While single-view 3D reconstruction has seen significant pro-gress, extrapolating complex 3D structures from inherently ambiguous2D observations remains fundamentally ill-posed, particularly in the crit-ically underexplored data-scarce regime. To address this challenge, wepropose Point Diffusion Mamba (PDM), a method that integrates thegenerative power of diffusion models with the efficiency of state-spacemodel for single-view 3D reconstruction under data-scarce conditions.Specifically, PDM employs a lightweight reconstruction module tailoredto handle unordered point-cloud inputs effectively. By combining a Lo-cal Geometric Aggregation module with Mamba blocks, our approachjointly models global geometric structures and local details. In 3D re-construction, each point in the initial noisy input requires a precise pre-diction, yet the high-level features extracted by the Mamba module cap-ture only abstract semantic information from sparse points. To bridgethis gap, we introduce the Hierarchical Feature Integration Network,which fuses high-level semantic and local geometric features for eachpoint, overcoming the limitations of token-based point-cloud reconstruc-tion. Furthermore, we propose a Dynamic Weighted Sampling strategythat adaptively unifies 3D generation with single-view reconstruction byleveraging generative priors to enhance reconstruction quality. Exper-imental results on the ShapeNet and Pix3D benchmarks demonstratethat PDM outperforms state-of-the-art methods, providing an effectivesolution for 3D reconstruction under data-scarce settings. Code is avail-able at: https://github.com/NWUzhouwei/PDM.

This paper addresses the fundamental challenge of accelerating the inherently autoregressive sampling process in Flow Matching (FM) models through a numerical systems perspective. We introduce ParaFlow, a training-free framework that recasts sampling as a system of Triangular Nonlinear Equations (TNEs) to enable step-level parallelism. Unlike existing parallel samplers for diffusion models that require 2–3× more function evaluations (NFEs), we propose a velocity approximation scheme that leverages the temporal smoothness of FM trajectories. This allows ParaFlow to perform parallel sampling on a single GPU while requiring less total NFE count to the standard sequential sampler. We theoretically guarantee that our method converges to the exact autoregressive trajectory and that our approximation maintains negligible error bounds. Extensive experiments on Stable Diffusion 3 and Flux demonstrate that ParaFlow achieves up to 4.3× wall-clock speedup with negligible impact on quality, making high-performance parallel sampling truly practical for resource-constrained settings. The source code is released in the supplementary materials. Code is available at https://github.com/Jianrong-Lu/ParaFlow.