ECCV 2026 Papers

Automated segmentation of cryo-electron tomograms rou-tinely produces masks that are voxel-accurate but topologically broken:membranes fragment, organelles merge into one another and enclosedcavities collapse. Existing topology-aware losses reduce these violationsbut cannot eliminate them because topology is encouraged through gradi-ent pressure rather than structurally enforced. We introduce TopoFuse,which reframes topology as a differentiable projection operator rather thana loss penalty. At each forward pass, the projection operator ProjT (a PH-guided sparse edit) identifies the critical voxels responsible for topologicalviolations via bottleneck matching and applies sparse edits to satisfy aspecified topology target (diagram feature counts and lifetime budgets)for dimensions d ∈ {0, 2}. If the projection converges, the output satisfiesthose constraints on the downsampled grid (s = 2), when it does not, arepair certificate exposes this explicitly enabling downstream filtering.A topology prior head predicts the correction target directly from inputfeatures, removing any dependence on ground-truth topology at inference.Across three cryo-ET benchmarks, TopoFuse reduces Betti Number Errorby 54% over the strongest soft-loss baseline (p < 0.001), improves Diceby 4.6 pp and edits only 3.1% of voxels to achieve this. Code and SYNdataset are available at: https://github.com/rohitsalla/TopoFuse

Active Speaker Detection determines whether a visible per-son in a video is speaking at each moment. While recent audio–visualfusion methods perform well on clean data, they degrade under real-world corruptions such as background noise, occlusion, or simultaneousmodality degradation. We attribute this limitation to the absence of ex-plicit consistency constraints that promote robust, semantically alignedrepresentations across modalities. Without such guidance, models tendto learn fragile modality-specific shortcuts that fail under corrupted con-ditions. We propose C 3 ASD, a multi-level consistency-driven frameworkwith three complementary constraints: embedding-level inter-modalityconsistency aligns audio-visual representations during speech; sequence-level intra-modality consistency separates speaking and non-speakingclusters via track-aware contrastive learning; and prediction-level con-sistency stabilizes fusion through knowledge distillation. Extensive ex-periments demonstrate significant improvements under diverse audio, vi-sual and joint corruptions, while maintaining competitive performanceon clean data.

As an important 3D sensing method, photometric stereo estimates surface normals from multiple photos taken under varying lighting. The surface normal quality highly relies on large number of input images, which requires long capturing time, high data burden and limits high-speed applications for photometric stereo. Recently, event-based photometric stereo has emerged as an efficient and rapid approach, but the low resolution of event sensors and the noise of event signals limit its high-quality application. In this paper, we propose DualResPS, a novel photometric stereo pipeline utilizing a frame-event hybrid camera. We design a lighting and capturing strategy tailored to a hybrid-camera setup to utilize observations of different spatial and temporal resolutions. The algorithm explicitly models non-ideal factors including shadow effects and specular reflection to achieve high-resolution, high-quality normal reconstruction. Our proposed DualResPS is validated on both semi-real datasets from the DiLiGenT, DiLiGenT-Pi, and real captured data. The results demonstrate that DualResPS surpasses its frame-based counterpart with 18.1% data bandwidth.

Connector-based video unified models have demonstratedstrong capability in instruction-grounded video synthesis, but integratinga large high-fidelity generator into the unified training loop is computa-tionally prohibitive, limiting achievable visual quality. We therefore pro-pose Lumos-Nexus, a training-efficient unified video generation frame-work that facilitates the development of strong reasoning-driven gen-eration capabilities while significantly enhancing visual fidelity. Lumos-Nexus adopts a two-stage design: 1) During training, only a lightweightgenerator is aligned with the understanding block to learn to take inreasoning-driven semantic control. 2) During inference, we introduceUnified Progressive Frequency Bridging (UPFB) to progressively handoff generation to a high-capacity pretrained generator in the shared la-tent space, enabling coarse-to-fine refinement and producing high-fidelityvideos without compromising reasoning quality. To fill the gap in reasoning-driven video generation benchmarks, we introduce VR-Bench, which as-sesses a model’s capability to translate inferred intent into coherent andsemantically aligned video content. Extensive experiments demonstratethat Lumos-Nexus achieves substantial gains in visual realism and tem-poral coherence on VBench, while exhibiting strong reasoning-based gen-erative performance on VR-Bench.

Vision-language models (VLMs) excel at many tasks, yetcontinue to struggle with spatial reasoning, problems where the key in-formation is not directly observable in the input. Many spatial questionsrequire imaginative perception: simulating an unseen viewpoint, tracinga trajectory through an occluded space, or integrating partial views intoa coherent spatial map. Humans naturally support this kind of reason-ing through imagination. Prior work has introduced intermediate visualrepresentations (e.g., visual thoughts, depth, or box tokens), but theseintermediates often refine structure already visible rather than predict-ing the missing spatial structure implied by the evidence. We introduceImaginative Perception Tokens (IPT), intermediate perceptual rep-resentations that externalize what a VLM would perceive under an alter-native spatial configuration while remaining consistent with the observedinput. To study this capability, we formulate three tasks that requireimaginative perception: Perspective Taking (PET), Path Tracing(PT), and Multiview Counting (MVC). For each task, we constructdatasets of →20K examples spanning simulated and real-world settings,paired with ground-truth intermediate imaginations, final answers, andcurated evaluation benchmarks. Using the unified VLM BAGEL [12] asour backbone, IPT supervision improves spatial reasoning across severalsettings and often outperforms textual chain-of-thought training, evenwhen no image is generated at inference time. For example, on MVC,IPT improves accuracy by 3.4% and achieves performance competitivewith strong closed-source models on Path Tracing. We also find thatmixed training with IPT and label-only data can further improve perfor-mance. In contrast, textual chain-of-thought can be detrimental on thesetasks, substantially degrading performance in some cases, highlighting amodality mismatch when forcing spatial computation through language.Overall, IPT provides a principled supervision signal for reasoning overunobserved structure, yielding stronger spatial generalization and a moreinterpretable intermediate aligned with the underlying geometry of thetask. Code will be released at the project page.Path Tracing (11k)Question: As you move from waypoint 1 to 2. This is what you see looking forward from Question: Which object can you seepoint 1: <image_2>. This is what you see looking forward from point 2: <image_3>. on your right side at waypoint M?Which object can you see on your left side at waypoint M?Options: Imagination1 M 2A. A dining table 2 22B. A fridge MC. A countertop 1D. A sofa 1 13Perspective Taking (55k)Question: After moving to the marked ‘X’ position on the ground while keeping my Question: Will the sideboard appearcurrent orientation, then turning left for 90 degrees, will the sideboard appear closer closer or further away?or further away?Options:A. CloserB. Further XX1 ImaginationMultiview Counting (17k)Question: What is the total number of office chairs in this scene? Question: How many refrigerator(s)are in this area?Answers: 8 Imagination1 23 4Fig. 1: Overview of the three spatial imagination tasks. The left columns showtraining examples with ground-truth imaginative perception; the right columns showevaluation examples.

3D representations are fundamental for scene rendering, un-derstanding, and interaction. Approaches like Neural Radiation Fieldsand 3D Gaussian Splatting achieve impressive photorealistic novel-viewsynthesis, but lack the ability to easily decompose scene elements intoobject level, requiring additional segmentation or grouping. We presentMLP-Splatting, a method that enables scene decomposition via a fewexpressive light-field primitives while providing photorealistic novel-viewsynthesis.MLP-Splatting models each primitive as an independent compact MLPwith localized spatial support that predicts radiance and opacity. In con-trast to low-level Gaussian primitives or a single global radiance field,our neural primitives provide greater expressive capacity while remain-ing spatially localized. Rendering is performed through efficient sparsevolumetric compositing over ray–primitive interactions.Our primitives are supervised using RGB supervision alone, which yieldsprimitives that represent local scene regions often corresponding to ob-jects or object parts, enabling interactive object-level editing withoutsegmentation masks by selecting a handful of primitives. Our method,augmented with optional semantic feature distillation, enables open-vocabulary scene interaction and open-set instant segmentation. Com-pared to state-of-the-art semantic 3DGS methods, we achieve substan-tially lower memory usage (1/7×) and faster rendering (5×).Project Page: https://shinjeongkim.com/mlp-splatting

We introduce Synthetic Visual Genome 2 (SVG2), alarge-scale panoptic video scene graph dataset. SVG2 contains over 636Kvideos with 6.6M objects, 52.0M attributes, and 6.7M relations, pro-viding an order-of-magnitude increase in scale and diversity over priorspatio-temporal scene graph datasets. To create SVG2, we design a fullyautomated pipeline that combines multi-scale panoptic segmentation,online–offline trajectory tracking with automatic new-object discovery,per-trajectory semantic parsing, and GPT-5-based spatio-temporal re-lation inference. Human verification of SVG2 annotation accuracy con-firms its reliability (objects: 93.8%, attributes: 88.3%, relations: 85.4%).Building on this resource, we train TraSeR, a trajectory-groundedvideo scene graph generation model. TraSeR augments VLMs with atrajectory-aligned token arrangement mechanism and new modules: anobject-trajectory resampler and a temporal-window resampler to convertraw videos and panoptic trajectories into compact spatio-temporal scenegraphs in a single forward pass. The temporal-window resampler bindsvisual tokens to short trajectory segments to preserve local motion andtemporal semantics, while the object-trajectory resampler aggregates en-tire trajectories to maintain global context for objects. On PVSG, VIPSeg,VidOR, and SVG2test , TraSeR outperforms the strongest open-sourcebaselines by 15∼20 points in relation detection, 20∼40 points in objectprediction, and 13 points in attribute prediction. It also surpasses GPT-5by 13 points in object prediction and 3 points in attribute prediction.When TraSeR’s generated scene graphs are sent to a VLM for videoquestion answering, it delivers a +1.5∼4.6 absolute accuracy gain overusing video alone or video augmented with Qwen2.5-VL’s generated scenegraphs, demonstrating the utility of explicit spatio-temporal scene graphsas an intermediate representation5 .

While humans naturally gesture during speech, only a sparse subset of these co-speech gestures are visually depictive and semantically linked to specific spoken words. In this paper, we introduce a large-scale dataset – Gesture Recognition in the Wild (GRW), comprising co-speech gestures corresponding to a diverse vocabulary of 155 words. GRW contains 140k manually annotated video clips where the word is spoken, with 17k instances of semantic co-speech gestures including their frame-level temporal boundaries. The video clips are collected ‘in the wild’ from public-facing discourse, including lectures, talk shows, and interviews, covering a diverse range of speakers and visual conditions. We also introduce video models to: (a) classify gestures as semantic or not; (b) recognize the word corresponding to a co-speech gesture; and (c) temporally localize the gesture. These models are trained and evaluated on the GRW dataset and compared against a range of strong baselines, establishing benchmark results for all three tasks. The dataset, annotations, and trained models are publicly available on the project website.

We introduce the Large Sparse Reconstruction Model tostudy how scaling transformer context windows affects feed-forward 3Dreconstruction. Although recent object-centric feed-forward methods pro-duce robust, high-quality reconstructions, they still lag behind dense-view optimization in recovering fine-grained texture and appearance. Weshow that expanding the context window—by substantially increasingthe number of active object and image tokens—narrows this gap andenables high-fidelity 3D object reconstruction and inverse rendering. Toscale effectively, we adapt native sparse attention [68] for 3D reconstruc-tion with three key contributions: (1) an efficient coarse-to-fine pipelinethat focuses computation on informative regions by predicting sparsehigh-resolution residuals; (2) a 3D-aware spatial routing mechanism thatestablishes accurate 2D-3D correspondences using explicit geometric dis-tances rather than standard attention scores; and (3) a custom block-aware sequence-parallel strategy with an All-gather-KV protocol to bal-ance dynamic, sparse workloads across GPUs. As a result, LSRM handles20× more object tokens and >2× more image tokens than prior state-of-the-art (SOTA) methods. Extensive evaluations on standard novel-viewsynthesis benchmarks show substantial gains over the current SOTA,yielding >2.4 dB higher PSNR and >40% lower LPIPS. Furthermore,when extending LSRM to inverse rendering, qualitative and quantitativeevaluations on widely used benchmarks demonstrate consistent improve-ments in texture and geometry details, achieving an LPIPS that matchesor exceeds that of SOTA dense-view optimization methods. Code andmodel weights are available on our project page.

Diffusion models have shown strong potential for multi-modalplanning in end-to-end autonomous driving. However, most existing meth-ods confine diffusion to the planning module, conditioning on fixed out-puts from separate discriminative perception networks. This decoupleddesign propagates perception errors to the planner, increasing optimiza-tion difficulty and reducing robustness. To overcome these limitations, wepropose UniTeD, a Unified Temporal Diffusion framework that jointlymodels perception and planning through iterative denoising in a sharedgenerative space. By enabling bidirectional information exchange, theframework facilitates mutual refinement between tasks and improves ro-bustness via noise-conditioned multi-task training. We further extendthis unified diffusion paradigm to a streaming setting by incorporat-ing temporal context. A Temporal Transition Module (TTM) is intro-duced to resolve the noise-level mismatch between historical and currentframes. In addition, we propose an Anchor Refresh Strategy (ARS) toalleviate the training–inference distribution shift commonly observed insparse diffusion-based end-to-end driving frameworks. Without bells andwhistles, UniTeD achieves state-of-the-art performance across multiplebenchmarks, surpassing both recent discriminative end-to-end methodsand diffusion-based planning approaches.

Situated reasoning often relies on active exploration, yet inmany real-world scenarios such exploration is infeasible due to phys-ical constraints of robots or safety concerns of visually impaired users.Given only a limited observation, can an agent mentally simulate a futuretrajectory toward a target situation and answer spatial “what-if” ques-tions? We introduce WanderDream, the first large-scale dataset designedfor the emulative simulation of mental exploration, enabling models toreason without active exploration. WanderDream-Gen comprises 15.8Kpanoramic videos across 1,088 real scenes from HM3D, ScanNet++,and real-world captures, depicting imagined trajectories from currentviewpoints to target situations. WanderDream-QA contains 158K ques-tion–answer pairs, covering starting states, paths, and end states alongeach trajectory to comprehensively evaluate exploration-based reason-ing. Extensive experiments with world models and MLLMs demonstrate(1) that mental exploration is essential for situated reasoning, (2) thatworld models achieve compelling performance on WanderDream-Gen, (3)that imagination substantially facilitates reasoning on WanderDream-QA, and (4) that WanderDream data exhibit remarkable transferabilityto real-world scenarios. The source code and all data are released athttps://github.com/RuipingL/WanderDream.

Deep neural networks remain vulnerable to imperceptible ad-versarial perturbations, yet efficient robustness certification for seman-tic segmentation is largely unexplored. We propose the first real-timecompatible certifiably robust semantic segmentation framework basedon networks with built-in Lipschitz constraints. Our method achievescompetitive pixel accuracy on Cityscapes while providing determinis-tic robustness guarantees, without the computational overhead of ran-domized smoothing or formal verification. We introduce a generalizedcertification framework for segmentation that computes worst-case per-formance under ℓ2 attacks of radius ϵ across diverse evaluation metrics.Crucially, our approach is approximately 600× faster than randomizedsmoothing at inference on an NVIDIA A100 GPU, delivering comparablecertificates. We validate our worst-case bounds against state-of-the-artadversarial attacks, demonstrating that Lipschitz-constrained architec-tures offer a practical and scalable path toward trustworthy dense pre-diction in safety-critical applications.

Error-diffusion-based binary-defocusing fringe projection pro-filometry has been widely adopted for high-speed 3D imaging of dynamicobjects due to its high measurement efficiency. However, existing errordiffusion methods often neglect the intrinsic relationship between fringepattern structures and the diffusion direction, resulting in systematicphase errors in the generated binary patterns. To overcome this limi-tation, we develop a theoretical model of phase errors in the ditheringalgorithm that elucidates the mechanisms linking phase errors to thediffusion direction. Based on this analysis, a binary fringe generationstrategy is designed. In this approach, fringe patterns in the four-stepphase-shifting method are grouped, and binarization is performed us-ing both the Floyd Steinberg algorithm and its inverse. This processexploits inter-fringe error-compensation effects to enhance measurementaccuracy. Experimental results demonstrate that the proposed methodreduces binarization-induced errors by 23.77% without additional com-putational overhead, confirming its superior performance in terms of bothaccuracy and efficiency.

Zero-Shot Skeleton Action Recognition (ZSAR) aims to rec-ognize unseen action categories by establishing a generalizable mappingbetween skeletal kinematics and semantic representations. However, ex-isting methods frequently struggle with two fundamental issues: (i) theloss of high-frequency dynamics caused by recursive feature aggrega-tion, and (ii) rigid semantic anchoring, which leaves cross-modal align-ments highly vulnerable to semantically irrelevant background noise.To overcome these limitations, we propose the Scale-Aware BipartiteEnergy-guided Registration (Saber) framework, a unified kinematics-driven paradigm for robust cross-modal alignment. Specifically: (i) Scale-Aware Bipartite Encoder actively reshapes the spatio-temporal topol-ogy and applies instance-adaptive filtering to effectively isolate high-frequency motion details, thereby preserving critical structural dynam-ics against “spectral smoothing”. (ii) A Kinetically Driven Probabilis-tic Alignment strategy, comprising energy-guided dynamic focusing anduncertainty-aware probabilistic anchoring, maps part-level representa-tions into variance-adaptive distributions. This mechanism dynamicallyresolves structural uncertainties and robustly anchors physical move-ments to explicit semantics. Extensive evaluations on various bench-mark datasets validate that Saber achieves state-of-the-art performance,demonstrating exceptional robustness and generalization.

While Video Virtual Try-on (VVT) has achieved remark-able progress in synthesizing realistic garment overlays on dynamic sub-jects, existing paradigms remains fundamentally constrained by a pas-sive dependency on source camera trajectories, failing to accommodatethe requisite interactive freedom for omnidirectional viewpoint explo-ration. To address this limitation, we define a pioneering research fron-tier: Camera-controllable Video Virtual Try-on (CaM-VVT). Unlike con-ventional VVT, CaM-VVT not only necessitates viewpoint-agnostic tex-ture hallucination but also strict structural synchronization between non-rigid human dynamics and background contexts under arbitrary, un-constrained camera movements. To tackle these challenges, we presentTryOnCrafter, the first unified DiT-based framework specifically archi-tected for the CaM-VVT task. Departing from implicit pixel-space ma-nipulation, we introduce a Renderable 4D Try-on Proxy that explicitlydecouples the human subject from the environment. This is achieved bydistilling high-fidelity 2D try-on priors into a clothed 3DGS-based avatar,which is subsequently animated via SMPL-X sequences and metric-alignedinto a reconstructed background point cloud. This proxy establishes a ro-bust structural foundation with superior texture density and motion in-tegrity. Our Proxy-Anchored Video DiT leverages this robust structuralfoundation as a primary geometric anchor, ensuring that the synthesizedphotorealistic videos are strictly constrained by prescribed trajectoriesand physically plausible deformations. Benefiting from the inherent ed-itability of the 4D proxy, TryOnCrafter facilitates diverse downstreamapplications, including human relocalization, “bullet time” effects, and360-degree orbital viewing. Extensive experiments on our establishedCaM-VVTBench demonstrate that TryOnCrafter significantly outper-forms existing baselines in preserving structural consistency and garmentidentity across complex camera maneuvers.

Reconstructing deformable objects from video requires a 4Drepresentation that should preserve geometry, explain motion throughphysics, and generalize to future or new physical conditions. Existingdynamic NeRF and Gaussian methods achieve strong view synthesis,yet their voxel or ellipsoidal primitives are primarily designed for ap-pearance rendering and typically driven by centers or predefined par-ticle bindings, limiting physically meaningful non-uniform deformationand sharp boundary evolution. We present PhysConvex, a physics-informed dynamic convex field for video-based reconstruction, physicalsystem identification, and simulation. PhysConvex represents a dynamicobject as material-space deformable convex primitives whose boundaryis advected by physical dynamics. The boundary-driven convex expressesnon-uniform deformation and evolving active supports while serving si-multaneously as a rendering element, deformation carrier, and physicalsupport where mass, elastic response, forces, and contacts are evalu-ated. We further introduce a mesh-free reduced-order convex simulatorin which neural skinning modes define physics-based deformation basesdirectly over deformable convex supports. Experiments show improveddynamic and physical reconstruction, efficiency, future prediction, andgeneralization to changed materials, forces, and boundary conditions.

Group Relative Policy Optimization (GRPO) has emergedas a powerful framework for preference alignment in text-to-image (T2I)flow models. However, we have observed that the standard paradigmthat evaluates a group of generated samples against a single conditionsuffers from insufficient exploration of inter-sample relationships, con-straining both alignment efficacy and performance ceilings. To addressthis sparse single-view evaluation scheme, we propose Multi-View GRPO(MV-GRPO), a novel algorithm that enhances relationship explorationby augmenting the condition space to create a dense multi-view rewardmapping. Specifically, for a group of samples generated from one prompt,MV-GRPO leverages a flexible Condition Enhancer to generate seman-tically adjacent yet diverse captions. These captions enable multi-viewadvantage re-estimation, capturing diverse semantic attributes and pro-viding richer optimization signals. By deriving the probability distribu-tion of the original samples conditioned on these new captions, they canbe incorporated into the training process without costly sample regen-eration. Extensive experiments demonstrate that MV-GRPO achievessuperior alignment performance over state-of-the-art methods.

Learning with noisy labels in large-scale classification re-mains challenging. Existing forward-correction methods estimate class-dependent label noise via a transition matrix, but this matrix is typ-ically assumed to be full-rank and unconstrained. However, real-worldlabel noise is typically structured and exhibits low-rank properties dueto semantic clustering, especially when the number of classes is large.This mismatch between modeling assumptions and the intrinsic struc-ture of label noise leads to unstable optimization and degraded general-ization performance. Motivated by this observation, we propose a novelLow-Effective-Rank Noisy-Label Learning (LENL) framework, whichexplicitly constrains the transition matrix via nuclear-norm regulariza-tion. This low-rank inductive bias captures structured confusion pat-terns while suppressing spurious noise modes, yielding both statisticalefficiency and improved generalization. We further provide theoreticalanalysis establishing the benefits of rank-constrained estimation understructured noise assumptions. To enable stable and scalable optimiza-tion, we introduce a Newton–Schulz optimizer tailored for label-noiselearning, which approximates the nuclear norm through decomposition-free iterative updates, avoiding the numerical instability and computa-tional overhead of explicit singular value decomposition. Extensive exper-iments on CIFAR-100, CIFAR-100N, ImageNet-1K, and Noisy Ostracodsdemonstrate consistent and significant improvements over state-of-the-art label-noise learning methods.

Interactive simulators have become powerful tools for train-ing embodied agents and generating synthetic visual data, but existingphotorealistic simulators suffer from limited generality, programmabil-ity, and rendering speed. We address these limitations by introducingSPEAR: A Simulator for Photorealistic Embodied AI Research. At itscore, SPEAR is a Python library that can connect to, and program-matically control, any Unreal Engine (UE) application via a modularplugin architecture. SPEAR exposes over 14K unique UE functions toPython, representing an order-of-magnitude increase in programmablefunctionality over existing UE-based simulators. Additionally, a singleSPEAR instance can render 1920×1080 photorealistic beauty images di-rectly into a user’s NumPy array at 73 frames per second – an order ofmagnitude faster than existing UE plugins – while also providing groundtruth image modalities that are not available in any existing UE-basedsimulator (e.g., a non-diffuse intrinsic image decomposition, material IDs,and physically based shading parameters). Finally, SPEAR introduces anexpressive high-level programming model that enables users to specifycomplex graphs of UE work with arbitrary data dependencies amongwork items, and to execute these graphs deterministically within a sin-gle UE frame. We demonstrate the utility of SPEAR through a diversecollection of example applications: controlling multiple embodied agentswith distinct action spaces (e.g., humans, cars, and robots) across sev-eral in-the-wild UE projects; rendering photorealistic city-scale environ-ments; manipulating UE’s procedural content generation systems; ren-dering synchronized multi-view images of detailed human faces; coordi-nating an interactive co-simulation with the MuJoCo physics simulator;and editing scenes with natural language via an AI coding assistant.