ECCV 2026 Papers

This paper introduces an inverse-rendering framework fornatural-light, uncalibrated photometric stereo (PS) that leverages back-ground cues captured alongside the target object. Natural-light PS (NaPS)acquires shading variations by moving or rotating the camera and objectunder fixed, uncontrolled illumination, such as indoor lighting, whilemaintaining their relative geometry. However, uncalibrated NaPS, inwhich the lighting conditions are unknown, remains inherently ill-posed.To tackle this challenge, we propose explicitly reconstructing the lightingenvironment from the image background, which is typically masked outin prior work, thereby converting uncalibrated NaPS into a tractableinverse-rendering problem. Specifically, we move and rotate the camera-object pair while keeping their relative pose fixed, and reconstruct thesurrounding illumination directly from the observed background usinga 3D Gaussian Splatting (3DGS) representation. We then optimize thetarget object’s shape and reflectance via inverse rendering under thereconstructed illumination. Experimental results demonstrate that ourinverse-rendering-based approach yields more accurate estimates of bothgeometry and reflectance than existing learning-based PS methods, espe-cially under realistic near-field indoor conditions.

Text Image Machine Translation (TIMT) aims to translatetext embedded in images in the source-language into target-language,requiring synergistic integration of visual perception and linguistic un-derstanding. Existing TIMT methods, whether cascaded pipelines orend-to-end multimodal large language models (MLLMs), struggle withhigh-resolution text-rich images due to cluttered layouts, diverse fonts,and non-textual distractions, resulting in text omission, semantic drift,and contextual inconsistency. To address these challenges, we proposeGLoTran, a global-local dual visual perception framework for MLLM-based TIMT. GLoTran integrates a low-resolution global image withmulti-scale region-level text image slices through pre-local region trans-lation replay and an instruction-guided alignment strategy, conditioningMLLMs to maintain scene-level contextual consistency while faithfullycapturing fine-grained textual details. Moreover, to realize this dual-perception paradigm, we construct GLoD, a large-scale text-rich TIMTdataset comprising 510K high-resolution global-local image-text pairscovering diverse real-world scenarios. Extensive experiments demonstratethat GLoTran substantially improves translation completeness and ac-curacy over state-of-the-art MLLMs.

Event cameras enable high-frequency visual perception withmicrosecond latency, offering advantages for dynamic scenes. However,event-based small object detection remains challenging due to sparseasynchronous measurements and weak object responses that are easilydisrupted by noise. Limited spatial support causes small-object signalsto lose temporal continuity, resulting in fragmented and unstable pre-dictions. To address this issue, we propose a physics-guided advection-consistent modeling framework, termed PACT, which formulates eventevolution as a motion-driven feature transport process. Instead of rely-ing solely on local spatio-temporal aggregation, PACT propagates fea-tures along estimated velocity fields and enforces trajectory-level consis-tency through advection constraints. This design preserves weak eventresponses over time and prevents their degradation under complex back-ground interference. Technically, PACT integrates motion-aware featureextraction with a differentiable advection-based transport operator, en-abling coherent motion representation and effective noise suppressionduring temporal evolution. Extensive experiments on benchmark event-based datasets demonstrate that PACT consistently outperforms state-of-the-art methods, achieving improvements of 20.72% in IoU and 15.03%in accuracy while maintaining comparable computational efficiency. Thecode will be made publicly available.

Generating human motion synchronized with speech is es-sential for creating realistic virtual avatars. While recent work has madestrides in generating gestures from speech, these approaches often fallshort in enabling avatars to perform meaningful, context-specific actions.This paper presents CtrlCoMo, a motion generation framework that usestextual input alongside speech audio to provide explicit control over ac-tions in co-speech scenarios. To address the interference between ges-tures and actions in co-speech motions, we introduce Pyramid-VQ, anautoencoder that separates gestures from actions through hierarchicalquantization, with shallow layers capturing global semantics and deeplayers encoding localized gestures, guided by layer-wise CLIP-based se-mantic regularization. Additionally, the model provides explicit controlover gesture intensity via an AdaLN mechanism. For evaluation, we intro-duce CoHuMo, a large-scale co-speech motion dataset of 370 hours withrich human actions. This dataset encompasses a wide range of scene typesand languages, providing a robust benchmark for future research. Exten-sive evaluations are conducted on CoHuMo which validate our model’seffectiveness.

Concept Bottleneck Models (CBMs) provide interpretabilityby routing predictions through a layer of human-understandable con-cepts. However, defining an optimal concept set for a specific datasetremains an open challenge. Existing approaches rely on expensive ex-pert annotations or LLM-generated lists based solely on class names.Even “open-vocabulary” variants typically depend on static concept sets,which restrict discovery and introduce label bias. Furthermore, tradi-tional CBMs often suffer from information leakage, where unmodeledvisual features bypass the bottleneck and compromise the integrity ofthe explanations. To overcome these limitations, we propose CaptionBottleneck Models (CaBM), a framework that circumvents the need forpredefined concept sets by replacing rigid concept layers with free-formnatural language. By representing images via LMM-generated captionsand training a classifier strictly on this text, CaBM ensures a leakage-freearchitecture by construction. Additionally, by analyzing the text classi-fier post-training, CaBM autonomously discovers high-quality, dataset-specific concepts. Our results across fine- and coarse-grained benchmarksdemonstrate that CaBM achieves competitive accuracy while preservinginterpretability without the constraints of external dictionaries or manuallabeling. Our code is available at https://github.com/bariscagliyan/CaptionBottleneckModels.

While feed-forward 3D reconstruction models have advancedrapidly, they still exhibit degraded performance on panoramas due tospherical distortions. Moreover, existing panoramic 3D datasets are pre-dominantly collected with 360◦ cameras fixed at discrete locations, re-sulting in discontinuous trajectories. These limitations critically hin-der the development of panoramic feed-forward 3D reconstruction, espe-cially for the multi-view setting. In this paper, we present Holo360D,a comprehensive dataset containing 109,495 panoramas paired with reg-istered point clouds, meshes, and aligned camera poses. To our knowl-edge, Holo360D is the first large-scale dataset that provides continuouspanoramic sequences with accurately aligned high-completenessdepth maps. The raw data are initially collected using a 3D laser scan-ner coupled with a 360◦ camera. Subsequently, the raw data are pro-cessed with both online and offline SLAM systems. Furthermore, to en-hance the 3D data quality, a post-processing pipeline tailored for the 360◦dataset is proposed, including geometry denoising, mesh hole filling, andregion-specific remeshing, etc. Finally, we establish a new benchmark byfine-tuning 3D reconstruction models on Holo360D, providing key in-sights into effective fine-tuning strategies. Our results demonstrate thatHolo360D delivers superior training signals and provides a comprehensivebenchmark for advancing panoramic 3D reconstruction models. Datasetsand Code will be made publicly available. Dataset Page: Holo360D

Despite modeling temporal motion, dynamic 3D GaussianSplatting (3DGS) methods still inherit a static densification strategy ill-suited for dynamic scenes. This neglect of temporal behavior leads tounder-reconstructed and blurry dynamic regions, as short-lived Gaus-sians receive sparse supervision and fail to densify effectively. We pro-pose a Visibility-Aware Densification (VAD) framework that integratestemporal visibility into the densification process, ensuring that Gaus-sians are refined based on their actual temporal presence. A Temporally-Adaptive Thresholding (TAT) mechanism further adjusts each Gaussian’sdensification threshold according to its temporal lifespan, promoting bal-anced refinement of both static and dynamic regions. Finally, a TemporalOffset Warping (TOW) design enhances deformation capacity aroundtemporal centers, extending the lifespan of highly dynamic Gaussiansand facilitating more effective densification. Our approach achieves sub-stantial improvements in the visual quality of dynamic regions, outper-forming existing methods across three dynamic multi-view benchmarkdatasets. Moreover, the proposed VAD module generalizes across diversedynamic 3DGS methods, consistently improving dynamic reconstructionas a plug-and-play component. The project page is available here.

Humanoid motion tracking is central to teleoperation andwhole-body imitation, yet evaluation often disagrees with what peo-ple perceive in videos. Kinematic errors average per-frame pose differ-ences but miss the physical artifacts that matter most, such as unstablesupport and incorrect contacts (e.g., foot skating and mistimed touch-downs). Meanwhile, widely used test suites are small and lack the diver-sity needed to stress contact-rich, long-horizon behaviors. We introduce# Corresponding author.HumanTracker to make humanoid tracking evaluation both perceptu-ally aligned and scalable. HumanTracker contributes 150 hours of newlycaptured optical motion from 24 professional performers, organized intofour motion families with text labels for fine-grained diagnosis. We fur-ther propose HumanScore, a preference-aligned metric trained from 12Khuman-labeled motion pairs on synchronized tracking videos via a trajec-tory reward model. Across representative state-of-the-art trackers, Hu-manScore better predicts held-out human preferences and reveals contactand stability failures that kinematic metrics often miss. The project pageis available at https://dairuliu.github.io/humantracker.

Event-based vision, characterized by low redundancy, focuson dynamic motion, and inherent privacy-preserving properties, natu-rally fits the demands of video anomaly detection (VAD). However, theabsence of dedicated event-stream anomaly detection datasets and effec-tive modeling strategies has significantly hindered progress in this field.In this work, we take the first major step toward establishing event-based VAD as a unified research direction. We first construct multi-ple simulated event-stream based benchmarks for video anomaly de-tection, featuring synchronized event and RGB recordings. Leveragingthe unique properties of events, we then propose an EVent-centric spa-tiotemporal Video Anomaly Detection framework, namely EWAD, withthree key innovations: an event density aware dynamic sampling strategyto select temporally informative segments; a density-modulated tempo-ral modeling approach that captures contextual relations from sparseevent streams; and an RGB-to-event knowledge distillation mechanismto enhance event-based representations under weak supervision. Exten-sive experiments on three benchmarks demonstrate that our EWADachieves significant improvements over existing approaches, highlight-ing the potential and effectiveness of event-driven modeling for videoanomaly detection. The benchmark datasets and code are available athttps://github.com/kanyutingfeng/EWAD.

Building robust 3D perception for self-driving still relies heav-ily on large-scale data collection and manual annotation, yet this paradigmbecomes impractical as deployment expands across diverse cities andregions. Meanwhile, modern cities are increasingly instrumented withroadside units (RSUs), static sensors deployed along roads and at inter-sections to monitor traffic. This raises a natural question: can the cityitself help train the vehicle? We propose infrastructure-taught, label-free3D perception, a paradigm in which RSUs act as stationary, unsuper-vised teachers for ego vehicles. Leveraging their fixed viewpoints and re-peated observations, RSUs learn local 3D detectors from unlabeled dataand broadcast predictions to passing vehicles, which are aggregated aspseudo-label supervision for training a standalone ego detector. The re-sulting model requires no infrastructure or communication at test time.We instantiate this idea as a fully label-free three-stage pipeline and con-duct a concept-and-feasibility study in a CARLA-based multi-agent en-vironment. With CenterPoint, our pipeline achieves 82.3% AP for detect-ing vehicles, compared to a fully supervised ego upper bound of 94.4%.We further systematically analyze each stage, evaluate its scalability, anddemonstrate complementarity with existing ego-centric label-free meth-ods. Together, these results suggest that city infrastructure itself canpotentially provide a scalable supervisory signal for autonomous vehi-cles, positioning infrastructure-taught learning as a promising orthogonalparadigm for reducing annotation cost in 3D perception.

Trajectory forecasting for autonomous driving has advancedrapidly, yet representative models often produce uninformative posteri-ors over forecast modes, causing problems for mode pruning. We tracethis to a modeling-training mismatch: forecasters are typically modeledas conditional Gaussian mixture models (GMMs) but trained with awinner-take-all (WTA) loss that assigns each sample to its nearest mode.We argue that this K-means-like hard assignment (one-hot), while pre-venting mode collapse, is the source of uninformative mode probabil-ities: it over-segments the trajectory space, ignores relatedness amongnearby modes, and yields assignment instability under small perturba-tions. Guided by this lens, we introduce two post-hoc treatments: (1)test-time posterior-weighted merging that aggregates nearby candidatetrajectories; and (2) a one-step expectation-maximization (EM) updatethat replaces hard labels with soft responsibilities, sharing probabilitymass across neighboring modes. Across several WTA-trained architec-tures, these lightweight steps produce more informative, faithfully rankedmode posteriors and strengthen final forecasts on popular displacementmetrics—without retraining. Our analysis unifies recent design choicesthrough a GMM-vs-K-means perspective and offers principled, practicalcorrections that better align training objectives with inference.

High dynamic range (HDR) novel view synthesis (NVS) aimsto reconstruct HDR scenes from multi-exposure low dynamic range (LDR)images. Existing HDR pipelines heavily rely on known camera poses,well-initialized dense point clouds, and time-consuming per-scene op-timization. Current feed-forward alternatives overlook the HDR prob-lem by assuming exposure-invariant appearance. To bridge this gap, wepropose InstantHDR, a feed-forward network that initializes 3D HDRscenes from uncalibrated multi-exposure LDR collections in a fast sin-gle forward pass. Specifically, we design a geometry-guided appearancemodeling for multi-exposure fusion, and a meta-network for generaliz-able scene-specific tone mapping. Due to the lack of HDR scene data,we build a pre-training dataset, called HDR-Pretrain, for generalizablefeed-forward HDR models, featuring 168 Blender-rendered scenes, di-verse lighting types, and multiple camera response functions. Compre-hensive experiments show that our InstantHDR delivers a single-forwardHDR initialization at ∼ 700× the speed of SoTA optimization-basedmethods, and reaches comparable quality in real settings after lightweightpost-optimization while remaining ∼ 20× faster. All code, models, anddatasets: https://github.com/Bugjudger/InstantHDR.

Modeling dexterous hand–object interactions is challengingas it requires understanding how subtle finger motions influence theenvironment through contact with objects. While recent world mod-els address interaction modeling, they typically rely on coarse actionspaces that fail to capture fine-grained dexterity. We, therefore, intro-duce DexWM, a Dexterous Interaction World Model that predicts futurelatent states of the environment conditioned on past states and dexterousactions. To overcome the scarcity of finely annotated dexterous datasets,DexWM represents actions using finger keypoints extracted from ego-centric videos, enabling training on over 900 hours of human and non-dexterous robot data. Further, to accurately model dexterity, we find thatpredicting visual features alone is insufficient; therefore, we incorporatean auxiliary hand consistency loss that enforces accurate hand config-urations. DexWM outperforms prior world models conditioned on text,navigation, or full-body actions in future-state prediction and demon-strates strong zero-shot transfer to unseen skills on a Franka Panda armwith an Allegro gripper, surpassing Diffusion Policy by over 50% on av-erage across grasping, placing, and reaching tasks. Codes and dataset areavailable on the project page: https://raktimgg.github.io/dexwm/.

3D reconstruction of articulated objects from a single im-age is challenging because large training datasets with paired image and3D supervision are difficult to obtain. Recent point map–based methodsachieve strong performance but rely on synthetic datasets rendered frommanually created articulated 3D assets with carefully curated pose dis-tributions. While camera viewpoints can be easily sampled, generatingrealistic object articulations remains costly and labor-intensive. We pro-pose a training framework that reduces this requirement by leveragingunannotated 2D images collections with only a single rigged canonicalmesh per category. Starting from a weak 3D shape predictor trained oncanonical-pose renders, we iteratively estimate object articulation andcamera pose by fitting the mesh to predicted point maps. The recov-ered articulations and viewpoints are then used to render updated syn-thetic training data, progressively improving the predictor. Despite us-ing substantially weaker 3D supervision, our models achieve performancecomparable with DualPM, which requires manually curated articulatedtraining datasets.

Building robust world models for visual reinforcement learn-ing remains challenging under complex visual conditions. When facedwith task-irrelevant visual distractions, the representations learned byworld models tend to collapse. Pre-trained vision models (PVMs) providestrong visual priors and structured representations, yet those learned forvisual perception are incompatible with dynamic control in RL. In thiswork, we present Vision-Priors-Aligned World Model (VPA-WM1 ): aunified framework that organically integrates PVMs and world models toachieve robust visual control. We first introduce Control-Oriented Align-ment, which effectively guides PVMs to focus on control-relevant signalsthrough two auxiliary objectives: instant action prediction and short-horizon return estimation. We then design Spatio-Temporal Collabora-tive Integration mechanism, which guides the world model to adaptivelyattend to the visual cues of PVMs through two complementary designs(spatially grounded guidance and experience query integration), therebylearning representations that are visually robust and dynamically con-sistent. Superior performance on visually-distracted control benchmarksdemonstrates that VPA-WM effectively bridges the semantic gap be-tween visual perception and dynamic control, and successfully leveragesvisual priors to enhance the robust control of world models.

We introduce LEGO for advanced open-vocabulary scene un-derstanding. Beyond basic concept recognition, its core innovation lies incapturing the intrinsic semantic hierarchies within the scene, such as the"flowerpot → bouquet → bud → petal" lineage. While foundation mod-els like SAM can identify multi-granular structures in 2D, their partitionsare strictly perspective-bound and lack cross-view consensus. LEGO self-adaptively re-grades volatile multi-view SAM granularities into a unified,3D-consistent hierarchy. This provides precise supervision for the struc-turally coherent, multi-level segmentation of 3D scenes. By groundingthese segments with CLIP embeddings, LEGO recovers open-vocabularysemantic logic across hierarchical levels. Furthermore, by incorporatingspatial relationships, we elevate these segments into level-wise languagescene graphs, effectively empowering Large Language Models to performcomplex, context-aware spatial reasoning and precise visual grounding.Experimental results demonstrate that LEGO establishes new state-of-the-art performance across both promptable and open-vocabulary 3Dsegmentation benchmarks, exhibiting advanced hierarchical scene de-composition and context-aware spatial reasoning. Project page: https://pz0826.github.io/LEGO-Webpage/

Diffusion and flow-based generative models produce strong images, yet their controllability remains largely endpoint-centric: users specify conditions and receive final outputs, while the intermediate generative dynamics remain hidden. Recent methods have begun to exploit generation order and process decomposition to improve sample quality, but still treat intermediate states as internal computation rather than objects for interaction. We propose Trajectory Forcing (TF), a trajectory-centric framework that makes the generation path explicit, semantic, and editable. TF organizes synthesis as a sequence of semantically structured stages, progressing from global layout to object-, part-, and detail-level representations. Each stage produces a decodable latent state that can be inspected, evaluated, and locally edited before the next stage begins. To instantiate this path, we derive coarse-to-fine teacher hierarchies by clustering pretrained visual representations such as DINOv2, and train a hierarchy-conditioned one-step flow-matching model at each level. We further introduce trajectory-aware metrics that measure structural consistency and local controllability beyond endpoint quality metrics such as FID. Experiments show that TF achieves competitive sample quality while exposing coherent intermediate states and supporting localized edits across semantic levels. By shifting the focus from final images to the generative path itself, TF opens a route toward controllable, trajectory-aware image synthesis.

Modern generative video models excel at producing convinc-ing, high-quality outputs, but struggle to maintain multi-view and spa-tiotemporal consistency in highly dynamic real-world environments. Inthis work, we introduce AnyView, a diffusion-based video generationframework for dynamic view synthesis with minimal inductive biasesor geometric assumptions. We leverage multiple data sources with var-ious levels of supervision, including monocular (2D), multi-view static(3D) and multi-view dynamic (4D) datasets, to train a generalist spa-tiotemporal implicit representation capable of producing zero-shot novelvideos from arbitrary camera locations and trajectories. We evaluateAnyView on standard benchmarks, showing competitive results with thecurrent state of the art, and propose AnyViewBench, a challengingnew benchmark tailored towards extreme dynamic view synthesis in di-verse real-world scenarios. In this more dramatic setting, we find thatmost baselines drastically degrade in performance, as they require signif-icant overlap between viewpoints, while AnyView maintains the abilityto produce realistic, plausible, and spatiotemporally consistent videoswhen prompted from any viewpoint.

Aerial 6DoF localization typically relies on precise GNSSsignals or radiometrically rich 3D reconstructions, limiting scalability andon-board deployment. We propose SemCityLoc, a semantic–geometricalignment system that reframes aerial pose estimation as structuredsurface registration between foundation-model-derived visual priors andstandardized LoD-compliant 3D city models. Instead of matching sparsecontours or dense texture, our method aligns semantic surfaces and monoc-ular depth with lightweight semantic 3D building models, increasing pose* Corresponding authordiscriminability in repetitive and occluded urban environments. To enableaccurate evaluation, we introduce SemCityLockeD, the first real-worldbenchmark combining centimeter-accurate UAV poses with standardizedLoD1–LoD3 semantic city models and challenging low-altitude imagery.Experiments demonstrate substantial improvements over existing map-based approaches, improving recall by up to 36% and reducing meanpositional error from 9.89 m to 2.62 m in challenging urban canyons. Ourresults indicate that semantically structured geometry provides sufficientand scalable constraints for high-precision aerial localization withoutradiometric scene reconstructions. The code and data are available athttps://albertchen98.github.io/SemCityLoc.