ECCV 2026 论文列表

Dynamical systems theory and reinforcement learning viewworld evolution as the dynamics of latent states driven by actions, withvisual observations providing partial information about those states. Re-cent video world models attempt to learn these action-conditioned dy-namics from data. However, existing datasets rarely meet these require-ments: they typically lack diverse and semantically meaningful actionspaces, and actions are directly tied to visual observations rather thanmediated by underlying states. As a result, actions are often entangledwith pixel-level changes, making it difficult for models to learn structuredworld dynamics and maintain consistent evolution over long horizons. Inthis paper, we propose WildWorld, a large-scale action-conditioned worldmodeling dataset with explicit state annotations, automatically collectedfrom a photorealistic AAA action role-playing game (Monster Hunter:Wilds). WildWorld contains over 108 million frames and features morethan 450 actions, including movement, attacks, and skill casting, togetherwith synchronized per-frame annotations of character skeletons, worldstates, camera poses, and depth maps. We further derive WildBench toevaluate models through Action Following and State Alignment. Exten-sive experiments reveal persistent challenges in modeling semanticallyrich actions and maintaining long-horizon state consistency, highlightingthe need for state-aware video generation.

Single-photon LiDAR (SP-LiDAR) simulators face a dilemma:fast but inaccurate Poisson models or accurate but prohibitively slow se-quential models. This paper breaks that compromise. We present a sim-ulator that achieves both fidelity and speed by focusing on the critical,yet overlooked, component of simulation: the photon count statistics. Ourkey contribution is a Markov-renewal process (MRP) formulation that,for the first time, analytically predicts the distribution of registered pho-ton counts under dead time. To make this MRP model computationallytractable, we introduce a spectral truncation rule that efficiently com-putes the complex covariance statistics. By proving the shift-invarianceof the process, we extend this per-pixel model to full histogram cubegeneration via a precomputed lookup table. Our method generates 3Dcubes indistinguishable from the sequential gold standard, yet is ordersof magnitude faster. This finally enables large-scale, physically faithfuldata generation for learning-based SP-LiDAR reconstruction. Projectpage: https://mars-splidar.github.io/.

Understanding emotions is a fundamental ability for intelligent systems to be able to interact with humans. Vision-language models (VLMs) have made tremendous progress in the last few years for many visual tasks, potentially o!ering a promising solution for understanding emotions. However, it is surprising that even the most sophisticated contemporary VLMs struggle to recognize human emotions or to outperform even specialized vision-only classifiers. In this paper we ask the question ‘Why do VLMs struggle to recognize human emotions?’, and observe that the inherently continuous and dynamic task of facial expression recognition (DFER) exposes two critical VLM vulnerabilities. First, emotion datasets are naturally long-tailed, and the web-scale data used to pre-train VLMs exacerbates this head-class bias, causing them to systematically collapse rare, under-represented emotions into common categories. We propose alternative sampling strategies that prevent favoring common concepts. Second, temporal information is critical for understanding emotions. However, VLMs are unable to represent temporal information over dense frame sequences, as they are limited by context size and the number of tokens that can fit in memory, which poses a clear challenge for emotion recognition. We demonstrate that the sparse temporal sampling strategy used in VLMs is inherently misaligned with the fleeting nature of micro-expressions (0.25–0.5 seconds), which are often the most critical a!ective signal. As a diagnostic probe, we propose a multi-stage context enrichment strategy that utilizes the information from ‘in-between’ frames by first converting them into natural language summaries. This enriched textual context is provided as input to the VLM alongside sparse keyframes, preventing attentional dilution from excessive visual data while preserving the emotional trajectory.

Recent advances in 3D Gaussian Splatting (3DGS) have ach-ieved significant success in high-fidelity Novel View Synthesis (NVS),yet the optimization process still introduces noisy Gaussian primitivesdue to the sparse and incomplete initialization from Structure-from-Motion (SfM) point clouds. Most existing methods focus only on adjust-ing the positions of primitives during optimization, while ignoring theunderlying spatial structure. To this end, we introduce a new perspec-tive by formulating the optimization of 3DGS as a primitive denoisingprocess and propose Denoising-GS, a spatial-aware denoising frame-work for Gaussian primitives by taking both the positions and spatialstructure into consideration. Specifically, we design an optimizer thatpreserves the spatial optimization flow of primitives, facilitating coher-ent and directed denoising rather than random perturbations. Buildingupon this, the Spatial Gradient-based Denoising strategy jointlyconsiders the spatial supports of primitives to ensure gradient-consistentupdates. Furthermore, the Uncertainty-based Denoising module es-timates primitive-wise uncertainty to prune redundant or noisy primi-tives, while the Spatial Coherence Refinement strategy selectivelysplits primitives in sparse regions to maintain structural completeness.Experiments conducted on three benchmark datasets demonstrate thatDenoising-GS consistently enhances NVS fidelity while maintaining rep-resentation compactness, achieving state-of-the-art performance acrossall benchmarks. Source code and models will be made publicly available.(a) (b) (c) (d)Point Cloud Spatial Gradient- Uncertainty-basedPosition DenoisingDenoising based Denoising Denoising↓ optimize and denoise ↓ ↓calculate uncertainty and prune↓Fig. 1: Illustrative comparison of different denoising methods: (a) Point cloud denois-ing, which moves noisy points toward the GT surface; (b) Position denoising of Gaus-sian primitives, which adjusts the mean positions of noisy primitives toward the GTsurface; (c) Spatial gradient-based denoising of Gaussian primitives, which optimizesboth the mean positions and the spatial structure of primitives based on gradients; (d)Uncertainty-based denoising of Gaussian primitives, which estimates the uncertaintyof each primitive and prunes those with high uncertainty.

Self-supervised learning (SSL) has produced a diverse land-scape of vision transformers (ViTs) whose pretrained representations sup-port a wide range of downstream tasks. Towards a better understandingof these models, a body of works has assessed the mechanics of their self-attention as well as which types of information they capture across theirrepresentations, revealing, e.g., stark differences between models trainedwith contrastive learning (CL) vs. masked image modeling (MIM). How-ever, the total of these advances on model understanding has to datenot yet fully permeated a larger community, where, e.g., insights thatare specific to CL models are still at times generalized to MIM models.To make model understanding straightforward and intuitive for a broadcommunity, we propose a simple and easily interpretable visualizationprotocol. Our protocol is based on visualizing unsupervised semanticsegmentation results – yet by no means do we focus on top segmenta-tion performance. Instead, our protocol allows us to easily convey modelbehavior that consistently emerges across images. Benchmarked on a di-verse set of SSL models across layers and representations, our protocolallows us to gain novel insights into distinct positional biases and scalingbehaviors, including, e.g., strong boundary artifacts in DINOv3-Largemodel tokens. These novel insights come on top of more easily conveyinga range of previous findings. Our protocol further allows us to clearly vi-sually convey and distinguish between positional effects and the closelyrelated yet distinct locality bias, the latter being much more extensivelystudied in the literature so far. Our protocol is publicly available6 , serv-ing to catalyze further model understanding for a broad community.

Sparse-view Computed Tomography (CT) reconstructs im-ages from a limited number of X-ray projections to reduce radiation andscanning time, which is an ill-posed inverse problem. Existing methodsachieve high-fidelity reconstructions but overfit to a fixed acquisitionsetup, failing to generalize well across sampling rates. For example, con-volutional neural networks (CNNs) use the same kernels across resolu-tions, leading to artifacts when data resolution changes. This is a criticallimitation in clinical practice, where acquisition sampling settings varyacross organs and diagnostic protocols.We propose Computed Tomography neural Operator (CTO), the firstneural operator (NO) framework for CT reconstruction. CTO extendslearning from fixed discretized grids to continuous function space, en-abling a single model to generalize across measurement sampling rateswithout retraining. We also propose new NO architectural designs forCT: (i) a dual-domain NO architecture in both sinogram and imagespaces, capturing complementary spatial–frequency information, and (ii)rotation-equivariant DIScrete–COntinuous convolutions (DISCO) thatexploit the rotational structure inherent in tomographic acquisition. Em-pirically, CTO outperform CNNs (> 3.4dB PSNR gain) and other base-lines in multi-resolution settings across multiple CT datasets. Comparedto state-of-the-art di!usion methods, CTO has 500→ faster inferencewith an average 3dB gain. CTO further demonstrates strong out-of-distribution robustness, maintaining gains under cross-dataset transferand noisy sinogram conditions. Ablation studies also validate each de-sign choice. CTO establishes neural operators as a principled and practi-cal paradigm for flexible, discretization-agnostic CT reconstruction. Ourcode is available at https://github.com/neuraloperator/sparse_ct.

RGBT tracking on edge devices is often constrained by lim-ited computational power and data transmission bandwidth, necessitat-ing the use of low-resolution inputs in real-world deployments. However,such resolution reduction typically leads to severe semantic loss and per-formance degradation. In this paper, we propose GAP-Track, an efficientframework that bridges the resolution gap by enabling high-precisiontracking of low-resolution inputs. Specifically, we introduce a hierarchicalknowledge distillation strategy that guides the low-resolution student bymulti-level teacher supervision. To recover the missing semantic details,a generative reconstruction module trained by a random masking strat-egy is introduced to strengthen the student’s feature representation byforcing the backbone to recover fine-grained structural information fromlow resolution data. Furthermore, we adapt a polar-geometric sensitivityloss that transforms the bounding box regression into a polar coordinatesystem, effectively mitigating localization ambiguities in extremely low-resolution scenarios by providing higher sensitivity to center-point driftand shape deformations. Extensive experiments on the comprehensiveLasHeR and RGBT234 datasets demonstrate that our method achievesstate-of-the-art performance. Notably, GAP-Track delivers superior pre-cision at 1/2 resolution and continues to significantly outperform thebaseline even at 1/4 resolution, ensuring a robust balance between track-ing accuracy and inference efficiency. The source code will be availableat https://github.com/shijiahahaha/GAPTrack.

Existing Multi-view Anomaly Detection (MAD) methods as-sume that all views are completely available and model each view sep-arately. However, in real industrial scenarios, information in the viewmay be missing due to faults such as occlusion, which leads to the per-formance degradation of existing methods due to the lack of a multi-viewconsistency prior. To address this, we explored a more challenging task:Incomplete Multi-View Anomaly Detection (IMVAD), in which someareas of each view were masked. We proposed a pipeline for automati-cally generating the IMVAD dataset and generated the RIMAD datasetbased on the Real-IAD dataset through this pipeline. In addition, in or-der to effectively utilize the information of multiple views in the absenceof view information, we propose IMMoE, which consists of two keymodules: (1) Multi-View Expert Fusion (MVEF) effectively fuses multi-view information through a multi-view expert network and guides thereconstruction of a single view; (2) Local Anomaly Enhancement En-coder (LAEE) effectively prevents the model from overfitting the maskregion by applying dropout to local features. Our method achieves state-of-the-art performance on both the RIMAD and Real-IAD datasets, es-pecially on RIMAD, we have increased the pixel-level and image-levelmetrics by 11.8% and 2.8%, respectively. Our source code is available athttps://github.com/HULEI7/IMMoE.

Continual learning aims to progressively learn from a sequenceof tasks, each containing a disjoint subset of classes, while preservingpreviously learned knowledge. Prompt-based continual learning methodspropose to learn a small set of parameters, i.e., prompts, by associatingthem with a query feature of an input image. These methods optimize theprompts, attempting to represent diverse patterns of images. However,we have observed that existing prompt-based methods suffer from aprompt collapse problem, that is, the prompts tend to be highly similarto each other, thereby failing to capture the diverse data distributionsin continual learning scenarios. To address this issue, we propose in thispaper a novel prompt-based continual learning framework that capturesdiverse patterns of images across a sequence of tasks. To this end, wemodel each prompt as a probabilistic distribution and construct a mixtureof these distributions, from which we sample diverse prompts. This enablesour model to effectively capture highly diverse image distributions in thecontinual learning process. We also present a distribution regularizationloss to prevent abrupt changes in the prompt distributions throughoutthe training process. We show extensive experimental results for continuallearning on standard benchmarks, including ImageNet-R, CIFAR-100,and CUB-200, demonstrating the effectiveness of our framework.

3D Gaussian Splatting (3DGS) enables high-quality real-time 3D rendering but faces challenges in efficiently scaling to ultra-dense scenes and high resolutions due to computational bottlenecks thatlimit its use in latency-sensitive applications. Instead of optimizing thesplatting pipeline itself, we propose 3DGS3 , a unified post-renderingframework that jointly performs super sampling and frame interpolationthrough differentiable processing of low-resolution outputs to achieveboth high-resolution and high-frame-rate rendering. Our Gradient -Aware Super Sampling (GASS) module leverages the continuousdifferentiability of 3DGS to extract image gradients that guide a GRU-based refinement network to enable high-fidelity super sampling. Fur-thermore, a Lightweight Temporal Frame Interpolation (LTFI)module based on a compact U-Net-like backbone fuses temporal anddifferentiable spatial cues from consecutive frames to synthesize tem-porally coherent intermediate frames. Experiments on public datasetsdemonstrate that 3DGS3 achieves superior rendering efficiency and vi-sual quality when compared with state-of-the-art methods and remainscompatible with existing 3DGS acceleration techniques. The project codeis available at: https://github.com/Eabo20001/3DGS3.

Dynamic 3D Gaussian splatting faces a fundamental tension between motion consistency and visual fidelity. Deformation-based approaches preserve temporal correspondence but suffer from motion over-factorization, oversmoothing high-frequency dynamics. In contrast, 4D-primitive methods capture fine visual details yet incur temporal overparameterization, breaking object identity and leading to severe storage overhead. To resolve this, we introduce Multi4D, a framework for highfidelity dynamic Gaussian Splatting based on multi-level competitive allocation. Instead of a monolithic representation, we distribute modeling capacity across three structured levels: static structure, persistent dynamic geometry, and transient appearance primitives. Through shared rasterization and residual-driven optimization, these levels dynamically compete to explain photometric error, enabling adaptive specialization without pre-assigned decomposition. This allocation preserves long-term motion consistency while capturing fine dynamic detail, achieving stateof-the-art rendering quality and real-time performance with significantly fewer dynamic primitives. Furthermore, because our representation explicitly tracks compact persistent Gaussians over time, semantic features can be embedded afterward, enabling Multi4D to achieve state-of-the-art 4D segmentation accuracy with an order-of-magnitude speedup.

Vision-language pre-trained models (VLPs) are widely used in real-world applications. However, they remain vulnerable to adversarial attacks. Although adversarial detection methods have demonstrated success in single-modality settings (either vision or language), their effectiveness and reliability in multimodal models such as VLPs remain largely unexplored. In this work, we study the geometry of VLP embedding spaces and observe structured anisotropy that differs from unimodal vision models. Our theoretical analysis shows that under this anisotropic structure, adversarial attacks increase the expected geometric separation between clean and adversarial examples (AEs). Specifically, we demonstrate that AEs consistently exhibit greater expected distances to randomly sampled points than their clean counterparts, indicating that AEs tend to push representations out of manifold regions. Building on these insights, we propose GeoDetect, which leverages these off-manifold deviations via geometric scores to identify AEs. Through comprehensive evaluations, we show that our approach reliably detects AEs across diverse VLP architectures and threat settings, covering unimodal and multimodal attacks as well as adaptive attacks, thereby providing a robust and practical approach to improving the safety and reliability of these models.

Existing vision-language models (VLMs) have demonstrated impressive performance in reasoning-based segmentation. However, current benchmarks are primarily constructed from high-quality images captured under idealized conditions. This raises a critical question: when visual cues are severely degraded by adverse weather conditions such as rain, snow, or fog, can VLMs sustain reliable reasoning segmentation capabilities? In response to this challenge, we introduce WeatherReasonSeg, a benchmark designed to evaluate VLM performance in reasoningbased segmentation under adverse weather conditions. It consists of two complementary components. First, we construct a controllable reasoning dataset by applying synthetic weather with varying severity levels to existing segmentation datasets, enabling fine-grained robustness analysis. Second, to capture real-world complexity, we curate a real-world adverse-weather reasoning segmentation dataset with semantically consistent queries generated via mask-guided LLM prompting. We further broaden the evaluation scope across five reasoning dimensions, including functionality, application scenarios, structural attributes, interactions, and requirement matching. Extensive experiments across diverse VLMs reveal two key findings: (1) VLM performance degrades monotonically with increasing weather severity, and (2) different weather types induce distinct vulnerability patterns. We hope WeatherReasonSeg will serve as a foundation for advancing robust, weather-aware reasoning.

Recent breakthroughs in 3D generation have enabled thesynthesis of high-fidelity individual assets. However, generating 3D com-positional objects from single images–particularly under occlusions–remainschallenging. Existing methods often degrade geometric details in hiddenregions and fail to preserve the underlying object-object spatial rela-tionships (OOR). We present a novel framework Interact3D designedto generate physically plausible interacting 3D compositional objects.Our approach first leverages advanced generative priors to curate high-quality individual assets with a unified 3D guidance scene. To physicallycompose these assets, we then introduce a robust two-stage compositionpipeline. Based on the 3D guidance scene, the primary object is anchoredthrough precise global-to-local geometric alignment (registration), whilesubsequent geometries are integrated using a differentiable Signed Dis-tance Field (SDF)-based optimization that explicitly penalizes geometryintersections. To reduce challenging collisions, we further deploy a closed-loop, agentic refinement strategy. A Vision-Language Model (VLM) au-tonomously analyzes multi-view renderings of the composed scene, for-mulates targeted corrective prompts, and guides an image editing mod-ule to iteratively self-correct the generation pipeline. Extensive exper-iments demonstrate that Interact3D successfully produces promisingcollision-aware compositions with improved geometric fidelity and con-sistent spatial relationships. The code and dataset will be available athttps://github.com/SII-Hui/Interact3D.

In recent years, significant progress has been made in estimat-ing 3D human joint positions from monocular 2D images or videos. How-ever, existing approaches that lift 2D to 3D often rely on mean per-jointposition error (MPJPE) as explicit supervision, while ignoring implicitsemantic consistency across 2D–3D sequences and disregarding humankinematic constraints. As a result, these methods may produce physio-logically implausible poses. To address this issue, we propose SemGAN,an adversarial learning framework that integrates explicit supervisionwith implicit semantic constraints. Specifically, we employ a two-stageadversarial learning approach that complements explicit supervision withmotion semantic consistency as an implicit constraint, which is furtherenforced through an Entropy-based Cross-Frequency Discriminator(ECF-Discriminator). This design jointly optimizes both joint coordinate ac-curacy and motion semantic consistency. Moreover, guided by humankinematic principles, we construct a Semantic Prototype-guided Hierarchi-cal Generator(SPH-Generator), which enhances local part-level uniformityand global inter-part coordination, thereby alleviating abnormal 3D posegeneration. Extensive experiments on the Human3.6M and MPI-INF-3DHP benchmark datasets demonstrate that our method outperformsprevious state-of-the-art approaches.

While 3D Gaussian Splatting (3DGS) has revolutionized novelview synthesis with its photorealistic and real-time rendering, its lengthytraining process remains a critical bottleneck. Existing acceleration meth-ods largely treat algorithm design and system optimization in isolation,failing to fully unleash hardware potential. To systematically addressthis, we propose LiteGS, a high-performance framework driven by afundamental system-algorithm codesign. At the GPU and data layers,LiteGS introduces a warp-based rasterization paradigm and a Cluster-Cull-Compact pipeline, fundamentally resolving data conflicts and spa-tial locality issues. Empowered by this robust backend, we introduce avariance-guided densification metric at the algorithm layer to accuratelyidentify under-reconstructed regions. This powerful synergy establishes anew state-of-the-art: LiteGS consistently achieves the highest renderingquality and the fastest training speed across all parameter scales. Exten-sive evaluations demonstrate our milestone performance. Compared tothe high-quality SOTA (3DGS-MCMC), LiteGS achieves at least a 10×speedup while matching or exceeding its fidelity. Furthermore, by unlock-ing a “space-for-time” aggressive densification strategy, LiteGS success-fully compresses the training time to the sub-minute level on RTX3090(further to ∼30 seconds on RTX 4090) while securing rendering qualitycomparable to the original 3DGS.

Vision Transformers (ViTs) are strong backbones for seman-tic segmentation, but their computational cost limits deployment. Recenttoken compression methods for efficient transformer-based segmentationreduce this cost by decreasing the number of tokens. However, exist-ing evaluations primarily focus on low-to-moderate compression, leavingtheir behavior under aggressive compression and corrupted inputs un-clear. Meanwhile, structural pruning provides an orthogonal route toefficiency by removing redundant components in the ViT architecture,but is rarely compared to token compression under a unified protocol.To bridge this gap, we benchmark representative token compression andstructural pruning methods for ViT-based semantic segmentation un-der matched FLOPs on ADE20K and Cityscapes, together with theircommon-corruption variants ADE20K-C and Cityscapes-C. Our resultsreveal a consistent trend on both clean and corrupted inputs: token com-pression is highly effective at mild reductions but degrades sharply whencompression becomes severe, consistent with substantial information lossfrom overly aggressive token reduction. In contrast, structural prun-ing exhibits a smoother degradation curve and is more stable at highcompression. Motivated by these findings, we study a prune-then-mergepipeline that applies moderate token compression on top of a moder-ately pruned backbone. At comparable FLOPs, this combined strategyconsistently achieves a better accuracy-robustness trade-off at high com-pression, offering a practical recipe for deployment-oriented ViT segmen-tation. Code is available at https://github.com/phatnguyencs/vit-seg-compression.

Inverse problems—inferring an underlying signal or statefrom incomplete or noisy measurements—are fundamental to science andengineering. Recent approaches employ diffusion models as powerful im-plicit priors for such problems, owing to their ability to capture com-plex data distributions. However, existing diffusion-based methods forinverse problems often rely on strong approximations of the posteriordistribution, require computationally expensive gradient backpropaga-tion through the score network, or are restricted to linear measurementmodels.In this work, we propose Restart for Posterior Sampling (RePS), a gen-eral and efficient framework for solving both linear and non-linear in-verse problems using pre-trained diffusion models. RePS builds on theidea of restart-based sampling, previously shown to improve sample qual-ity in unconditional diffusion, and extends it to posterior inference. Ourmethod employs a conditioned ODE applicable to any differentiable mea-surement model and introduces a simplified restart strategy that con-tracts accumulated approximation errors during sampling. Unlike someof the prior approaches, RePS avoids backpropagation through the scorenetwork, substantially reducing computational cost. We demonstrate thatRePS achieves faster convergence and superior reconstruction qualitycompared to existing diffusion-based baselines across a range of inverseproblems, including both linear and non-linear settings.

Visual Speech Recognition (VSR) tasks in complex multi-speaker scenarios are severely hindered by rapid head motions, occlu-sions, and subtle lip articulations. Traditional RGB-based methods strug-gle here due to low rates and motion blur of frames. To overcome these,we propose LipsFlow, a neuromorphic-inspired VSR framework thatconverts RGB videos into high-temporal-resolution event streams. Formulti-speaker, we employ ByteTrack tracking and TalkNet active speakerdetection to temporally segment scenes into single-speaker clips, enablingfocused per-speaker analysis. By explicitly capturing microsecond-levelarticulatory dynamics via learnable event-based representations, Lips-Flow achieves inherent robustness against visual degradation. To effi-ciently model these dense event-based features and adapt to speaker-specific articulatory patterns, we introduce Optimal Transport Condi-tional Flow Matching (OT-CFM). It enforces deterministic, straight-line trajectory generation in a semantic latent space, slashing inferencelatency to just two Ordinary Differential Equation (ODE) steps. Further-more, we design a Dual-Level Semantic Supervision mechanism combin-ing token-level BERT weight tying and sentence-level priors to resolvehomophene ambiguities. Validated on competitive benchmarks, LipsFlowachieves a state-of-the-art WER of 22.3% at 240 ms latency, estab-lishing a highly robust and efficient paradigm for event-based VSR.