ECCV 2026 Papers

Continual learning aims to progressively learn from a sequenceof tasks, each containing a disjoint subset of classes, while preservingpreviously learned knowledge. Prompt-based continual learning methodspropose to learn a small set of parameters, i.e., prompts, by associatingthem with a query feature of an input image. These methods optimize theprompts, attempting to represent diverse patterns of images. However,we have observed that existing prompt-based methods suffer from aprompt collapse problem, that is, the prompts tend to be highly similarto each other, thereby failing to capture the diverse data distributionsin continual learning scenarios. To address this issue, we propose in thispaper a novel prompt-based continual learning framework that capturesdiverse patterns of images across a sequence of tasks. To this end, wemodel each prompt as a probabilistic distribution and construct a mixtureof these distributions, from which we sample diverse prompts. This enablesour model to effectively capture highly diverse image distributions in thecontinual learning process. We also present a distribution regularizationloss to prevent abrupt changes in the prompt distributions throughoutthe training process. We show extensive experimental results for continuallearning on standard benchmarks, including ImageNet-R, CIFAR-100,and CUB-200, demonstrating the effectiveness of our framework.

3D Gaussian Splatting (3DGS) enables high-quality real-time 3D rendering but faces challenges in efficiently scaling to ultra-dense scenes and high resolutions due to computational bottlenecks thatlimit its use in latency-sensitive applications. Instead of optimizing thesplatting pipeline itself, we propose 3DGS3 , a unified post-renderingframework that jointly performs super sampling and frame interpolationthrough differentiable processing of low-resolution outputs to achieveboth high-resolution and high-frame-rate rendering. Our Gradient -Aware Super Sampling (GASS) module leverages the continuousdifferentiability of 3DGS to extract image gradients that guide a GRU-based refinement network to enable high-fidelity super sampling. Fur-thermore, a Lightweight Temporal Frame Interpolation (LTFI)module based on a compact U-Net-like backbone fuses temporal anddifferentiable spatial cues from consecutive frames to synthesize tem-porally coherent intermediate frames. Experiments on public datasetsdemonstrate that 3DGS3 achieves superior rendering efficiency and vi-sual quality when compared with state-of-the-art methods and remainscompatible with existing 3DGS acceleration techniques. The project codeis available at: https://github.com/Eabo20001/3DGS3.

Dynamic 3D Gaussian splatting faces a fundamental tension between motion consistency and visual fidelity. Deformation-based approaches preserve temporal correspondence but suffer from motion over-factorization, oversmoothing high-frequency dynamics. In contrast, 4D-primitive methods capture fine visual details yet incur temporal overparameterization, breaking object identity and leading to severe storage overhead. To resolve this, we introduce Multi4D, a framework for highfidelity dynamic Gaussian Splatting based on multi-level competitive allocation. Instead of a monolithic representation, we distribute modeling capacity across three structured levels: static structure, persistent dynamic geometry, and transient appearance primitives. Through shared rasterization and residual-driven optimization, these levels dynamically compete to explain photometric error, enabling adaptive specialization without pre-assigned decomposition. This allocation preserves long-term motion consistency while capturing fine dynamic detail, achieving stateof-the-art rendering quality and real-time performance with significantly fewer dynamic primitives. Furthermore, because our representation explicitly tracks compact persistent Gaussians over time, semantic features can be embedded afterward, enabling Multi4D to achieve state-of-the-art 4D segmentation accuracy with an order-of-magnitude speedup.

Contrastive vision-language models such as CLIP map se-mantically opposite phrases (e.g., “a dog” vs. “not a dog”) to nearly iden-tical embeddings, rendering them insensitive to negation. We attributethis failure to a phenomenon we call Representational Collapse: by track-ing compositional divergence and visual alignment across the CLIP textencoder, we show that middle layers build compositional syntax, but thefinal layers collapse this structure as visual alignment rises, producinga syntax-blind final representation. To recover the lost negation signalwithout altering pretrained weights, we propose PeakPatch, a lightweightpost-hoc correction system that intercepts the encoder at its composi-tional peak while keeping CLIP fully frozen. An Embedding CorrectionNetwork (ECN) uses cross-attention to extract a negation-specific sig-nal from the peak layer, anchored to a stable baseline, and predicts adeviation vector that re-injects the lost syntax into the final-layer em-bedding space. A complementary Score Correction Network (SCN) pre-dicts bounded scalar score offsets for discriminative tasks. Both modulesare trained jointly end-to-end while all CLIP parameters remain frozen,adding only 5.2M parameters (3.5% of the backbone) and preserving thestandard cosine similarity interface. On NegBench, PeakPatch achieves74.3% on COCO MCQ (+35.1 over CLIP, +17.8 over the best encoderfine-tuning method) and 65.5% on VOC MCQ, while outperforming allfine-tuning baselines on fully out-of-distribution negation retrieval de-spite training only 3.5% of the parameters. The corrected embeddingsalso transfer to text-to-image generation (+18.4 negation score) and gen-eralize across ViT-B/32, ViT-L/14, and SigLIP backbones.Project page: https://stevencylu.github.io/PeakPatch/

Vision-language pre-trained models (VLPs) are widely used in real-world applications. However, they remain vulnerable to adversarial attacks. Although adversarial detection methods have demonstrated success in single-modality settings (either vision or language), their effectiveness and reliability in multimodal models such as VLPs remain largely unexplored. In this work, we study the geometry of VLP embedding spaces and observe structured anisotropy that differs from unimodal vision models. Our theoretical analysis shows that under this anisotropic structure, adversarial attacks increase the expected geometric separation between clean and adversarial examples (AEs). Specifically, we demonstrate that AEs consistently exhibit greater expected distances to randomly sampled points than their clean counterparts, indicating that AEs tend to push representations out of manifold regions. Building on these insights, we propose GeoDetect, which leverages these off-manifold deviations via geometric scores to identify AEs. Through comprehensive evaluations, we show that our approach reliably detects AEs across diverse VLP architectures and threat settings, covering unimodal and multimodal attacks as well as adaptive attacks, thereby providing a robust and practical approach to improving the safety and reliability of these models.

Existing vision-language models (VLMs) have demonstrated impressive performance in reasoning-based segmentation. However, current benchmarks are primarily constructed from high-quality images captured under idealized conditions. This raises a critical question: when visual cues are severely degraded by adverse weather conditions such as rain, snow, or fog, can VLMs sustain reliable reasoning segmentation capabilities? In response to this challenge, we introduce WeatherReasonSeg, a benchmark designed to evaluate VLM performance in reasoningbased segmentation under adverse weather conditions. It consists of two complementary components. First, we construct a controllable reasoning dataset by applying synthetic weather with varying severity levels to existing segmentation datasets, enabling fine-grained robustness analysis. Second, to capture real-world complexity, we curate a real-world adverse-weather reasoning segmentation dataset with semantically consistent queries generated via mask-guided LLM prompting. We further broaden the evaluation scope across five reasoning dimensions, including functionality, application scenarios, structural attributes, interactions, and requirement matching. Extensive experiments across diverse VLMs reveal two key findings: (1) VLM performance degrades monotonically with increasing weather severity, and (2) different weather types induce distinct vulnerability patterns. We hope WeatherReasonSeg will serve as a foundation for advancing robust, weather-aware reasoning.

Recent breakthroughs in 3D generation have enabled thesynthesis of high-fidelity individual assets. However, generating 3D com-positional objects from single images–particularly under occlusions–remainschallenging. Existing methods often degrade geometric details in hiddenregions and fail to preserve the underlying object-object spatial rela-tionships (OOR). We present a novel framework Interact3D designedto generate physically plausible interacting 3D compositional objects.Our approach first leverages advanced generative priors to curate high-quality individual assets with a unified 3D guidance scene. To physicallycompose these assets, we then introduce a robust two-stage compositionpipeline. Based on the 3D guidance scene, the primary object is anchoredthrough precise global-to-local geometric alignment (registration), whilesubsequent geometries are integrated using a differentiable Signed Dis-tance Field (SDF)-based optimization that explicitly penalizes geometryintersections. To reduce challenging collisions, we further deploy a closed-loop, agentic refinement strategy. A Vision-Language Model (VLM) au-tonomously analyzes multi-view renderings of the composed scene, for-mulates targeted corrective prompts, and guides an image editing mod-ule to iteratively self-correct the generation pipeline. Extensive exper-iments demonstrate that Interact3D successfully produces promisingcollision-aware compositions with improved geometric fidelity and con-sistent spatial relationships. The code and dataset will be available athttps://github.com/SII-Hui/Interact3D.

In recent years, significant progress has been made in estimat-ing 3D human joint positions from monocular 2D images or videos. How-ever, existing approaches that lift 2D to 3D often rely on mean per-jointposition error (MPJPE) as explicit supervision, while ignoring implicitsemantic consistency across 2D–3D sequences and disregarding humankinematic constraints. As a result, these methods may produce physio-logically implausible poses. To address this issue, we propose SemGAN,an adversarial learning framework that integrates explicit supervisionwith implicit semantic constraints. Specifically, we employ a two-stageadversarial learning approach that complements explicit supervision withmotion semantic consistency as an implicit constraint, which is furtherenforced through an Entropy-based Cross-Frequency Discriminator(ECF-Discriminator). This design jointly optimizes both joint coordinate ac-curacy and motion semantic consistency. Moreover, guided by humankinematic principles, we construct a Semantic Prototype-guided Hierarchi-cal Generator(SPH-Generator), which enhances local part-level uniformityand global inter-part coordination, thereby alleviating abnormal 3D posegeneration. Extensive experiments on the Human3.6M and MPI-INF-3DHP benchmark datasets demonstrate that our method outperformsprevious state-of-the-art approaches.

While 3D Gaussian Splatting (3DGS) has revolutionized novelview synthesis with its photorealistic and real-time rendering, its lengthytraining process remains a critical bottleneck. Existing acceleration meth-ods largely treat algorithm design and system optimization in isolation,failing to fully unleash hardware potential. To systematically addressthis, we propose LiteGS, a high-performance framework driven by afundamental system-algorithm codesign. At the GPU and data layers,LiteGS introduces a warp-based rasterization paradigm and a Cluster-Cull-Compact pipeline, fundamentally resolving data conflicts and spa-tial locality issues. Empowered by this robust backend, we introduce avariance-guided densification metric at the algorithm layer to accuratelyidentify under-reconstructed regions. This powerful synergy establishes anew state-of-the-art: LiteGS consistently achieves the highest renderingquality and the fastest training speed across all parameter scales. Exten-sive evaluations demonstrate our milestone performance. Compared tothe high-quality SOTA (3DGS-MCMC), LiteGS achieves at least a 10×speedup while matching or exceeding its fidelity. Furthermore, by unlock-ing a “space-for-time” aggressive densification strategy, LiteGS success-fully compresses the training time to the sub-minute level on RTX3090(further to ∼30 seconds on RTX 4090) while securing rendering qualitycomparable to the original 3DGS.

Vision Transformers (ViTs) are strong backbones for seman-tic segmentation, but their computational cost limits deployment. Recenttoken compression methods for efficient transformer-based segmentationreduce this cost by decreasing the number of tokens. However, exist-ing evaluations primarily focus on low-to-moderate compression, leavingtheir behavior under aggressive compression and corrupted inputs un-clear. Meanwhile, structural pruning provides an orthogonal route toefficiency by removing redundant components in the ViT architecture,but is rarely compared to token compression under a unified protocol.To bridge this gap, we benchmark representative token compression andstructural pruning methods for ViT-based semantic segmentation un-der matched FLOPs on ADE20K and Cityscapes, together with theircommon-corruption variants ADE20K-C and Cityscapes-C. Our resultsreveal a consistent trend on both clean and corrupted inputs: token com-pression is highly effective at mild reductions but degrades sharply whencompression becomes severe, consistent with substantial information lossfrom overly aggressive token reduction. In contrast, structural prun-ing exhibits a smoother degradation curve and is more stable at highcompression. Motivated by these findings, we study a prune-then-mergepipeline that applies moderate token compression on top of a moder-ately pruned backbone. At comparable FLOPs, this combined strategyconsistently achieves a better accuracy-robustness trade-off at high com-pression, offering a practical recipe for deployment-oriented ViT segmen-tation. Code is available at https://github.com/phatnguyencs/vit-seg-compression.

Inverse problems—inferring an underlying signal or statefrom incomplete or noisy measurements—are fundamental to science andengineering. Recent approaches employ diffusion models as powerful im-plicit priors for such problems, owing to their ability to capture com-plex data distributions. However, existing diffusion-based methods forinverse problems often rely on strong approximations of the posteriordistribution, require computationally expensive gradient backpropaga-tion through the score network, or are restricted to linear measurementmodels.In this work, we propose Restart for Posterior Sampling (RePS), a gen-eral and efficient framework for solving both linear and non-linear in-verse problems using pre-trained diffusion models. RePS builds on theidea of restart-based sampling, previously shown to improve sample qual-ity in unconditional diffusion, and extends it to posterior inference. Ourmethod employs a conditioned ODE applicable to any differentiable mea-surement model and introduces a simplified restart strategy that con-tracts accumulated approximation errors during sampling. Unlike someof the prior approaches, RePS avoids backpropagation through the scorenetwork, substantially reducing computational cost. We demonstrate thatRePS achieves faster convergence and superior reconstruction qualitycompared to existing diffusion-based baselines across a range of inverseproblems, including both linear and non-linear settings.

Visual Speech Recognition (VSR) tasks in complex multi-speaker scenarios are severely hindered by rapid head motions, occlu-sions, and subtle lip articulations. Traditional RGB-based methods strug-gle here due to low rates and motion blur of frames. To overcome these,we propose LipsFlow, a neuromorphic-inspired VSR framework thatconverts RGB videos into high-temporal-resolution event streams. Formulti-speaker, we employ ByteTrack tracking and TalkNet active speakerdetection to temporally segment scenes into single-speaker clips, enablingfocused per-speaker analysis. By explicitly capturing microsecond-levelarticulatory dynamics via learnable event-based representations, Lips-Flow achieves inherent robustness against visual degradation. To effi-ciently model these dense event-based features and adapt to speaker-specific articulatory patterns, we introduce Optimal Transport Condi-tional Flow Matching (OT-CFM). It enforces deterministic, straight-line trajectory generation in a semantic latent space, slashing inferencelatency to just two Ordinary Differential Equation (ODE) steps. Further-more, we design a Dual-Level Semantic Supervision mechanism combin-ing token-level BERT weight tying and sentence-level priors to resolvehomophene ambiguities. Validated on competitive benchmarks, LipsFlowachieves a state-of-the-art WER of 22.3% at 240 ms latency, estab-lishing a highly robust and efficient paradigm for event-based VSR.

Sparse-view computed tomography (CT) is critical for re-ducing radiation exposure to patients. Recent advances in radiative 3DGaussian Splatting (3DGS) have enabled fast and accurate sparse-viewCT reconstruction. Despite these algorithmic advancements, practicalreconstruction fidelity remains fundamentally bounded by the qualityof the captured data, raising the crucial yet underexplored problem ofX-ray active view selection. Existing active view selection methods areprimarily designed for natural-light scenes and fail to capture the uniquegeometric ambiguities and physical attenuation properties inherent inX-ray imaging. In this paper, we present Perturbed Gaussian Ensemble,an active view selection framework that integrates uncertainty modelingwith sequential decision-making, tailored for X-ray Gaussian Splatting.Specifically, we identify low-density Gaussian primitives that are likely tobe uncertain and apply stochastic density scaling to construct an ensem-ble of plausible Gaussian density fields. For each candidate projection,we measure the structural variance of the ensemble predictions and selectthe one with the highest variance as the next best view. Extensive experi-mental results on arbitrary-trajectory CT benchmarks demonstrate thatour density-guided perturbation strategy effectively eliminates geometricartifacts and consistently outperforms existing baselines in progressivetomographic reconstruction under unified view selection protocols.

Transformer-based object trackers are renowned for theirstrong performance, yet dense token processing often leads to prohibitivecomputational cost, limiting real-time deployment on edge devices. Whilerecent works explore token pruning to reduce computation, they oftenstop short of an end-to-end sparse pipeline, as early-layer token scores canbe noisy without a motion prior, and many trackers ultimately fall back todense reshaping to feed the dense prediction head that partially negatesthe savings. We introduce Motion-aware Sparse Tracker (MaST), a sparsetracking framework that makes sparsity effective from tokens to boxes.First, MaST injects a lightweight motion prior to refine cross-attention-based importance scores, enabling earlier and more stable token reductionin the search region. Second, we introduce a natively sparse predictionhead that operates directly on the retained unstructured tokens witha score-first, regress-once design, eliminating dense padding/reshapingand reducing redundant computation. Extensive experiments on multiplebenchmarks demonstrate that MaST establishes new state of the artamong lightweight trackers, where MaST-tiny attains 63.8 AUC on LaSOTand 80.1 SUC on TrackingNet, surpassing the prior best AsymTrack-Sby +1.0 AUC and +2.2 SUC while running at 152 FPS on Jetson Nano,nearly twice as fast as AsymTrack-S at 88 FPS. Code is available atgithub.com/TsingWei/MaST.

Deep vision models exploit shortcuts, relying on cues thatcorrelate with supervision signals. Prior work has focused on visiblebiases, such as object-background or texture correlations. We identifya different source of shortcut learning: invisible metadata traces em-bedded at the pixel level, for metadata such as image processing andphoto acquisition. We hypothesize that large-scale semantic supervi-sion, whether through categorical labels (ImageNet) or billion-scale cap-tions (LAION), naturally induces metadata-semantics correlations dur-ing pretraining, leading models to convert low-level signals into predic-tive features. By introducing controlled metadata-semantics correlations,we show that stronger ones produce systematically higher sensitivity tometadata traces and larger performance degradation under metadatadistribution shifts. We further explore mitigation strategies applied dur-ing and after pretraining that reduce sensitivity not only to targetedmetadata but also to unseen ones, without sacrificing performance ondownstream tasks. Metadata sensitivity also has a positive side: it partlyexplains the strong generated-image detection ability of some encoders,while its mitigation can improve out-of-distribution generalization. Code:

Deep learning based object detectors require trustworthinessbeyond competitive detection performance, but deep neural networks areprone to overconfident predictions, assigning high confidence scores topredictions that are likely to be inaccurate. To improve the alignmentbetween confidence scores and prediction accuracy, existing methodscalibrate confidence scores based on box-level localization, such as pre-cision or intersection over union with the ground truth bounding box.However, box-level localization reflects only a measure of agreement be-tween the predicted box and the ground truth, resulting in calibratedconfidence scores for box-level accuracy failing to capture the localizationaccuracy of coordinates of box. To tackle this issue, we propose a novelpost-hoc calibration framework, rethinking detection calibration (ReDC),which provides reliable coordinate-level confidence scores, including di-rectional information. The proposed framework defines coordinate-wisealignment and deviation direction between predictions and ground truth.Based on the alignment measure, confidence re-encoding produces reliablecoordinate-level confidence scores, while directional displacement estima-tion predicts coordinate-wise deviation directions. Extensive experimentsunder in-domain and out-domain scenarios demonstrate that the proposedapproach expresses the coordinate-wise localization of detected objectsmore precisely than existing methods. Furthermore, our method coversthe representational scope of prior calibration approaches by aggregatingcoordinate-level confidence scores into box-level localization.

Empowered by large-scale training, vision-language models (VLMs) achieve strong image and video understanding, yet their ability to perform spatial reasoning in both static scenes and dynamic videos remains limited. Recent advances try to handle this limitation by injecting geometry tokens from pretrained 3D foundation models into VLMs. Nevertheless, we observe that naive token fusion followed by standard finetuning in this line of work often leaves such geometric cues underutilized for spatial reasoning, as VLMs tend to rely heavily on 2D visual cues. In this paper, we propose GeoSR, a framework designed to make geometry matter by encouraging VLMs to actively reason with geometry tokens. GeoSR introduces two key components: (1) Geometry-Unleashing Masking, which strategically masks portions of 2D vision tokens during training to weaken non-geometric shortcuts and force the model to consult geometry tokens for spatial reasoning; and (2) Geometry-Guided Fusion, a gated routing mechanism that adaptively amplifies geometry token contributions in regions where geometric evidence is critical. Together, these designs unleash the potential of geometry tokens for spatial reasoning tasks. Extensive experiments on both static and dynamic spatial reasoning benchmarks demonstrate that GeoSR consistently outperforms prior methods and establishes new state-of-the-art performance by effectively leveraging geometric information. The project page is available at https://suhzhang.github.io/GeoSR/.

Vision-Language-Action (VLA) policies fine-tuned with RLand frozen VLM reward evaluators can produce observations that lookcorrect without physically completing the task—a phenomenon we for-malize as stage hallucination. To suppress it, we propose EvoVLA, aself-supervised framework with three synergistic modules: Stage-AlignedReward (SAR) penalizes near-miss states via counterfactual hard nega-tives in CLIP-based scoring; Pose-Based Object Exploration (POE)grounds curiosity in relative gripper–object pose; and Long-HorizonMemory uses selective context retrieval with gated fusion to stabi-lize credit assignment. We also introduce Discoverse-L, a long-horizonbenchmark (3 tasks, 18–74 stages) with a formal Hallucination Rate met-ric. Under matched backbones and budgets, EvoVLA achieves 69.2% suc-cess (+10.2 over OpenVLA-OFT), 1.5× sample efficiency, and reduceshallucination from 38.5% to 14.8%; real-robot deployment attains 54.6%(+11.0) across four tasks, confirming Sim2Real transfer.

Open-vocabulary scene understanding aims to recognize ar-bitrary semantic concepts in 3D beyond a fixed set of categories. Re-cent methods built upon 3D Gaussian Splatting (3DGS) enable sceneunderstanding by distilling features from vision–language models, butmost rely on per-scene optimization and dense observations, which limitscalability. More recent generalizable 3DGS methods show promise, butthey perform poorly on sparse-view satellite imagery due to the uniquepushbroom imaging mode, limited geometric constraints, and extremescale variations. To address these limitations, we propose SkySplat-OV,a feed-forward framework that integrates the rational polynomial co-efficient (RPC) model into a generalizable language 3DGS pipeline. AGeometry–Radiance–Semantics Fusion module, together with monoc-ular relative height supervision, improves Gaussian parameter predic-tion under limited geometric constraints. Moreover, SkySplat-OV em-ploys single-stage SAM3 distillation to handle extreme scale variationsin satellite imagery for accurate segmentation. Compared to per-sceneoptimization methods, SkySplat-OV achieves up to 1070 times speedupover LangSplatV2 with higher accuracy. It also outperforms generaliz-able 3DGS baselines, achieving 12.91%–25.75% mIoU improvements ontwo benchmarks, and demonstrates strong cross-dataset generalizationon the MVS3D test set. Additional qualitative results are available athttps://skysplat-ov.github.io/.