WACV 2025 论文列表

This paper introduces the Elemental Composite Prototypical Network (ECPN) a novel approach to few-shot learning (FSL) in outdoor 3D point cloud object detection. Such point clouds are inherently non-uniformly packed and show marked intra-class variations due to aberrations in lidar scanning methods. Due to the limited availability of examples in the FSL setting the intra-class variations serve as a much more formidable challenge to traditional detection algorithms. ECPN employs a novel prototypical learning method that solves the issues mentioned above. We generate and leverage multiple elemental prototypes for each class to capture essential geometric features from limited examples. These elemental prototypes are then combined in a weighted manner to arrive at composite prototypes that score relevant and irrelevant features in the elemental prototypes with respect to the query point cloud scene. Moreover we introduce a novel feature-similarity-discrimination loss to refine the model's ability to distinguish between relevant objects and their background significantly improving object detection accuracy in FSL scenarios. Our extensive testing on the nuScenes dataset demonstrates that ECPN significantly outperforms existing baselines offering a robust solution to the complexities of outdoor few-shot 3D object detection (O-FS3D) and setting a new standard for future research.

The task of lane detection involves identifying the boundaries of driving areas in real-time. Recognizing lanes with variable and complex geometric structures remains a challenge. In this paper we explore a novel and flexible way of implicit lanes representation named Elastic Lane map (ELM) and introduce an efficient physics-informed end-to-end lane detection framework namely ElasticLaneNet (Elastic interaction energy-informed Lane detection Network). The approach considers predicted lanes as moving zero-contours on the flexibly shaped ELM that are attracted to the ground truth guided by an elastic interaction energy-loss function (EIE loss). Our framework well integrates the global information and low-level features. The method performs well in complex lane scenarios including those with large curvature turns intersections various crossing lanes Y-shapes lanes dense lanes etc. We apply our approach on three datasets: SDLane TuSimple and CULane. The results demonstrate exceptional performance of our method with the state-of-the-art results on the structurally diverse SDLane achieving F1-score of 89.51 Recall of 87.50 and Precision of 91.61 with fast inference speed.

We introduce EgoSonics a method to generate semantically meaningful and synchronized audio tracks conditioned on silent egocentric videos. Generating audio for silent egocentric videos could open new applications in virtual reality assistive technologies or for augmenting existing datasets. Existing work has been limited to domains like speech music or impact sounds and cannot easily capture the broad range of audio frequencies found in egocentric videos. EgoSonics addresses these limitations by building on the strength of latent diffusion models for conditioned audio synthesis. We first encode and process audio and video data into a form that is suitable for generation. The encoded data is used to train our model to generate audio tracks that capture the semantics of the input video. Our proposed SyncroNet builds on top of ControlNet to provide control signals that enables temporal synchronization to the synthesized audio. Extensive evaluations show that our model outperforms existing work in audio quality and in our newly proposed synchronization evaluation method. Furthermore we demonstrate downstream applications of our model in improving video summarization.

We introduce EgoPoints a benchmark for point tracking in egocentric videos. We annotate 4.7K challenging tracks in egocentric sequences. Compared to the popular TAP-Vid-DAVIS evaluation benchmark we include 9x more points that go out-of-view and 59x more points that require re-identification (ReID) after returning to view. To measure the performance of models on these challenging points we introduce evaluation metrics that specifically monitor tracking performance on points in-view out-of-view and points that require re-identification. We then propose a pipeline to create semi-real sequences with automatic ground truth. We generate 11K such sequences by combining dynamic Kubric objects with scene points from EPIC Fields. When fine-tuning point tracking methods on these sequences and evaluating on our annotated EgoPoints sequences we improve CoTracker across all metrics including the tracking accuracy d^*_avg by 2.7 percentage points and accuracy on ReID sequences (ReIDd_avg) by 2.4 points. We also improve d^*_avg and ReIDd_avg of PIPs++ by 0.3 and 2.8 respectively.

Accurately estimating and forecasting human body pose is important for enhancing the user's sense of immersion in Augmented Reality. Addressing this need our paper introduces EgoCast a bimodal method for 3D human pose forecasting using egocentric videos and proprioceptive data. We study the task of human pose forecasting in a realistic setting extending the boundaries of temporal forecasting in dynamic scenes and building on the current framework for current pose estimation in the wild. We introduce a current-frame estimation module that generates pseudo-groundtruth poses for inference eliminating the need for past groundtruth poses typically required by current methods during forecasting. Our experimental results on the recent Ego-Exo4D and Aria Digital Twin datasets validate EgoCast for real-life motion estimation. On the Ego-Exo4D Body Pose 2024 Challenge our method significantly outperforms the state-of-the-art approaches laying the groundwork for future research in human pose estimation and forecasting in unscripted activities with egocentric inputs.

Video understanding typically requires fine-tuning the large backbone when adapting to new domains. In this paper we leverage the egocentric video foundation models (Ego-VFMs) based on video-language pre-training and propose a parameter-efficient adaptation for egocentric video tasks namely Ego-VPA. It employs a local sparse approximation for each video frame/text feature using the basis prompts and the selected basis prompts are used to synthesize video/text prompts. Since the basis prompts are shared across frames and modalities it models context fusion and cross-modal transfer in an efficient fashion. Experiments show that Ego-VPA excels in lightweight adaptation (with only 0.84% learnable parameters) largely improving over baselines and reaching the performance of full fine-tuning.

Transformers have emerged as the state-of-the-art architecture in medical image registration outperforming convolutional neural networks (CNNs) by addressing their limited receptive fields and overcoming gradient instability in deeper models. Despite their success transformer-based models require substantial resources for training including data memory and computational power which may restrict their applicability for end users with limited resources. In particular existing transformer-based 3D image registration architectures face two critical gaps that challenge their efficiency and effectiveness. Firstly although window-based attention mechanisms reduce the quadratic complexity of full attention by focusing on local regions they often struggle to effectively integrate both local and global information. Secondly the granularity of tokenization a crucial factor in registration accuracy presents a performance trade-off: smaller voxel-size tokens enhance detail capture but come with increased computational complexity higher memory usage and a greater risk of overfitting. We present EFFICIENTMORPH a transformerbased architecture for unsupervised 3D image registration that balances local and global attention in 3D volumes through a plane-based attention mechanism and employs a Hi-Res tokenization strategy with merging operations thus capturing finer details without compromising computational efficiency. Notably EFFICIENTMORPH sets a new benchmark for performance on the OASIS dataset with 16-27x fewer parameters. https://github.com/MedVICLab/Efficient_Morph_Registration

Crack detection particularly from pavement images presents a formidable challenge in computer vision due to inherent complexities such as intensity inhomogeneity intricate topologies low contrast and noisy backgrounds. Automated crack detection is crucial for maintaining the structural integrity of essential infrastructures including buildings pavements and bridges. Existing lightweight methods often face challenges including computational inefficiency complex crack patterns and difficult backgrounds leading to inaccurate detection and impracticality for real-world applications. We propose EfficientCrackNet a lightweight hybrid model combining Convolutional Neural Networks (CNNs) and transformers for precise crack segmentation to address these limitations. EfficientCrackNet integrates depthwise separable convolutions (DSC) layers and MobileViT block to capture global and local features. The model employs an Edge Extraction Method (EEM) for efficient crack edge detection without pretraining and an Ultra-Lightweight Subspace Attention Module (ULSAM) to enhance feature extraction. Extensive experiments on three benchmark datasets Crack500 DeepCrack and GAPs384 demonstrate that EfficientCrackNet achieves superior performance compared to existing lightweight models requiring only 0.26M parameters and 0.483 GFLOPs. The proposed model offers an optimal balance between accuracy and computational efficiency outperforming state-of-the-art lightweight models and providing a robust and adaptable solution for real-world crack segmentation.

Recently transformer-based approaches have shown promising results for semi-supervised video object segmentation. However these approaches typically struggle on long videos due to increased GPU memory demands as they frequently expand the memory bank every few frames. We propose a transformer-based approach named MAVOS that introduces an optimized and dynamic long-term modulated cross-attention (MCA) memory to model temporal smoothness without requiring frequent memory expansion. The proposed MCA effectively encodes both local and global features at various levels of granularity while efficiently maintaining consistent speed regardless of the video length. Extensive experiments on multiple benchmarks LVOS Long-Time Video and DAVIS 2017 demonstrate the effectiveness of our proposed contributions leading to real-time inference and markedly reduced memory demands without any degradation in segmentation accuracy on long videos. Compared to the best existing transformer-based approach our MAVOS increases the speed by 7.6x while significantly reducing the GPU memory by 87% with comparable segmentation performance on short and long video datasets. Notably on the LVOS dataset our MAVOS achieves a J&F score of 63.3% while operating at 37 frames per second (FPS) on a single V100 GPU. Our code and models will be publicly released at https://github.com/Amshaker/MAVOS.

Learned progressive image compression is gaining momentum as it allows improved image reconstruction as more bits are decoded at the receiver. We propose a progressive image compression method in which an image is first represented as a pair of base-quality and top-quality latent representations. Next a residual latent representation is encoded as the element-wise difference between the top and base representations. Our scheme enables progressive image compression with element-wise granularity by introducing a masking system that ranks each element of the residual latent representation from most to least important dividing it into complementary components which can be transmitted separately to the decoder in order to obtain different reconstruction quality. The masking system does not add further parameters nor complexity. At the receiver any elements of the top latent representation excluded from the transmitted components can be independently replaced with the mean predicted by the hyperprior architecture ensuring reliable reconstructions at any intermediate quality level. We also introduced Rate Enhancement Modules (REMs) which refine the estimation of entropy parameters using already decoded components. We obtain results competitive with state-of-the-art competitors while significantly reducing computational complexity decoding time and number of parameters.

The U-Net models have become the predominant architecture within the domain of medical image segmentation. Recent advancements have showcased the potential of incorporating attention-based techniques into U-Net structures. Nevertheless the inclusion of attention mechanisms often leads to a substantial increase in both computational demands and the number of parameters with only a marginal improvement in the performance. This observation raises a critical evaluation of the efficiency associated with the integration of attention modules. In this paper we propose a novel methodology termed Hierarchical Context Interaction (HCI) a parameter-efficient attention-free enhancement that can be seamlessly incorporated into U-Net-based models. Experimental results demonstrate that our proposed HCI module attains state-of-the-art performance on two widely used benchmarks i.e. Medical Segmentation Decathlon Datasets and Synapse Datasets while concurrently sustaining a computationally efficient profile comparable to conventional U-Net configurations.

Annotating scene graphs for images is a time-consuming task resulting in many instances of missing relations within existing datasets. In this paper we introduce the Statistical Relation Distillation (SRD) method to enhance scenegraph datasets. SRD leverages human-annotated relations alongside object-to-object and predicate-to-predicate similarities to reinforce the existence likelihood of scene graph relations. Moreover SRD can augment relational frequency using relations of non-selected object and predicate categories that are usually omitted by scene graph generation (SGG) tasks. The output from SRD derives the prior probability which is combined with model-predicted probabilities to annotate missing relations in training images and sub-sequently re-train SGG models on the augmented dataset. We evaluate our proposed method on Visual Genome and GQA-200 datasets. Experimental results show that training on the augmented dataset enhances the performance of prominent scene-graph generation models. The implementation code is at https://github.com/LUNAProject22/SRD.

Backdoor attacks pose a serious threat to the security of face recognition systems. These involve the insertion of poisoned inputs into the training data to manipulate the model's behavior at inference time and can cause severe consequences such as unauthorized access to secure systems or impersonation of legitimate users. Previous works on backdoor attacks have primarily focused on closed-set classification systems. However open-set face recognition systems are commonly utilized in practical applications which operate fundamentally differently from closedset systems. In this paper we propose two main contributions. First we demonstrate that closed-set backdoor attacks are effective in basic classification scenarios but fail to perform well in complex open-set face recognition tasks. Second we introduce Feature Stabilized Trigger Loss (FSTL) a novel loss function designed to facilitate the learning of backdoors in open-set recognition models. The experiments were conducted on two large-scale datasets using a variety of high-performing face recognition systems and by training with both physical and digital triggers. Since developing effective attack countermeasures requires knowledge of effective attacks this work will enable future works on developing more secure recognition systems.

With their meaningful geometry and omnipresence in the 3D world edges are extremely useful primitives in computer vision. Methods for 3D edge reconstruction have 1) either focused on reconstructing 3D edges by triangulating tracks of 2D line segments across images or 2) more recently learning a 3D edge distance field from multi-view images. The triangulation-based methods struggle to repeatedly detect and robustly match line segments resulting in noisy and incomplete reconstructions in many cases. Methods in the latter class rely on sampling edge points from the learnt implicit field which is limited by the spatial resolution of the voxel grid used for sampling resulting in imprecise points that require refinement. Further such methods require a long training that scales poorly with the size of the scene. In this paper we propose a method that explicitly learns 3D edge points with a 3D Gaussian Splatting representation trained from edge images. Backed by efficient training the proposed method produces results better than or at-par with the current state-of-the-art methods while being an order of magnitude faster. The code is released at https://github.com/kunalchelani/EdgeGaussians.

The application of machine learning to medical ultrasound videos of the heart i.e. echocardiography has recently gained traction with the availability of large public datasets. Traditional supervised tasks such as ejection fraction regression are now making way for approaches focusing more on the latent structure of data distributions as well as generative methods. We propose a model trained exclusively by knowledge distillation either on real or synthetical data involving retrieving masks suggested by a teacher model. We achieve state-of-the-art (SOTA) values on the task of identifying end-diastolic and end-systolic frames. By training the model only on synthetic data it reaches segmentation capabilities close to the performance when trained on real data with a significantly reduced number of weights. A comparison with the 5 main existing methods shows that our method outperforms the others in most cases. We also present a new evaluation method that does not require human annotation and instead relies on a large auxiliary model. We show that this method produces scores consistent with those obtained from human annotations. Relying on the integrated knowledge from a vast amount of records this method overcomes certain inherent limitations of human annotator labeling. Code: https://github.com/GregoirePetit/EchoDFKD

Current methods performing 3D human pose estimation from multi-view still bear several key limitations. First most methods require manual intrinsic and extrinsic camera calibration which is laborious and difficult in many settings. Second more accurate models rely on further training on the same datasets they evaluate severely limiting their generalizability in real-world settings. We address these limitations with EasyRet3D (Easy REconstruction and Tracking in 3D) which simultaneously reconstructs and tracks 3D humans in a global coordinate frame across all views with uncalibrated cameras and videos in the wild. EasyRet3D is a compositional framework that composes our proposed modules (Automatic Calibration module Adaptive Stitching Module and Optimization Module) and off-the-shelf large pre-trained models at intermediate steps to avoid manual intrinsic and extrinsic calibration and task-specific training. EasyRet3D outperforms all existing multi-view 3D tracking or pose estimation methods in Panoptic EgoHumans Shelf and Human3.6M datasets. Codebase and demos will be released on the project website.

We propose an image-adaptive object detection method for adverse weather conditions such as fog and low-light. Our framework employs differentiable preprocessing filters to perform image enhancement suitable for later-stage object detections. Our framework introduces two differentiable filters: a Bezier curve-based pixel-wise (BPW) filter and a kernel-based local (KBL) filter. These filters unify the functions of classical image processing filters and improve performance of object detection. We also propose a domain-agnostic data augmentation strategy using the BPW filter. Our method does not require data-specific customization of the filter combinations parameter ranges and data augmentation. We evaluate our proposed approach called Enhanced Robustness by Unified Image Processing (ERUP)-YOLO by applying it to the YOLOv3 detector. Experiments on adverse weather datasets demonstrate that our proposed filters match or exceed the expressiveness of conventional methods and our ERUP-YOLO achieved superior performance in a wide range of adverse weather conditions including fog and low-light conditions.

Domain Generalization (DG) aims to develop classifiers that can generalize to new unseen data distributions a critical capability when collecting new domain-specific data is impractical. A common DG baseline minimizes the empirical risk on the source domains. Recent studies have shown that this approach known as Empirical Risk Minimization (ERM) can outperform most more complex DG methods when properly tuned. However these studies have primarily focused on a narrow set of hyperparameters neglecting other factors that can enhance robustness and prevent overfitting and catastrophic forgetting properties which are critical for strong DG performance. In our investigation of training data utilization (i.e. duration and setting validation splits) initialization and additional regularizers we find that tuning these previously overlooked factors significantly improves model generalization across diverse datasets without adding much complexity. We call this improved yet simple baseline ERM++. Despite its ease of implementation ERM++ improves DG performance by over 5% compared to prior ERM baselines on a standard benchmark of 5 datasets with a ResNet-50 and over 15% with a ViT-B/16. It also outperforms all state-of-the-art methods on DomainBed datasets with both architectures. Importantly ERM++ is easy to integrate into existing frameworks like DomainBed making it a practical and powerful tool for researchers and practitioners. Overall ERM++ challenges the need for more complex DG methods by providing a stronger more reliable baseline that maintains simplicity and ease of use. Code is available at https://github.com/piotr-teterwak/erm_plusplus.

To accelerate single image super-resolution (SISR) networks on large images (2K-8K) many recent approaches decompose an image into small patches and dynamically determine an execution path according to its difficulty (referred to as a dynamic network). To quantify the hardness of a patch they mainly rely on a handcrafted assessment score e.g. edge which weakly associates a patch's texture with the computational complexity of a SISR model. To address the problem we introduce ENAF - a dynamic network for SISR with an adaptive patch fusion. Built on top of a backbone ENAF incorporates multiple early exits (EEs) to tackle the over-parameterized SISR model. More importantly ENAF plugs a tiny network that estimates PSNR to associate data texture with a computation cost at an EE. Based on the scores ENAF effectively assigns image patches to an exit enhancing the quality-complexity trade-off. Extensive experiments on common datasets with popular SISR backbones demonstrate the effectiveness of ENAF in various settings. The source code will be available.