WACV 2025 论文列表

Precise retina Optical Coherence Tomography (OCT) image classification and segmentation are important for diagnosing various retinal diseases and identifying specific regions. Alongside comprehensive lesion identification reducing the predictive uncertainty of models is crucial for improving reliability in clinical retinal practice. However existing methods have primarily focused on a limited set of regions identified in OCT images and have often faced challenges due to aleatoric and epistemic uncertainty. To address these issues we propose CAMEL (Confidence-Aware Multi-task Ensemble Learning) a novel framework designed to reduce task-specific uncertainty in multi-task learning. CAMEL achieves this by estimating model confidence at both pixel and image levels and leveraging confidence-aware ensemble learning to minimize the uncertainty inherent in single-model predictions. CAMEL demonstrates state-of-the-art performance on a comprehensive retinal OCT image dataset containing annotations for nine distinct retinal regions and nine retinal diseases. Furthermore extensive experiments highlight the clinical utility of CAMEL especially in scenarios with minimal regions significant class imbalances and diverse regions and diseases. Our code is publicly available at: https://github.com/DSAIL-SKKU/CAMEL.

Indoor 3D domain adaptation for semantic segmentation is an understudied task. The first unsupervised sim-to-real benchmark was only proposed recently. Existing methods try to modify the source domain data by simulating the occlusion and noise pattern of the target domain. However this methodology unrealistically demands a clear definition of the real-world data patterns and is highly dependent on the simulation quality. In this paper we propose a novel adaptation framework via Context-aware Augmentation and Consistency Enforcement (CACE). Our CACE framework consists of two modules a space and context-aware augmentation module that is invariant of target data pattern and domain gaps and a carefully designed self-supervision module that maximizes the utility of the augmented data. Our CACE surpasses the state-of-the-art method by over 6% on the indoor 3D sim-to-real benchmark 3D-FRONT - ScanNet.

Camera calibration and localization sometimes simply named camera calibration enables many applications in the context of soccer broadcasting for instance regarding the interpretation and analysis of the game or the insertion of augmented reality graphics for storytelling or refereeing purposes. To contribute to such applications the research community has typically focused on single-view calibration methods leveraging the near-omnipresence of soccer field markings in wide-angle broadcast views but leaving all temporal aspects if considered at all to general-purpose tracking or filtering techniques. Only a few contributions have been made to leverage any domain-specific knowledge for this tracking task and as a result there lacks a truly performant and off-the-shelf camera tracking system tailored for soccer broadcasting specifically for elevated tripod-mounted cameras around the stadium. In this work we present such a system capable of addressing the task of soccer broadcast camera tracking efficiently robustly and accurately outperforming by far the most precise methods of the state-of-the-art. By combining the available open-source soccer field detectors with carefully designed camera and tripod models our tracking system BroadTrack halves the mean reprojection error rate and gains more than 15% in terms of Jaccard index for camera calibration on the SoccerNet dataset. Furthermore as the SoccerNet dataset videos are relatively short (30 seconds) we also present qualitative results on a 20-minute broadcast clip to showcase the robustness and the soundness of our system.

Visual place recognition methods struggle with occlusion and partial visual overlaps. We propose a novel visual place recognition approach based on overlap prediction called VOP shifting from traditional reliance on global image similarities and local features to image overlap prediction. VOP proceeds co-visible image sections by obtaining patch-level embeddings using a Vision Transformer backbone and establishing patch-to-patch correspondences without requiring expensive feature detection and matching. Our approach uses a voting mechanism to assess overlap scores for potential database images. It provides a nuanced image retrieval metric in challenging scenarios. Experimental results show that VOP leads to more accurate relative pose estimation and localization results on the retrieved image pairs than state-of-the-art baselines on a number of large-scale real-world indoor and outdoor benchmarks. The code is available at https://github.com/weitong8591/vop.git.

This paper studies semi-supervised learning of video action detection (VAD) which assumes that only a small portion of training videos are labeled and the others remain unlabeled. The existing semi-supervised methods for VAD mainly focus on leveraging spatial context of unlabeled video lacking its exploration of temporal context. To resolve this we present a novel semi-supervised learning framework that effectively incorporates spatio-temporal context during training. We first introduce a new augmentation strategy called temporal cross-view augmentation to achieve robust representation across clips depicting the same action but not aligned on the time axis. We also propose a new context fusion method called global-local context fusion that effectively utilizes the spatio-temporal context of videos to enhances the features of each frame by incorporating those of other frames within a clip; this method aids in actively leveraging spatio-temporal context of video leading to significant performance improvement. Our framework was evaluated on UCF101-24 and JHMDB-21 where it outperformed all existing methods in every evaluation setting.

Recently diffusion-based blind super-resolution (SR) methods have shown great ability to generate high-resolution images with abundant high-frequency detail but the detail is often achieved at the expense of fidelity. Meanwhile another line of research focusing on rectifying the reverse process of diffusion models (i.e. diffusion guidance) has demonstrated the power to generate high-fidelity results for non-blind SR. However these methods rely on known degradation kernels making them difficult to apply to blind SR. To address these issues we introduce degradation-aware models that can be integrated into the diffusion guidance framework eliminating the need to know degradation kernels. Additionally we propose two novel techniques--input perturbation and guidance scalar--to further improve our performance. Extensive experimental results show that our proposed method has superior performance over state-of-the-art methods on blind SR benchmarks.

Blind image deblurring is the process of recovering a sharp image from a blurred one without prior knowledge about the blur kernel. It is a small data problem since the key challenge lies in estimating the unknown degrees of blur from a single image or limited data instead of learning from large datasets. The solution depends heavily on developing algorithms that effectively model the image degradation process. We introduce a method that leverages a prior which targets the blur kernel to achieve effective deblurring across a wide range of image types. In our extensive empirical analysis our algorithm achieves results that are competitive with the state-of-the-art blind image deblurring algorithms and it offers up to two times faster inference making it a highly efficient solution.

Deep learning and computer vision have experienced significant advancements particularly in critical applications such as autonomous driving and real-time surveillance where object detection (OD) plays a pivotal role. Ensuring the accuracy and speed of these systems is paramount to prevent accidents or failures. Recently latency-based attacks have emerged as a new threat driven by the essential need for real-time performance in various applications. These attacks target model responsiveness to disrupt system performance without necessarily compromising accuracy. Our preliminary experiments show that introducing just a few bit flips to key parameters in OD models can significantly increase latency degrading performance. Meanwhile recent advancements in memory-based attacks such as Row Hammer demonstrate the ability to conveniently introduce bit flips at desired locations without physical hardware interaction. Based on the observations we propose a novel attack on OD models that leverages row-hammer to introduce bit-flips via side channels targeting the non-maximum suppression (NMS) filter and significantly increasing latency. Unlike previous methods that modify input data our technique ensures efficiency by minimizing bit-flips through critical path exploitation and achieves practical applicability with only a subset of validation data. Experiments across various datasets and models validate our approach demonstrating latency increases up to 71.6 ms (20.4x) with just 31 bit-flips.

Recent advancements in 3D human pose estimation from single-camera images and videos have relied on parametric models like SMPL. However these models oversimplify anatomical structures limiting their accuracy in capturing true joint locations and movements which reduces their applicability in biomechanics healthcare and robotics. Biomechanically accurate pose estimation on the other hand typically requires costly marker-based motion capture systems and optimization techniques in specialized labs. To bridge this gap we propose BioPose a novel learning-based framework for predicting biomechanically accurate 3D human pose directly from monocular videos. BioPose includes three key components: a multi-query human mesh recovery model (MQ-HMR) a neural inverse kinematics (NeurIK) model and a 2D-informed pose refinement technique. MQ-HMR leverages a multi-query deformable transformer to extract multi-scale fine-grained image features enabling precise human mesh recovery. NeurIK treats the mesh vertices as virtual markers applying a spatial-temporal network to regress biomechanically accurate 3D poses under anatomical constraints. To further improve 3D pose estimations a 2D-informed refinement step optimizes the query tokens during inference by aligning the 3D structure with 2D pose observations. Experiments on benchmark datasets demonstrate that BioPose significantly outperforms state-of-the-art methods.

Crop biomass offers crucial insights into plant health and yield making it essential for crop science farming systems and agricultural research. However current measurement methods which are labor-intensive destructive and imprecise hinder large-scale quantification of this trait. To address this limitation we present a biomass prediction network (BioNet) designed for adaptation across different data modalities including point clouds and drone imagery. Our BioNet utilizing a sparse 3D convolutional neural network (CNN) and a transformer-based prediction module processes point clouds and other 3D data representations to predict biomass. To further extend BioNet for drone imagery we integrate a neural feature field (NeFF) module enabling 3D structure reconstruction and the transformation of 2D semantic features from vision foundation models into the corresponding 3D surfaces. For the point cloud modality BioNet demonstrates superior performance on two public datasets with an approximate 6.1% relative improvement (RI) over the state-of-the-art. In the RGB image modality the combination of BioNet and NeFF achieves a 7.9% RI. Additionally the NeFF-based approach utilizes inexpensive portable drone-mounted cameras providing a scalable solution for large field applications.

A key challenge in visible-infrared person re-identification (V-I ReID) is training a backbone model capable of effectively addressing the significant discrepancies across modalities. State-of-the-art methods that generate a single intermediate bridging domain are often less effective as this generated domain may not adequately capture sufficient common discriminant information. This paper introduces Bidirectional Multi-step Domain Generalization (BMDG) a novel approach for unifying feature representations across diverse modalities. BMDG creates multiple virtual intermediate domains by learning and aligning body part features extracted from both I and V modalities. In particular our method aims to minimize the cross-modal gap in two steps. First BMDG aligns modalities in the feature space by learning shared and modality-invariant body part prototypes from V and I images. Then it generalizes the feature representation by applying bidirectional multi-step learning which progressively refines feature representations in each step and incorporates more prototypes from both modalities. Based on these prototypes multiple bridging steps enhance the feature representation. Experiments conducted on V-I ReID datasets indicate that our BMDG approach can outperform state-of-the-art part-based and intermediate generation methods and can be integrated into other part-based methods to enhance their V-I ReID performance.

The need for more transparent face recognition (FR) along with other visual-based decision-making systems has recently attracted more attention in research society and industry. The reasons why two face images are matched or not matched by a deep learning-based face recognition system are not obvious due to the high number of parameters and the complexity of the models. However it is important for users operators and developers to ensure trust and accountability of the system and to analyze drawbacks such as biased behavior. While many previous works use spatial semantic maps to highlight the regions that have a significant influence on the decision of the face recognition system frequency components which are also considered by CNNs are neglected. In this work we take a step forward and investigate explainable face recognition in the unexplored frequency domain. This makes this work the first to propose explainability of verification-based decisions in the frequency domain thus explaining the relative influence of the frequency components of each input toward the obtained outcome. To achieve this we manipulate face images in the spatial frequency domain and investigate the impact on verification outcomes. In extensive quantitative experiments along with investigating two special scenarios cases cross-resolution FR and morphing attacks (the latter in supplementary material) we observe the applicability of our proposed frequency-based explanations.

Vision transformers have excelled in various computer vision tasks but mostly rely on rigid input sampling using a fixed-size grid of patches. It limits their applicability in real-world problems such as active visual exploration where patches have various scales and positions. Our paper addresses this limitation by formalizing the concept of input elasticity for vision transformers and introducing an evaluation protocol for measuring this elasticity. Moreover we propose modifications to the transformer architecture and training regime which increase its elasticity. Through extensive experimentation we spotlight opportunities and challenges associated with such architecture.

The primary challenge in Video Object Detection (VOD) is effectively exploiting temporal information to enhance object representations. Traditional strategies such as aggregating region proposals often suffer from feature variance due to the inclusion of background information. We introduce a novel instance mask-based feature aggregation approach significantly refining this process and deepening the understanding of object dynamics across video frames. We present FAIM a new VOD method that enhances temporal Feature Aggregation by leveraging Instance Mask features. In particular we propose the lightweight Instance Feature Extraction Module (IFEM) to learn instance mask features and the Temporal Instance Classification Aggregation Module (TICAM) to aggregate instance mask and classification features across video frames. Using YOLOX as a base detector FAIM achieves 87.9% mAP on the ImageNet VID dataset at 33 FPS on a single 2080Ti GPU setting a new benchmark for the speed-accuracy trade-off. Additional experiments on multiple datasets validate that our approach is robust method-agnostic and effective in multi-object tracking demonstrating its broader applicability to video understanding tasks.

Generative diffusion models have emerged as a powerful tool for high-quality image synthesis yet their iterative nature demands significant computational resources. This paper proposes an efficient time step sampling method based on an image spectral analysis of the diffusion process aimed at optimizing the denoising process. Instead of the traditional uniform distribution-based time step sampling we introduce a Beta distribution-like sampling technique that prioritizes critical steps in the early and late stages of the process. Our hypothesis is that certain steps exhibit significant changes in image content while others contribute minimally. We validated our approach using Fourier transforms to measure frequency response changes at each step revealing substantial low-frequency changes early on and high-frequency adjustments later. Experiments with ADM and Stable Diffusion demonstrated that our Beta Sampling method consistently outperforms uniform sampling achieving better FID and IS scores and offers competitive efficiency relative to state-of-the-art methods like AutoDiffusion. This work provides a practical framework for enhancing diffusion model efficiency by focusing computational resources on the most impactful steps with potential for further optimization and broader application.

Vision language models (VLMs) have shown strong zero-shot generalization across various tasks especially when integrated with large language models (LLMs). However their ability to comprehend rhetorical and persuasive visual media such as advertisements remains understudied. Ads often employ atypical imagery using surprising object juxtapositions to convey shared properties. For example Fig. 1 (e) shows a beer with a feather-like texture. This requires advanced reasoning to deduce that this atypical representation signifies the beer's lightness. We introduce three novel tasks Multi-label Atypicality Classification Atypicality Statement Retrieval and Atypical Object Recognition to benchmark VLMs' understanding of atypicality in persuasive images. We evaluate how well VLMs use atypicality to infer an ad's message and test their reasoning abilities by employing semantically challenging negatives. Finally we pioneer atypicality-aware verbalization by extracting comprehensive image descriptions sensitive to atypical elements. Findings reveal that: (1) VLMs lack advanced reasoning capabilities compared to LLMs; (2) simple effective strategies can extract atypicality-aware information leading to comprehensive image verbalization; (3) atypicality aids persuasive ad understanding. Code and data is available at aysanaghazadeh.github.io/PersuasiveAdVLMBenchmark/

The advancement of makeup transfer editing and image encoding has demonstrated their effectiveness and superior quality. However existing makeup works primarily focus on low-dimensional features such as color distributions and patterns limiting their versatillity across a wide range of makeup applications. Futhermore existing high-dimensional latent encoding methods mainly target global features such as structure and style and are less effective for tasks that require detailed attention to local color and pattern features of makeup. To overcome these limitations we propose BeautyBank a novel makeup encoder that disentangles pattern features of bare and makeup faces. Our method encodes makeup features into a high-dimensional space preserving essential details necessary for makeup reconstruction and broadening the scope of potential makeup research applications. We also propose a Progressive Makeup Tuning (PMT) strategy specifically designed to enhance the preservation of detailed makeup features while preventing the inclusion of irrelevant attributes. We further explore novel makeup applications including facial image generation with makeup injection and makeup similarity measure. Extensive empirical experiments validate that our method offers superior task adaptability and holds significant potential for widespread application in various makeup-related fields. Furthermore to address the lack of large-scale high-quality paired makeup datasets in the field we constructed the Bare-Makeup Synthesis Dataset (BMS) comprising 324000 pairs of 512x512 pixel images of bare and makeup-enhanced faces.

In recent years image restoration using large-scale latent diffusion generative models (DGM) has attracted increasing attention and achieved significant progress. Most of these latent DGM-based image restoration methods require predicting the original clean image in each iteration which is then used to estimate the image for the next iteration. However these predicted original clean images are often inaccurate leading to errors in the subsequent image estimation. In other words there is a significant deviation between the final sampling restoration trajectory and the ground truth trajectory. The purpose of this paper is to narrow the gap between these two trajectories and enhance the performance of image restoration. We propose the Bayesian Optimal Latent Projection (BOLP) algorithm which identifies the optimal random noise within the Gaussian distribution to iteratively correct the estimated image at each step thereby minimizing the distance to the ground truth image. Experiments in deblurring super-resolution and inpainting on FFHQ and ImageNet datasets demonstrate that the BOLP outperforms the previously established best algorithms and sets a new state of the art.

Accurate perception systems are crucial for intelligent vehicle motion planning and control. Cooperative perception can significantly enhance the perception capabilities of autonomous vehicles by sharing information beyond the field of view of individual vehicles. However most recent studies on cooperative perception have overlooked the bandwidth limitations of information sharing. To address this issue this paper presents a novel multi-agent cooperative 3D object detection framework to improve the detection accuracy of connected autonomous vehicles (CAVs) in bandwidth-constrained wireless communication environments. The framework includes an attention-based communication mechanism to select optimal communication partners and a foreground-background separation strategy to prioritize key information. The effectiveness of the proposed framework is evaluated on three benchmark datasets CARLA-3D V2XSet and OPV2V demonstrating the state-of-the-art performance-bandwidth trade-off.