WACV 2025 论文列表

Synthesizing accurate hands-object interactions (HOI) is critical for applications in Computer Vision Augmented Reality (AR) and Mixed Reality (MR). Despite recent advances the accuracy of reconstructed or generated HOI leaves room for refinement. Some techniques have improved the accuracy of dense correspondences by shifting focus from generating explicit contacts to using rich HOI fields. Still they lack full differentiability or continuity and are tailored to specific tasks. In contrast we present a Coarse Hand-Object Interaction Representation (CHOIR) a novel versatile and fully differentiable field for HOI modelling. CHOIR leverages discrete unsigned distances for continuous shape and pose encoding alongside multivariate Gaussian distributions to represent dense contact maps with few parameters. To demonstrate the versatility of CHOIR we design JointDiffusion a diffusion model to learn a grasp distribution conditioned on noisy hand-object interactions or only object geometries for both refinement and synthesis applications. We demonstrate JointDiffusion's improvements over the SOTA in both applications: it increases the contact F1 score by 5% for refinement and decreases the sim. displacement by 46% for synthesis. Our experiments show that JointDiffusion with CHOIR yields superior contact accuracy and physical realism compared to SOTA methods designed for specific tasks. Project page: https://theomorales.com/CHOIR

Average-K classification is an alternative to top-K classification in which the number of labels returned varies with the ambiguity of the input image but must average to K over all the samples. A simple method to solve this task is to threshold the softmax output of a model trained with the cross-entropy loss. This approach is theoretically proven to be asymptotically consistent but it is not guaranteed to be optimal for a finite set of samples. In this paper we propose a new loss function based on a multi-label classification head in addition to the classical softmax. This second head is trained using pseudo-labels generated by thresholding the softmax head while guaranteeing that K classes are returned on average. We show that this approach allows the model to better capture ambiguities between classes and as a result to return more consistent sets of possible classes. Experiments on two datasets from the literature demonstrate that our approach outperforms the softmax baseline as well as several other loss functions more generally designed for weakly supervised multi-label classification. The gains are larger the higher the uncertainty especially for classes with few samples.

This work addresses the challenges associated with the use of glosses in both Sign Language Translation (SLT) and Sign Language Production (SLP). While the glosses have long been used as a bridge between sign language and spoken language they come with two major limitations that impede the advancement of sign language systems. First annotating the glosses is a labor-intensive and time-consuming process which limits the scalability of datasets. Second the glosses oversimplify sign language by stripping away its spatio-temporal dynamics reducing complex signs to basic labels and missing the subtle movements essential for precise interpretation. To address these limitations we introduce Universal Gloss-level Representation (UniGloR) a framework designed to capture the spatio-temporal features inherent in sign language providing a more dynamic and detailed alternative to the use of the glosses. The core idea of UniGloR is simple yet effective: We derive dense spatio-temporal representations from sign keypoint sequences using self-supervised learning and seamlessly integrate them into SLT and SLP tasks. Our experiments in a keypoint-based setting demonstrate that UniGloR either outperforms or matches the performance of previous SLT and SLP methods on two widely-used datasets: PHOENIX14T and How2Sign. Code is available at https://github.com/eddie-euijun-hwang/UniGloR.

Class-Agnostic Counting (CAC) seeks to accurately count objects in a given image with only a few reference examples. While previous methods achieving this relied on additional training recent efforts have shown that it's possible to accomplish this without training by utilizing pre-existing foundation models particularly the Segment Anything Model (SAM) for counting via instance-level segmentation. Although promising current training-free methods still lag behind their training-based counterparts in terms of performance. In this research we present a straightforward training-free solution that effectively bridges this performance gap serving as a strong baseline. The primary contribution of our work lies in the discovery of four key technologies that can enhance performance. Specifically we suggest employing a superpixel algorithm to generate more precise initial point prompts utilizing an image encoder with richer semantic knowledge to replace the SAM encoder for representing candidate objects and adopting a multiscale mechanism and a transductive prototype scheme to update the representation of reference examples. By combining these four technologies our approach achieves significant improvements over existing training-free methods and delivers performance on par with training-based ones.

We investigate how to characterize semantic significance (SS) in detecting image manipulations (IMD) for media forensics. We introduce the Characterization of Semantic Impact for IMD (CSI-IMD) dataset which focuses on localizing and evaluating the semantic impact of image manipulations to counter advanced generative techniques. Our evaluation of 10 state-of-the-art IMD and localization methods on CSI-IMD reveals key insights. Unlike existing datasets CSI-IMD provides detailed semantic annotations beyond traditional manipulation masks aiding in the development of new defensive strategies. The dataset features manipulations from advanced generation methods offering various levels of semantic significance. It is divided into two parts: a gold-standard set of 1000 manually annotated manipulations with high-quality control and an extended set of 500000 automated manipulations for large-scale training and analysis. We also propose a new SS-focused task to assess the impact of semantically targeted manipulations. Our experiments show that current IMD methods struggle with manipulations created using stable diffusion with TruFor and Cat-Net performing the best among those tested. The CSI-IMD dataset will become available at https://github.com/csiimd/csiimd.

Achieving timely and accurate snow detection on road surfaces in extreme weather conditions is vital for both transportation and computer vision applications. However conventional object detection models particularly those designed for small targets fall short in addressing the challenge that is posed by special regional-level multi-scale recognition task. To this end an end-to-end precise and swift road surface snow detection architecture termed the Resource-Saving Snow Detect Model (RSSD) that includes a multidimensional directional attention mechanism is proposed. In this model we designed three dedicated modules namely Multi-dimensional Bidirectional Attention Module (MDBA) Split-EMA-Convolution (SEC) and Equal Split Convolution (ESC) to address the essential feature extraction and fusion tasks in snow detection. MDBA is able to promote lateral interaction and comprehensive feature fusion across scales while SEC can not only enhance feature extraction for regional awareness but also reduces computational load making it efficient under minimal computational power consumption. ESC preserves feature height fusion while significantly reducing computational costs thereby enhancing the real-time detection capability of the model. In experimental evaluations conducted with data collected by in-vehicle cameras from various roads in the United States and Canada the results demonstrate higher detection accuracy and speed compared to the latest Transformer-based real-time object detection methods and other exiting methods in the literature. Furthermore we validated the model's performance and data sensitivity through semi-supervised learning with 50000 unlabeled images. This research holds significant implications for winter road traffic and provides valuable insights for similar computer vision tasks.

Utilizing patch-based transformers for unstructured geometric data such as polygon meshes presents significant challenges primarily due to the absence of a canonical ordering and variations in input sizes. Prior approaches to handling 3D meshes and point clouds have either relied on computationally intensive node-level tokens for large objects or resorted to resampling to standardize patch size. Moreover these methods generally lack a geometry-aware stable Structural Embedding (SE) often depending on simplistic absolute SEs such as 3D coordinates which compromise isometry invariance essential for tasks like semantic segmentation. In our study we meticulously examine the various components of a geometry-aware 3D mesh transformer from tokenization to structural encoding assessing the contribution of each. Initially we introduce a spectral-preserving tokenization rooted in algebraic multi-grid methods. Subsequently we detail an approach for embedding features at the patch level accommodating patches with variable node counts. Through comparative analyses against a baseline model employing simple point-wise Multi-Layer Perceptrons (MLP) our research highlights critical insights: 1) the importance of structural and positional embeddings facilitated by heat diffusion in general 3D mesh transformers; 2) the effectiveness of novel components such as geodesic masking and feature interaction via cross-attention in enhancing learning; and 3) the superior performance and efficiency of our proposed methods in challenging segmentation and classification tasks.

Exemplar-free class-incremental learning (EFCIL) aims to classify streaming data without storing examples from the past. Recent EFCIL works suggest that (i) models pre-trained with large amounts of data should be used to initialize learning (ii) self-supervised learned transformers generalize better than supervised convolutional models (iii) adding generated data to the pre-training dataset can improve incremental accuracy. In this article we question the above assertions by comprehensively evaluating various initial training strategies combined with four EFCIL algorithms using four large-scale datasets. Our results indicate that: (i) pre-trained models are preferable when the domain of the incremental classification task is well represented in the pre-training datasets but training with initial data remains useful when the domain shift is significant (ii) supervised convolutional networks remain competitive particularly when improving representation transferability using data augmentation or a projector (iii) adding classes from an external dataset to train the initial model boosts performance when the initial set of classes is small but has a limited effect otherwise (iv) additional classes generated with a diffusion model are not necessarily more useful than a well-chosen set of ImageNet classes to improve model transferability. We provide a nuanced analysis of these results and formulate recommendations to facilitate the practical adoption of EFCIL algorithms.

Weakly Supervised Object Localization (WSOL) allows training deep learning models for classification and localization (LOC) using only global class-level labels. The absence of bounding box (bbox) supervision during training raises challenges in the literature for hyper-parameter tuning model selection and evaluation. WSOL methods rely on a validation set with bbox annotations for model selection and a test set with bbox annotations for threshold estimation for producing bboxes from localization maps. This approach however is not aligned with the WSOL setting as these annotations are typically unavailable in real-world scenarios. Our initial empirical analysis shows a significant decline in LOC performance when model selection and threshold estimation rely solely on class labels and the image itself respectively compared to using manual bbox annotations. This highlights the importance of incorporating bbox labels for optimal model performance. In this paper a new WSOL evaluation protocol is proposed that provides LOC information without the need for manual bbox annotations. In particular we generated noisy pseudo-boxes from a pretrained off-the-shelf region proposal method such as Selective Search CLIP and RPN for model selection. These bboxes are also employed to estimate the threshold from LOC maps circumventing the need for test-set bbox annotations. Our experiments with several WSOL methods on challenging natural and medical image datasets show that using the proposed pseudo-bboxes for validation facilitates the model selection and threshold estimation with LOC performance comparable to models selected using GT bboxes on the validation set and threshold estimation on the test set. It also outperforms models selected using class-level labels and then dynamically thresholded with only LOC maps.

Measuring the accuracy of face recognition (FR) systems is essential for improving performance and ensuring responsible use. Accuracy is typically estimated using large annotated datasets which are costly and difficult to obtain. We propose a novel method for 1:1 face verification that benchmarks FR systems quickly and without manual annotation starting from approximate labels (e.g. from web search results). Unlike previous methods for training set label cleaning ours leverages the embedding representation of the models being evaluated achieving high accuracy in smaller-sized test datasets. Our approach reliably estimates FR accuracy and ranking significantly reducing the time and cost of manual labeling. We also introduce the first public benchmark of five FR cloud services revealing demographic biases particularly lower accuracy for Asian women. Our rapid test method can democratize FR testing promoting scrutiny and responsible use of the technology. Our method is provided as a publicly accessible tool at https://github.com/caltechvisionlab/frt-rapid-test.

Discussions of minimum parking requirement policies often include maps of parking lots which are time-consuming to construct manually. Open-source datasets for such parking lots are scarce particularly for US cities. This paper introduces the idea of using Near-Infrared (NIR) channels as input and several post-processing techniques to improve the prediction of off-street surface parking lots using satellite imagery. We constructed two datasets with 12617 image-mask pairs each: one with 3-channel (RGB) and another with 4-channel (RGB + NIR). The datasets were used to train five deep learning models (OneFormer Mask2Former SegFormer DeepLabV3 and FCN) for semantic segmentation classifying images to differentiate between parking and non-parking pixels. Our results demonstrate that the NIR channel improved accuracy because parking lots are often surrounded by grass--even though the NIR channel needed to be upsampled from a lower resolution. Post-processing including eliminating erroneous "holes" simplifying edges and removing road and building footprints further improved the accuracy. Best model OneFormer trained on 4-channel input and paired with post-processing techniques achieves a mean Intersection over Union (mIoU) of 84.9% and a pixel-wise accuracy of 96.3%.

Iris-based biometric systems are vulnerable to presentation attacks (PAs) where adversaries present physical artifacts (e.g. printed iris images textured contact lenses) to defeat the system. This has led to the development of various presentation attack detection (PAD) algorithms which typically perform well in intra-domain settings. However they often struggle to generalize effectively in cross-domain scenarios where training and testing employ different sensors PA instruments and datasets. In this work we use adversarial training samples of both bonafide irides and PAs to improve the cross-domain performance of a PAD classifier. The novelty of our approach lies in leveraging transformation parameters from classical data augmentation schemes (e.g. translation rotation) to generate adversarial samples. We achieve this through a convolutional autoencoder ADV-GEN that inputs original training samples along with a set of geometric and photometric transformations. The transformation parameters act as regularization variables guiding ADV-GEN to generate adversarial samples in a constrained search space. Experiments conducted on the LivDet-Iris 2017 database comprising four datasets and the LivDet-Iris 2020 dataset demonstrate the efficacy of our proposed method. The code is available at https://github.com/iPRoBe-lab/ADV-GEN-IrisPAD.

The efficacy of deep learning-based Computer-Aided Diagnosis (CAD) methods for skin diseases relies on analyzing multiple data modalities (i.e. clinical+dermoscopic images and patient metadata) and addressing the challenges of multi-label classification. Current approaches tend to rely on limited multi-modal techniques and treat the multi-label problem as a multiple multi-class problem overlooking issues related to imbalanced learning and multi-label correlation. This paper introduces the innovative Skin Lesion Classifier utilizing a Multi-modal Multi-label TransFormer-based model (SkinM2Former). For multi-modal analysis we introduce the Tri-Modal Cross-attention Transformer (TMCT) that fuses the three image and metadata modalities at various feature levels of a transformer encoder. For multi-label classification we introduce a multi-head attention (MHA) module to learn multi-label correlations complemented by an optimisation that handles multi-label and imbalanced learning problems. SkinM2Former achieves a mean average accuracy of 77.27% and a mean diagnostic accuracy of 77.85% on the public Derm7pt dataset outperforming state-of-the-art (SOTA) methods.

Existing image inpainting methods have shown impressive completion results for low-resolution images. However most of these algorithms fail at high resolutions and require powerful hardware limiting their deployment on edge devices. Motivated by this we propose the first baseline for REal-Time High-resolution image INpainting on Edge Devices (RETHINED) that is able to inpaint at ultra-high-resolution and can run in real-time (30ms) in a wide variety of mobile devices. A simple yet effective novel method formed by a lightweight Convolutional Neural Network (CNN) to recover structure followed by a resolution-agnostic patch replacement mechanism to provide detailed texture. Specially our pipeline leverages the structural capacity of CNN and the high-level detail of patch-based methods which is a key component for high-resolution image inpainting. To demonstrate the real application of our method we conduct an extensive analysis on various mobile-friendly devices and demonstrate similar inpainting performance while being 100 times faster than existing state-of-the-art methods. Furthemore we release DF8K-Inpainting the first free-from mask UHD inpainting dataset.

Split computing ( split learning) is a promising approach to deep learning models for resource-constrained edge computing systems where weak sensor (mobile) devices are wirelessly connected to stronger edge servers through channels with limited communication capacity. State-of-the-art work on split computing presents methods for single tasks such as image classification object detection or semantic segmentation. The application of existing methods to multi-task problems degrades model accuracy and/or significantly increase runtime latency. In this study we propose Ladon the first multi-task-head supervised compression model for multi-task split computing. Experimental results show that the multi-task supervised compression model either outperformed or rivaled strong lightweight baseline models in terms of predictive performance for ILSVRC 2012 COCO 2017 and PASCAL VOC 2012 datasets while learning compressed representations at its early layers. Furthermore our models reduced end-to-end latency (by up to 95.4%) and energy consumption of mobile devices (by up to 88.2%) in multi-task split computing scenarios.

Change detection in remote sensing images is an essential tool for analyzing a region at different times. It finds varied applications in monitoring environmental changes man-made changes as well as corresponding decision-making and prediction of future trends. Deep learning methods like Convolutional Neural Networks (CNNs) and Transformers have achieved remarkable success in detecting significant changes given two images at different times. In this paper we propose a Mamba-based Change Detector (M-CD) that segments out the regions of interest even better. Mamba-based architectures demonstrate linear-time training capabilities and an improved receptive field over transformers. Our experiments on four widely used change detection datasets demonstrate significant improvements over existing state-of-the-art (SOTA) methods. Code: https://github.com/JayParanjape/M-CD

In autonomous driving systems online vehicle-to-sensor (v2s) calibration is a critical component for ensuring safe perception-based control. Since sensor pose may shift during the life-time of a vehicle online calibration is essential to maintain safe driving conditions. To this end this paper introduces Epipoles as a 3D Directional Compass (E3DC) a sensor-agnostic v2s online calibration method. Leveraging the nonholonomic nature of vehicles a hand-eye constraint between the vehicle and the sensor naturally emerges. Consequently we require only the sensor's data to determine the v2s extrinsic rotation. More specifically since we only require sensor odometry estimates to perform v2s calibration E3DC can leverage off-the-shelf odometry estimation pipelines. This offers vast flexibility and wide applicability as the odometry estimation pipeline can be tailored to the specific sensor type and driving environment. We demonstrate that our method is robust and achieves state-of-the-art performance on both the KITTI dataset and a new dataset which will be made publicly available. To the best of our knowledge this is the first v2s calibration dataset for autonomous driving scenarios.

Large text-to-image models have revolutionized the ability to generate imagery using natural language. However particularly unique or personal visual concepts such as pets and furniture will not be captured by the original model. This has led to interest in how to personalize a text-to-image model. Despite significant progress this task remains a formidable challenge particularly in preserving the subject's identity. Most researchers attempt to address this issue by modifying model architectures. These methods are capable of keeping the subject structure and color but fail to preserve identity details. Towards this issue our approach takes a data-centric perspective. We introduce a novel regularization dataset generation strategy on both the text and image level. This strategy enables the model to preserve fine details of the desired subjects such as text and logos. Our method is architecture-agnostic and can be flexibly applied on various text-to-image models. We show on established benchmarks that our data-centric approach forms the new state of the art in terms of identity preservation and text alignment.

We propose a conic transformation method to solve the Perspective-Three-Point (P3P) problem. In contrast to the current state-of-the-art solvers which formulate the P3P problem by intersecting two conics and constructing a degenerate conic to find the intersection our approach builds upon a new formulation based on a transformation that maps the two conics to a new coordinate system where one of the conics becomes a standard parabola in a canonical form. This enables expressing one variable in terms of the other variable and as a consequence substantially simplifies the problem of finding the conic intersection. Moreover the polynomial coefficients are fast to compute and we only need to determine the real-valued intersection points which avoids the requirement of using computationally expensive complex arithmetic. While the current state-of-the-art methods reduce the conic intersection problem to solving a univariate cubic equation our approach despite resulting in a quartic equation is still faster thanks to this new simplified formulation. Extensive evaluations demonstrate that our method achieves higher speed while maintaining robustness and stability comparable to state-of-the-art methods.