WACV 2025 Papers

Recent advancements in bird's-eye view (BEV) perception models have highlighted the superior performance of LiDAR-camera fusion systems over single-modality approaches garnering considerable interest in the field. Despite the progress the integration of temporal information a technique that has considerably benefitted camera-only BEV models remains underexplored for LiDAR-camera fusion. This paper presents Q-TempFusion a novel approach for temporal multi-sensor fusion designed to enhance the BEV model's inference speed while keeping high predictive performance compared with the current state-of-the-art. Moreover we are the first to make the multi-modality BEV model profiling on hardware devices. To address the challenges of substantial memory demands and non-trivial latency that hinder deployment in on-vehicle systems particularly when temporal dynamics are incorporated into complex multi-sensor models we introduce an activation-aware quantization framework to generate the fully 8-bit quantized Q-TempFusion model based on the profiling result which can be directly deployed to target devices with negligible detection performance degradation. Our experiments show that our Q-TempFusion (8-bit) achieves 70.3% mAP and 72.7% NDS with 3x - 18x FPS improvement over leading multi-modality baselines and the Q-TempFusion (32-bit) achieves 72.1% mAP and 74.8% NDS comparable to SOTA multi-modality approaches. The results suggest that Q-TempFusion is a promising step toward real-time multi-sensor BEV applications setting a new benchmark for efficient and reliable perception.

Sustainable forest management is a cornerstone of climate and environmental action. Responsible management relies on forest models such as biomass or fire vulnerability estimates which depend on mapping the spatial distribution of tree species. Because forest mapping relies heavily on visual identification it is a time-consuming endeavor that would benefit from more automation. To develop scalable automated methods researchers need suitable benchmarks but unfortunately tree species classification datasets are scarce. Most of them contain only imagery data which capture signal only at the canopy level and ignore the three-dimensional structure of the forest making it very difficult to distinguish between different species. Lidar which penetrates the canopy and captures the geometry of trees provides a rich signal for distinguishing tree species. As an increasingly available and cost-effective remote sensing modality it could become the new standard for large-scale tree species mapping. However current Lidar benchmarks for tree species classification are extremely limited and all insufficient in size and diversity; with less than 1500 annotated trees from at most a dozen different sites they fall short from the requirements of deep learning development. To fill this data gap we release PureForest: a large-scale open multimodal dataset designed for tree species classification from both Aerial Lidar Scanning point clouds and Very High Resolution aerial images. PureForest covers 339 km2 in 449 different monospecific forests with verified labels for 18 tree species grouped into 13 semantic classes. It is the largest and most comprehensive Lidar dataset for tree species identification. By making PureForest publicly available we hope to provide a challenging benchmark to support the development of deep learning approaches for tree species identification from Lidar and aerial imagery. In this data paper we describe the annotation workflow and the dataset and we establish a baseline performance from both 2D and 3D modalities.

The success of Emergency Response (ER) scenarios such as search and rescue is often dependent upon the prompt location of a lost or injured person. With the increasing use of small Unmanned Aerial Systems (sUAS) as "eyes in the sky" during ER scenarios efficient detection of persons from aerial views plays a crucial role in achieving a successful mission outcome. Fatigue of human operators during prolonged ER missions coupled with limited human resources highlights the need for sUAS equipped with Computer Vision (CV) capabilities to aid in finding the person from aerial views. However the performance of CV models onboard sUAS substantially degrades under real-life rigorous conditions of a typical ER scenario where person search is hampered by occlusion and low target resolution. To address these challenges we extracted images from the NOMAD dataset and performed a crowdsource experiment to collect behavioural measurements when humans were asked to "find the person in the picture". We exemplify the use of our behavioral dataset Psych-ER by using its human accuracy data to adapt the loss function of a detection model. We tested our loss adaptation on a RetinaNet model evaluated on NOMAD against increasing distance and occlusion with our psychophysical loss adaptation showing improvements over the baseline at higher distances across different levels of occlusion without degrading performance at closer distances. To the best of our knowledge our work is the first human-guided approach to address the location task of a detection model while addressing real-world challenges of aerial search and rescue. All datasets and code can be found at: https://github.com/ArtRuss/NOMAD.

This paper investigates how to efficiently deploy vision transformers on edge devices for small workloads. Recent methods reduce the latency of transformer neural networks by removing or merging tokens with small accuracy degradation. However these methods are not designed with edge device deployment in mind: they do not leverage information about the latency-workload trends to improve efficiency. We address this shortcoming in our work. First we identify factors that affect ViT latency-workload relationships. Second we determine token pruning schedule by leveraging non-linear latency-workload relationships. Third we demonstrate a training-free token pruning method utilizing this schedule. We show other methods may increase latency by 2-30% while we reduce latency by 9-26%. For similar latency (within 5.2% or 7ms) across devices we achieve 78.6%-84.5% ImageNet1K classification accuracy while the state-of-the-art Token Merging achieves 45.8%-85.4%.

We address privacy issues in applications where images captured by an edge device (camera) are sent to the cloud for inference on utility tasks such as classification. Sending raw images to the cloud exposes them to data sniffing attacks and misuse by untrusted third-party service providers beyond the user's intended tasks. We propose an encoding scheme that not only evades direct visual inspection to the images or image reconstruction but also prevents sensitive information from being ascertained. Unlike commonly used adversarial learning approaches the proposed method is two-fold: first it uses a diffractive optical neural network to spatially separate features corresponding to different tasks on the sensor plane in the optical domain. Then only the pixels corresponding to the utility task region are read. This encoding ensures that private features are never digitally stored on the edge device thereby preventing privacy leakage. The proposed method successfully reduces the privacy retrieval in binary tasks with minimal accuracy loss ( 2%) of the utility task while reducing private task accuracy by 35% and defending against reconstruction attacks with SSIM score of 0.43.

Learning-based solutions for long-tailed recognition face difficulties in generalizing on balanced test datasets. Due to imbalanced data prior the learned a posteriori distribution is biased toward the most frequent (head) classes leading to an inferior performance on the least frequent (tail) classes. In general the performance can be improved by removing such a bias by eliminating the effect of imbalanced prior modeled using the number of class samples (frequencies). We first observe that the effective prior on the classes learned by the model at the end of the training can differ from the empirical prior obtained using class frequencies. Thus we propose a novel approach to accurately model the effective prior of a trained model using a posteriori probabilities. We propose to correct the imbalanced prior by adjusting the predicted a posteriori probabilities (Prior2Posterior: P2P) using the calculated prior in a post-hoc manner after the training and show that it can result in improved model performance. We present theoretical analysis showing the optimality of our approach for models trained with naive cross-entropy loss as well as logit adjusted loss. Our experiments show that the proposed approach achieves new state-of-the-art (SOTA) on several benchmark datasets from the long-tail literature in the category of logit adjustment methods. Further the proposed approach can be used to inspect any existing method to capture the effective prior and remove any residual bias to improve its performance post-hoc without model retraining. We also show that by using the proposed post-hoc approach the performance of many existing methods can be improved further.

Temporal information is crucial for detecting occluded instances. Existing temporal representations have progressed from BEV or PV (perspective view) features to more compact query features. Compared to these aforementioned features predictions offer the highest level of abstraction providing explicit information. In the context of online vectorized HD map construction this unique characteristic of predictions is potentially advantageous for long-term temporal modeling and the integration of map priors. This paper introduces PrevPredMap a pioneering temporal modeling framework that leverages previous predictions for constructing online vectorized HD maps. We have meticulously crafted two essential modules for PrevPredMap: the previous-predictions-based query generator and the dynamic-position-query decoder. Specifically the previous-predictions-based query generator is designed to separately encode different types of information from previous predictions which are then effectively utilized by the dynamic-position-query decoder to generate current predictions. Furthermore we have developed a dual-mode strategy to ensure PrevPredMap's robust performance across both single-frame and temporal modes. Extensive experiments demonstrate that PrevPredMap achieves state-of-the-art performance on the nuScenes and Argoverse2 datasets. Code is available at https://github.com/pnnnnnnn/PrevPredMap.

Lifelog images are very useful as memory cues for recalling past events. Estimating the level of event memory recall induced by a given lifelog image (event memorability) is useful for selecting images for cognitive interventions. Previous works for predicting event memorability follow a centralized model training paradigm that requires several users to share their lifelog images. This risks violating the privacy of individual lifeloggers. Alternatively a personal model trained with a lifelogger's own data guarantees privacy. However it imposes significant effort on the lifelogger to provide a large enough sample of self-rated images to develop a well-performing model for event memorability. Therefore we propose a clustered personalized federated learning setup FedMEM that avoids sharing raw images but still enables collaborative learning via model sharing. For an enhanced learning performance in the presence of data heterogeneity FedMEM evaluates similarity among users to group them into clusters. We demonstrate that our approach furnishes high-performing personalized models compared to the state-of-the-art.

Neural Radiance Fields (NeRFs) use neural networks to translate spatial coordinates to corresponding volume density and directional radiance enabling realistic novel view synthesis through volume rendering. Rendering new viewpoints involves computing volume rendering integrals along rays usually approximated by numerical quadrature because of lacking closed-form solutions. In this paper utilizing Taylor expansion we demonstrate that numerical quadrature causes inevitable approximation error in NeRF integrals due to ignoring the parameter associated with the Lagrange remainder. To mitigate the approximation error we propose a novel neural field with segment representation as input to implicitly model the remainder parameter. In theory our proposed method is proven to possess the potential to achieve fully precise rendering integral as demonstrated by comprehensive experiments on several commonly used datasets with state-of-the-art results.

Attackers can deliberately perturb classifiers' input with subtle noise altering final predictions. Among proposed countermeasures adversarial purification employs generative networks to preprocess input images filtering out adversarial noise. In this study we propose specific generators defined Multiple Latent Variable Generative Models (MLVGMs) for adversarial purification. These models possess multiple latent variables that naturally disentangle coarse from fine features. Taking advantage of these properties we autoencode images to maintain class-relevant information while discarding and re-sampling any detail including adversarial noise. The procedure is completely training-free exploring the generalization abilities of pre-trained MLVGMs on the adversarial purification downstream task. Despite the lack of large models trained on billions of samples we show that smaller MLVGMs are already competitive with traditional methods and can be used as foundation models. Official code released at https://github.com/SerezD/gen_adversarial.

As cameras become ubiquitous in our living environment invasion of privacy is becoming a significant concern. A common approach to privacy preservation is to remove personally identifiable information from a captured image but there is a risk of the original image being leaked. In this paper we propose a pre-capture privacy-aware imaging method that captures images from which the details of a pre-specified anonymized target have been eliminated. The proposed method applies a single-pixel imaging framework in which we introduce a feedback mechanism called an aperture pattern generator (APG). The introduced APG adaptively outputs the next aperture pattern to avoid sampling the anonymized target by using already acquired data as a clue. Furthermore the anonymized target can be set to any object without changing hardware. Except for the removed detailed features of the anonymized target the captured images are of comparable quality to those captured by a general camera and can be used for various computer vision applications. We target faces and license plates and experimentally show that the proposed method can capture clear images in which detailed features of the anonymized target are eliminated achieving both privacy and utility.

With the recent advancements in single-image-based 3D human pose and shape estimation (3DHPSE) there is a growing amount of works that can achieve good results on standard benchmarks but struggle to yield accurate human mesh in extreme scenarios like occlusion. Previous works propose to leverage 2D poses to help 3D HPSE model improve performance under occlusion but usually rely on manual design to integrate 2D poses and only aim for specific kinds of occlusion. In this paper we present PostoMETRO (POSe TOken enhanced MEsh TRansfOrmer) which integrates 2D pose prior knowledge as tokens into transformers to improve model's performance under occlusion. Using a VQ-VAE-based pose tokenizer we efficiently represent 2D poses as tokens and feed them to transformers together with image tokens. Subsequently these tokens are queried by vertex and joint tokens to decode 3D coordinates of mesh vertices and human joints. Our proposed 2D poses integration strategy is manual-design-free and suitable for various kinds of occlusion. Experiments on both standard and occlusion-specific benchmarks demonstrate the effectiveness of PostoMETRO.

Vision-language models (VLMs) like CLIP have been adapted for Multi-Label Recognition (MLR) with partial annotations by leveraging prompt-learning where positive and negative prompts are learned for each class to associate their embeddings with class presence or absence in the shared vision-text feature space. While this approach improves MLR performance by relying on VLM priors we hypothesize that learning negative prompts may be suboptimal as the datasets used to train VLMs lack image-caption pairs explicitly focusing on class absence. To analyze the impact of positive and negative prompt learning on MLR we introduce PositiveCoOp and NegativeCoOp where only one prompt is learned with VLM guidance while the other is replaced by an embedding vector learned directly in the shared feature space without relying on the text encoder. Through empirical analysis we observe that negative prompts degrade MLR performance and learning only positive prompts combined with learned negative embeddings (PositiveCoOp) outperforms dual prompt learning approaches. Moreover we quantify the performance benefits that prompt-learning offers over a simple vision-features-only baseline observing that the baseline displays strong performance comparable to dual prompt learning approach (DualCoOp) when the proportion of missing labels is low while requiring half the training compute and 16 times fewer parameters.

Morphing attacks can successfully deceive face recognition systems resulting in unreliable access control especially in the border control scenario. Consequently the development of Morphing Attack Detection (MAD) algorithms is crucial for detecting morphing attacks based on either a single facial image (S-MAD) or two facial images (Differential-MAD or D-MAD). In this work we proposed a novel D-MAD approach PoolAtnRes to reliably detect morphing attacks. The proposed PoolAtnRes architecture is constructed using three main functional blocks namely convolution pooling Hybrid Attention and Residual blocks which are serially connected to detect morphing attacks. Extensive experiments were performed on the newly constructed morphing dataset using nine morphing-generation techniques. The detection performance of the proposed PoolAtnRes model was compared with three state-of-the-art (SOTA) D-MAD techniques with different performance evaluation protocols to benchmark its generalizability to unseen morphing generation. The results obtained indicated the best performance of the proposed PoolAtnRes D-MAD.

Defocus is a crucial cue for image-based microscale depth estimation yet its measurement depends on spatial appearance changes such as texture. We show that passively observed polarization is responsive to small irregularities of the surface visible in the microscopic world and can be leveraged for focus measure as a strong texture. Our key idea is to leverage texture from polarization for blur analysis and accurately estimate the focus level of microscopic polarization images. We further utilize normal cues from polarization to create a prior distribution of the focus level between neighboring pixels. We then interpolatively propagate the focus level of discrete image slices at different focus depths while denoising. We implement our method with a single polarization camera with a microscope and recover the per-pixel depth from the multi-focus images. The reconstructed results demonstrate the effectiveness of our method for various microscale objects regardless of the surface texture.

Recent advancements in self-supervised learning in the point cloud domain have demonstrated significant potential. However these methods often suffer from drawbacks such as lengthy pre-training time the necessity of reconstruction in the input space and the necessity of additional modalities. In order to address these issues we introduce Point-JEPA a joint embedding predictive architecture designed specifically for point cloud data. To this end we introduce a sequencer that orders point cloud patch embeddings to efficiently compute and utilize their proximity based on their indices during target and context selection. The sequencer also allows shared computations of the patch embeddings' proximity between context and target selection further improving the efficiency. Experimentally our method demonstrates state-of-the-art performance while avoiding the reconstruction in the input space or additional modality. In particular Point-JEPA attains a classification accuracy of 93.7% for linear SVM on ModelNet40 surpassing all other self-supervised models. Moreover Point-JEPA also establishes new state-of-the-art performance levels across all four few-shot learning evaluation frameworks. The code is available at https://github.com/Ayumu-JS/Point-JEPA

This paper introduces Point-GN a novel non-parametric network for efficient and accurate 3D point cloud classification. Unlike conventional deep learning models that rely on a large number of trainable parameters Point-GN leverages non-learnable components-specifically Farthest Point Sampling (FPS) k-Nearest Neighbors (k-NN) and Gaussian Positional Encoding (GPE)-to extract both local and global geometric features. This design eliminates the need for additional training while maintaining high performance making Point-GN particularly suited for real-time resource-constrained applications. We evaluate Point-GN on two benchmark datasets ModelNet40 and ScanObjectNN achieving classification accuracies of 85.29% and 85.89% respectively while significantly reducing computational complexity. Point-GN outperforms existing non-parametric methods and matches the performance of fully trained models all with zero learnable parameters. Our results demonstrate that Point-GN is a promising solution for 3D point cloud classification in practical real-time environments.

Point cloud color upsampling is an important and less explored research topic. State-of-the-art methods colorize points based on the colors of neighboring points and geometric distances. However these methods often suffer from blurring and noise at color boundaries since object textures can have large color variations even between geometrically neighboring positions. In this paper we propose a point cloud color upsampling method with attention weights for neighboring points. The proposed method first performs coarse colorization with the colors of low-resolution points neighboring the high-resolution points and predicted weights. Then it refines the colors by predicting offsets for high-resolution points with aggregate features obtained from the low-resolution points. Both quantitative and qualitative experimental results on datasets acquired in real-world environments demonstrate that the proposed method achieves significantly superior color upsampling performance compared to state-of-the-art methods.

Modeling a human avatar that can plausibly deform to articulations is an active area of research. We present PocoLoco - the first template-free point-based pose-conditioned generative model for 3D humans in loose clothing. We motivate our work by noting that most methods require a parametric model of the human body to ground pose-dependent deformations. Consequently they are restricted to modeling clothing that is topologically similar to the naked body and do not extend well to loose clothing. The few methods that attempt to model loose clothing typically require either canonicalization or a UV-parameterization and need to address the challenging problem of explicitly estimating correspondences for the deforming clothes. In this work we formulate avatar clothing deformation as a conditional point-cloud generation task within the denoising diffusion framework. Crucially our framework operates directly on unordered point clouds eliminating the need for a parametric model or a clothing template. This also enables a variety of practical applications such as point-cloud completion and pose-based editing - important features for virtual human animation. As current datasets for human avatars in loose clothing are far too small for training diffusion models we release a dataset of two subjects performing various poses in loose clothing with a total of 75K point clouds. By contributing towards tackling the challenging task of effectively modeling loose clothing and expanding the available data for training these models we aim to set the stage for further innovation in digital humans. The source code is available at https: //github.com/sidsunny/pocoloco.