WACV 2025 Papers

Over the past year 3D Gaussian Splatting (3DGS) has received significant attention for its ability to represent 3D scenes in a perceptually accurate manner. However it can require a substantial amount of storage since each splat's individual data must be stored. While compression techniques offer a potential solution by reducing the memory footprint they still necessitate retrieving the entire scene before any part of it can be rendered. In this work we introduce a novel approach for progressively rendering such scenes aiming to display visible content that closely approximates the final scene as early as possible without loading the entire scene into memory. This approach benefits both on-device rendering applications limited by memory constraints and streaming applications where minimal bandwidth usage is preferred. To achieve this we approximate the contribution of each Gaussian to the final scene and construct an order of prioritization on their inclusion in the rendering process. Additionally we demonstrate that our approach can be combined with existing compression methods to progressively render (and stream) 3DGS scenes optimizing bandwidth usage by focusing on the most important splats within a scene. Overall our work establishes a foundation for making remotely hosted 3DGS content more quickly accessible to end-users in over-the-top consumption scenarios with our results showing significant improvements in quality across all metrics compared to existing methods.

Recently the usage of Contrastive Representation Learning (CRL) as a pre-training technique improves the performance of learning with noisy labels (LNL) methods. However instead of pre-training when trivially combining CRL loss with LNL methods as an end-to-end framework the empirical experiments show severe degeneration of the performance. We verify through experiments that this issue is caused by optimization conflicts of losses and propose an end-to-end PLReMix framework by introducing a Pseudo-Label Relaxed (PLR) contrastive loss. This PLR loss constructs a reliable negative set of each sample by filtering out its inappropriate negative pairs alleviating the loss conflicts by trivially combining these losses. The proposed PLR loss is pluggable and we have integrated it into other LNL methods observing their improved performance. Furthermore a two-dimensional Gaussian Mixture Model is adopted to distinguish clean and noisy samples by leveraging semantic information and model outputs simultaneously. Experiments on multiple benchmark datasets demonstrate the effectiveness of the proposed method. Codes will be available.

Brain tumor detection in multiplane Magnetic Resonance Imaging (MRI) slices is a challenging task due to the various appearances and relationships in the structure of the multiplane images. In this paper we propose a new You Only Look Once (YOLO)-based detection model that incorporates Pretrained Knowledge (PK) called PK-YOLO to improve the performance for brain tumor detection in multiplane MRI slices. To our best knowledge PK-YOLO is the first pretrained knowledge guided YOLO-based object detector. The main components of the new method are a pretrained pure lightweight convolutional neural network-based backbone via sparse masked modeling a YOLO architecture with the pretrained backbone and a regression loss function for improving small object detection. The pretrained backbone allows for feature transferability of object queries on individual plane MRI slices into the model encoders and the learned domain knowledge base can improve in-domain detection. The improved loss function can further boost detection performance on small-size brain tumors in multiplanar two-dimensional MRI slices. Experimental results show that the proposed PK-YOLO achieves competitive performance on the multiplanar MRI brain tumor detection datasets compared to state-of-the-art YOLO-like and DETR-like object detectors. The code is available at https://github.com/mkang315/PK-YOLO.

This work introduces PICASSO a framework for the parameterization of 2D CAD sketches from hand-drawn and precise sketch images. PICASSO converts a given CAD sketch image into parametric primitives that can be seamlessly integrated into CAD software. Our framework leverages rendering self-supervision to enable the pre-training of a CAD sketch parameterization network using sketch renderings only thereby eliminating the need for corresponding CAD parameterization. Thus we significantly reduce reliance on parameter-level annotations which are often unavailable particularly for hand-drawn sketches. The two primary components of PICASSO are (1) a Sketch Parameterization Network (SPN) that predicts a series of parametric primitives from CAD sketch images and (2) a Sketch Rendering Network (SRN) that renders parametric CAD sketches in a differentiable manner and facilitates the computation of a rendering (image-level) loss for self-supervision. We demonstrate that the proposed PICASSO can achieve reasonable performance even when finetuned with only a small number of parametric CAD sketches. Extensive evaluation on the widely used SketchGraphs and CAD as Language datasets validates the effectiveness of the proposed approach on zero- and few-shot learning scenarios.

As of 2023 a record 117 million people have been displaced worldwide more than double the number from a decade ago [22]. Of these 32 million are refugees under the UNHCR mandate with 8.7 million residing in refugee camps. A critical issue faced by these populations is the lack of access to electricity with 80% of the 8.7 million refugees and displaced persons in camps globally relying on traditional biomass for cooking and lacking reliable power for essential tasks such as cooking and charging phones. Often the burden of collecting firewood falls on women and children who frequently travel up to 20 kilometers into dangerous areas increasing their vulnerability. [7] Electricity access could significantly alleviate these challenges but a major obstacle is the lack of accurate power grid infrastructure maps particularly in resource-constrained environments like refugee camps needed for energy access planning. Existing power grid maps are often outdated incomplete or dependent on costly complex technologies limiting their practicality. To address this issue PGRID is a novel application-based approach which utilizes high-resolution aerial imagery to detect electrical poles and segment electrical lines creating precise power grid maps. PGRID was tested in the Turkana region of Kenya specifically the Kakuma and Kalobeyei Camps covering 84 km2 and housing over 200000 residents. Our findings show that PGRID delivers high-fidelity power grid maps especially in unplanned settlements with F1-scores of 0.71 and 0.82 for pole detection and line segmentation respectively. This study highlights a practical application for leveraging open data and limited labels to improve power grid mapping in unplanned settlements where the growing number of displaced persons urgently need sustainable energy infrastructure solutions.

Pre-training on large-scale datasets and utilizing margin-based loss functions have been highly successful in training models for high-resolution face recognition. However these models struggle with low-resolution face datasets in which the faces lack the facial attributes necessary for distinguishing different faces. Full fine-tuning on low-resolution datasets a naive method for adapting the model yields inferior performance due to catastrophic forgetting of pre-trained knowledge. Additionally the domain difference between high-resolution (HR) gallery images and low-resolution (LR) probe images in low resolution datasets leads to poor convergence for a single model to adapt to both gallery and probe after fine-tuning. To this end we propose PETALface a Parameter-Efficient Transfer Learning approach for low-resolution face recognition. Through PETALface we attempt to solve both the aforementioned problems. (1) We solve catastrophic forgetting by leveraging the power of parameter efficient fine-tuning(PEFT). (2) We introduce two low-rank adaptation modules to the backbone with weights adjusted based on the input image quality to account for the difference in quality for the gallery and probe images. To the best of our knowledge PETALface is the first work leveraging the powers of PEFT for low resolution face recognition. Extensive experiments demonstrate that the proposed method outperforms full fine-tuning on low-resolution datasets while preserving performance on high-resolution and mixed-quality datasets all while using only 0.48% of the parameters.

Generalized Zero Shot Learning (GZSL) aims at achieving a good accuracy on both seen and unseen classes by relying on the information acquired from auxiliary attributes. Existing approaches have devised many frameworks to make this knowledge transfer more efficient and informative. Despite their effectiveness on boosting the overall performance there has always been a strong bias in the model towards the seen classes which makes GZSL problem more challenging. The effect of this bias on the model performance has never been properly explored. We observe that GZSL algorithms in literature have an evident bias towards the seen classes. Further we also show that techniques like calibrated stacking fall short of resolving this conflict between the seen and unseen classes effectively. In this work we analyze and develop a logit-adjustment approach in GZSL setting and propose a simple yet effective method to remove the bias from trained models in a post-hoc manner. Moreover as a consequence of the post-hoc nature of the proposed approach there is no additional training cost. We exhaustively compare the proposed method on both embedding-based and generative-based GZSL frameworks surpassing the SOTA results by 3.1% 4.6% and 3.1% on CUB SUN and AwA2 datasets. We also present theoretical analysis showing effectiveness of proposed approach.

In pursuit of more inclusive Vision-Language Models (VLMs) this study introduces a Large Multilingual Multimodal Model called PALO. PALO offers visual reasoning capabilities in 10 major languages including English Chinese Hindi Spanish French Arabic Bengali Russian Urdu and Japanese that span a total of 5B people (65% of the world population). Our approach involves a semi-automated translation approach to adapt the multimodal instruction dataset from English to the target languages using a fine-tuned Large Language Model thereby ensuring high linguistic fidelity while allowing scalability due to minimal manual effort. The incorporation of diverse instruction sets helps us boost overall performance across multiple languages especially those that are underrepresented like Hindi Arabic Bengali and Urdu. The resulting models are trained across three scales (1.7B 7B and 13B parameters) to show the generalization and scalability where we observe substantial improvements compared to strong baselines. We also propose the first multilingual multimodal benchmark for the forthcoming approaches to evaluate their vision-language reasoning capabilities across languages. Code: https://github.com/mbzuai-oryx/PALO.

Scene rearrangement like table tidying is a challenging task in robotic manipulation due to the complexity of predicting diverse object arrangements. Web-scale trained generative models such as Stable Diffusion can aid by generating natural scenes as goals. To facilitate robot execution object-level representations must be extracted to match the real scenes with the generated goals and to calculate object pose transformations. Current methods typically use a multi-step design that involves separate models for generation segmentation and feature encoding which can lead to a low success rate due to error accumulation. Furthermore they lack control over the viewing perspectives of the generated goals restricting the tasks to 3-DoF settings. In this paper we propose PACA a zero-shot pipeline for scene rearrangement that leverages perspective-aware cross-attention representation derived from Stable Diffusion. Specifically we develop an object-level representation that integrates generation segmentation and feature encoding into a single step. Additionally we introduce perspective control thus enabling the matching of 6-DoF camera views and extending past approaches that were limited to 3-DoF top-down settings. The efficacy of our method is demonstrated through its zero-shot performance in real robot experiments across various scenes achieving an average matching accuracy and execution success rate of 87% and 67% respectively.

This work presents a new public dataset for cell detection in bright-field microscopy images annotated with Oriented Bounding Boxes (OBBs) named Oriented Cell Dataset (OCD). Our dataset also contains a subset of images with five independent expert annotations which allows inter-annotation analysis to determine a suitable IoU acceptance threshold for evaluating cell detectors. We show that OBBs and a derived representation Oriented Ellipses (OEs) provide a more accurate shape representation than standard Horizontal Bounding Boxes (HBBs) with a slight overhead of one extra click in the annotation process. We benchmarked OCD using 14 state-of-the-art oriented object detectors and explored two main problems in cancer biology: cell confluence and polarity determination. Our code and dataset are available at https://github.com/LucasKirsten/Deep-Cell-Tracking-EBB.

Patient-level diagnosis of severity in ulcerative colitis (UC) is common in clinical practice where the most severe score for a patient is typically recorded as the diagnosis result. However previous UC classification methods (i.e. image-level estimation) mainly assumed the input was a single image. Thus these methods can not utilize severity labels recorded in real clinical settings. In this paper we propose a patient-level severity estimation method by a transformer with selective aggregator tokens where a severity label is estimated from multiple images taken from a patient similar to a clinical setting. Our method can effectively aggregate features of severe parts from a set of images captured in each patient and it facilitates improving the discriminative ability between adjacent severity classes. Experiments demonstrate the effectiveness of the proposed method on two datasets compared with the state-of-the-art MIL methods. Moreover we evaluated our method using real clinical data and confirmed that our method outperformed the previous image-level methods. The code is publicly available at https://github.com/Shiku-Kaito/Ordinal-Multiple-instance-Learning-for-Ulcerative-Colitis-Severity-Estimation.

Identifying a pedestrian's intention to cross the road is crucial for autonomous driving as it alerts the system to stop or slow down. However determining crossing intention from video is challenging due to the need for extracting complex high-level semantics. This paper introduces ClipCross a novel classification framework optimized to extract high-level semantic features using the vision-language model CLIP for determining crossing intention. Existing CLIP-based methods perform poorly in this task as CLIP's image and text encoders fail to capture the nuanced semantic distinctions between crossing and non-crossing intention images. ClipCross addresses this by optimizing a set of CLIP text embeddings to extract high-level semantic features which a multi-layer perceptron uses to distinguish between crossing and non-crossing intentions. ClipCross achieves state-of-the-art performance on crossing intention estimation benchmark datasets: PIE PSI and JAAD.

Activation functions play a crucial role in artificial neural networks by introducing non-linearities that enable networks to learn complex patterns in data. An appropriate choice of an activation function plays a crucial role in the training dynamics of a neural network which can boost network performance significantly. Rectified Linear Unit (ReLU) and its variants like leaky ReLU and parametric ReLU have emerged as the most popular activations due to their ability to enable faster training and generalization in deep neural networks despite having some significant issues like vanishing gradient problems. In this paper we have proposed smooth functions which we call the AMSU family which are smooth approximations of the maximum function. We derive three activations from the AMSU family namely AMSU-1 AMSU-2 & AMSU-3 and show their effectiveness in different deep learning problems. By simply replacing the ReLU function Top-1 accuracy improves by 5.88% 5.96% and 5.32% on the CIFAR100 dataset on the ShuffleNet V2 model. Also replacing ReLU with AMSU-1 AMSU-2 and AMSU-3 Top-1 accuracy improves by 8.50% 8.29% and 7.70% on the CIFAR100 dataset on the ShuffleNet V2 model with FGSM attack. Also Replacing ReLU with AMSU-1 AMSU-2 and AMSU-3 on ImageNet-1K data we got 3%-5% improvement on ShuffleNet and MobileNet models. The source code is publicly available at https://github.com/koushik313/AMSU.

Multi-Task Learning (MTL) involves the concurrent training of multiple tasks offering notable advantages for dense prediction tasks in computer vision. MTL not only reduces training and inference time as opposed to having multiple single-task models but also enhances task accuracy through the interaction of multiple tasks. However existing methods face limitations. They often rely on suboptimal cross-task interactions resulting in task-specific predictions with poor geometric and predictive coherence. In addition many approaches use inadequate loss weighting strategies which do not address the inherent variability in task evolution during training. To overcome these challenges we propose an advanced MTL model specifically designed for dense vision tasks. Our model leverages state-of-the-art vision transformers with task-specific decoders. To enhance cross-task coherence we introduce a trace-back method that improves both cross-task geometric and predictive features. Furthermore we present a novel dynamic task balancing approach that projects task losses onto a common scale and prioritizes more challenging tasks during training. Extensive experiments demonstrate the superiority of our method establishing new state-of-the-art performance across two benchmark datasets. The code is available at: https://github.com/Klodivio355/MT-CP.

The rise of 2D vision-language models (VLMs) has enabled new possibilities for language-driven 3D scene understanding tasks. Existing works focus on indoor scenes or autonomous driving scenarios and typically validate against a pre-defined set of semantic object classes. In this work we analyze the capabilities of vision-language models for large-scale urban 3D scene understanding and propose new applications of VLMs that directly operate on aerial 3D reconstructions of cities. In particular we address higher-level 3D scene understanding tasks such as population density building age property prices crime rate and noise pollution. Our analysis reveals surprising zero-shot and few-shot performance of VLMs in urban environments.

Pose estimation has promised to impact healthcare by enabling more practical methods to quantify nuances of human movement and biomechanics. However despite the inherent connection between pose estimation and biomechanics these disciplines have largely remained disparate. For example most current pose estimation benchmarks use metrics such as Mean Per Joint Position Error Percentage of Correct Keypoints or mean Average Precision to assess performance without quantifying kinematic and physiological correctness - key aspects for biomechanics. To alleviate this challenge we develop OpenCapBench to offer an easy-to-use unified benchmark to assess common tasks in human pose estimation evaluated under physiological constraints. OpenCapBench computes consistent kinematic metrics through joints angles provided by an open-source musculoskeletal modeling software (OpenSim). Through OpenCapBench we demonstrate that current pose estimation models use keypoints that are too sparse for accurate biomechanics analysis. To mitigate this challenge we introduce SynthPose a new approach that enables finetuning of pre-trained 2D human pose models to predict an arbitrarily denser set of keypoints for accurate kinematic analysis through the use of synthetic data. Incorporating such finetuning on synthetic data of prior models leads to twofold reduced joint angle errors. Moreover OpenCapBench allows users to benchmark their own developed models on our clinically relevant cohort. Overall OpenCapBench bridges the computer vision and biomechanics communities aiming to drive simultaneous advances in both areas.

Catastrophic forgetting is a significant challenge in online continual learning (OCL) especially for non-stationary data streams that do not have well-defined task boundaries. This challenge is exacerbated by the memory constraints and privacy concerns inherent in rehearsal buffers. To tackle catastrophic forgetting in this paper we introduce Online-LoRA a novel framework for task-free OCL. Online-LoRA allows to finetune pre-trained Vision Transformer (ViT) models in real-time to address the limitations of rehearsal buffers and leverage pre-trained models' performance benefits. As the main contribution our approach features a novel online weight regularization strategy to identify and consolidate important model parameters. Moreover Online-LoRA leverages the training dynamics of loss values to enable the automatic recognition of the data distribution shifts. Extensive experiments across many task-free OCL scenarios and benchmark datasets (including CIFAR-100 ImageNet-R ImageNet-S CUB-200 and CORe50) demonstrate that Online-LoRA can be robustly adapted to various ViT architectures while achieving better performance compared to SOTA methods.

Vision-Language Models (VLMs) have shown significant promise in Visual Question Answering (VQA) tasks by leveraging web-scale multimodal datasets. However these models often struggle with continual learning due to catastrophic forgetting when adapting to new tasks. As an effective remedy to mitigate catastrophic forgetting rehearsal strategy uses the data of past tasks upon learning new task. However such strategy incurs the need of storing past data which might not be feasible due to hardware constraints or privacy concerns. In this work we propose the first data-free method that leverages the language generation capability of a VLM instead of relying on external models to produce pseudo-rehearsal data for addressing continual VQA. Our proposal named as GaB generates pseudo-rehearsal data by posing previous task questions on new task data. Yet despite being effective the distribution of generated questions skews towards the most frequently posed questions due to the limited and task-specific training data. To mitigate this issue we introduce a pseudo-rehearsal balancing module that aligns the generated data towards the ground-truth data distribution using either the question meta-statistics or an unsupervised clustering method. We evaluate our proposed method on two recent benchmarks i.e. VQACL-VQAv2 and CLOVE-function benchmarks. GaB outperforms all the data-free baselines with substantial improvement in maintaining VQA performance across evolving tasks while being on-par with methods with access to the past data. Code will be made public soon.

Point clouds are essential for object modeling and play a critical role in assisting driving tasks for autonomous vehicles (AVs). However the significant volume of data generated by AVs creates challenges for storage bandwidth and processing cost. To tackle these challenges we propose a representative point selection framework for point cloud downsampling which preserves critical object-related information while effectively filtering out irrelevant background points. Our method involves two steps: (1) Object Presence Detection where we introduce an unsupervised density peak-based classifier and a supervised Naive Bayes classifier to handle diverse scenarios and (2) Sampling Budget Allocation where we propose a strategy that selects object-relevant points while maintaining a high retention rate of object information. Extensive experiments on the KITTI and nuScenes datasets demonstrate that our method consistently outperforms state-of-the-art baselines in both efficiency and effectiveness across varying sampling rates. As a model-agnostic solution our approach integrates seamlessly with diverse downstream models making it a valuable and scalable addition to the 3D point cloud downsampling toolkit for AV applications.