WACV 2025 Papers

Scaling laws dictate that the performance of AI models is proportional to the amount of available data. Data augmentation is a promising solution to expanding the dataset size. Traditional approaches focused on augmentation using rotation translation and resizing. Recent approaches use generative AI models to improve dataset diversity. However the generative methods struggle with issues such as subject corruption and the introduction of irrelevant artifacts. In this paper we propose the Automated Generative Data Augmentation (AGA). The framework combines the utility of large language models (LLMs) diffusion models and segmentation models to augment data. AGA preserves foreground authenticity while ensuring background diversity. Specific contributions include: i) segment and superclass based object extraction ii) prompt diversity with combinatorial complexity using prompt decomposition and iii) affine subject manipulation. We evaluate AGA against state-of-the-art (SOTA) techniques on three representative datasets ImageNet CUB and iWildCam. The experimental evaluation demonstrates an accuracy improvement of 15.6% and 23.5% for in and out-of-distribution data compared to baseline models respectively. There is also 64.3% improvement in SIC score compared to the baselines.

Motorized two-wheelers are a prevalent and economical means of transportation particularly in the Asia-Pacific region. However hazardous driving practices such as triple riding and non-compliance with helmet regulations contribute significantly to accident rates. Addressing these violations through automated enforcement mechanisms can enhance traffic safety. In this paper we propose DashCop an end-to-end system for automated E-ticket generation. The system processes vehicle-mounted dashcam videos to detect two-wheeler traffic violations. Our contributions include: (1) a novel Segmentation and Cross-Association (SAC) module to accurately associate riders with their motorcycles (2) a robust cross-association-based tracking algorithm optimized for the simultaneous presence of riders and motorcycles and (3) the RideSafe-400 dataset a comprehensive annotated dashcam video dataset for triple riding and helmet rule violations. Our system demonstrates significant improvements in violation detection validated through extensive evaluations on the RideSafe-400 dataset. Project page: https://dash-cop.github.io/

Wide angle fisheye images are becoming increasingly common for perception tasks in applications such as robotics security and mobility (e.g. drones avionics). However current models often either ignore the distortions in wide angle images or are not suitable to perform pixel-level tasks. In this paper we present an encoder-decoder model based on a radial transformer architecture that adapts to distortions in wide angle lenses by leveraging the physical characteristics defined by the radial distortion profile. In contrast to the original model which only performs classification tasks we introduce a U-Net architecture DarSwin-Unet designed for pixel level tasks. Furthermore we propose a novel strategy that minimizes sparsity when sampling the image for creating its input tokens. Our approach enhances the model capability to handle pixel-level tasks in wide angle fisheye images making it more effective for real-world applications. Compared to other baselines DarSwin-Unet achieves the best results across different datasets with significant gains when trained on bounded levels of distortions (very low low medium and high) and tested on all including out-of-distribution distortions. We demonstrate its performance on depth estimation and show through extensive experiments that DarSwin-Unet can perform zero-shot adaptation to unseen distortions of different wide angle lenses. The code and models are publicly available at https://lvsn.github.io/darswin-unet/.

Generating dance from music is crucial for advancing automated choreography. Current methods typically produce skeleton keypoint sequences instead of dance videos and lack the capability to make specific individuals dance which reduces their real-world applicability. These methods also require precise keypoint annotations complicating data collection and limiting the use of self-collected video datasets. To overcome these challenges we introduce a novel task: generating dance videos directly from images of individuals guided by music. This task enables the dance generation of specific individuals without requiring keypoint annotations making it more versatile and applicable to various situations. Our solution the Dance Any Beat Diffusion model (DabFusion) utilizes a reference image and a music piece to generate dance videos featuring various dance types and choreographies. The music is analyzed by our specially designed music encoder which identifies essential features including dance style movement and rhythm. DabFusion excels in generating dance videos not only for individuals in the training dataset but also for any previously unseen person. This versatility stems from its approach of generating latent optical flow which contains all necessary motion information to animate any person in the image. We evaluate DabFusion's performance using the AIST++ dataset focusing on video quality audio-video synchronization and motion-music alignment. We propose a 2D Motion-Music Alignment Score (2D-MM Align) which builds on the Beat Alignment Score to more effectively evaluate motion-music alignment for this new task. Experiments show that our DabFusion establishes a solid baseline for this innovative task. Video results can be found on our project page: https://DabFusion.github.io.

Spiking Neural Networks (SNNs) present a more energy-efficient alternative to Artificial Neural Networks (ANNs) by harnessing spatio-temporal dynamics and event-driven spikes. Effective utilization of temporal information is crucial for SNNs leading to the exploration of attention mechanisms to enhance this capability. Conventional attention operations either apply identical operation or employ non-identical operations across target dimensions. We identify that these approaches provide distinct perspectives on temporal information. To leverage the strengths of both operations we propose a novel Dual Temporal-channel-wise Attention (DTA) mechanism that integrates both identical/non-identical attention strategies. To the best of our knowledge this is the first attempt to concentrate on both the correlation and dependency of temporal-channel using both identical and non-identical attention operations. Experimental results demonstrate that the DTA mechanism achieves state-of-the-art performance on both static datasets (CIFAR10 CIFAR100 ImageNet-1k) and dynamic dataset (CIFAR10-DVS) elevating spike representation and capturing complex temporal-channel relationship. We open-source our code: https://github.com/MnJnKIM/DTA-SNN.

Line segment detection is a fundamental low-level task in computer vision and improvements in this task can impact more advanced methods that depend on it. Most new methods developed for line segment detection are based on Convolutional Neural Networks (CNNs). Our paper seeks to address challenges that prevent the wider adoption of transformer-based methods for line segment detection. More specifically we introduce a new model called Deformable Transformer-based Line Segment Detection (DT-LSD) that supports cross-scale interactions and can be trained quickly. This work proposes a novel Deformable Transformer-based Line Segment Detector (DT-LSD) that addresses LETR's drawbacks. For faster training we introduce Line Contrastive DeNoising (LCDN) a technique that stabilizes the one-to-one matching process and speeds up training by 34X. We show that DT-LSD is faster and more accurate than its predecessor transformer-based model (LETR) and outperforms all CNN-based models in terms of accuracy. In the Wireframe dataset DT-LSD achieves 71.7 for sAP^10 and 73.9 for sAP^15; while 33.2 for sAP^10 and 35.1 for sAP^15 in the YorkUrban dataset. Code available at https://github.com/SebastianJanampa/DT-LSD.

Increasing attention has been garnered by LiDAR points and multi-view images fusion based on Transformer to supplement another modality in 3D object detection. However most current methods perform data fusion based on the entire scene which entails substantial redundant background information and lacks fine-grained local details of the foreground objects to be detected. Furthermore global scene fusion results in coarse fusion granularity and the excessive redundancy leads to slow convergence and reduced accuracy. In this work a novel Dual Scenes Transformer pipeline (DSTR) which comprises a Global-Scene Integration (GSI) module Local-Scene Integration (LSI) module and Dual Scenes Fusion (DSF) module is presented to tackle the above challenge. Concretely features from point clouds and images are utilized for gathering the global scene information in GSI. The insufficiency issues of global scene fusion are addressed by extracting local instance features for both modalities in LSI supplementing GSI in a more fine-grained way. Furthermore DSF is proposed to aggregate the local scene to the global scene which fully explores dual-modal information. Experiments on the nuScenes dataset show that our DSTR has state-of-the-art (SOTA) performance in certain 3D object detection benchmark categories on validation and test sets.

Vision-language models (VLMs) e.g. CLIP have shown remarkable potential in zero-shot image classification. However adapting these models to new domains remains challenging especially in unsupervised settings where labeled data is unavailable. Recent research has proposed pseudo-labeling approaches to adapt CLIP in an unsupervised manner using unlabeled target data. Nonetheless these methods struggle due to noisy pseudo-labels resulting from the misalignment between CLIP's visual and textual representations. This study introduces DPA an unsupervised domain adaptation method for VLMs. DPA introduces the concept of dual prototypes acting as distinct classifiers along with the convex combination of their outputs thereby leading to accurate pseudo-label construction. Next it ranks pseudo-labels to facilitate robust self-training particularly during early training. Finally it addresses visual-textual misalignment by aligning textual prototypes with image prototypes to further improve the adaptation performance. Experiments on 13 downstream vision tasks demonstrate that DPA significantly outperforms zero-shot CLIP and the state-of-the-art unsupervised adaptation baselines.

High-fidelity 3D reconstruction of common indoor scenes is crucial for VR and AR applications. 3D Gaussian splatting a novel differentiable rendering technique has achieved state-of-the-art novel view synthesis results with high rendering speeds and relatively low training times. However its performance on scenes commonly seen in indoor datasets is poor due to the lack of geometric constraints during optimization. In this work we explore the use of readily accessible geometric cues to enhance Gaussian splatting optimization in challenging ill-posed and textureless scenes. We extend 3D Gaussian splatting with depth and normal cues to tackle challenging indoor datasets and showcase techniques for efficient mesh extraction. Specifically we regularize the optimization procedure with depth information enforce local smoothness of nearby Gaussians and use off-the-shelf monocular networks to achieve better alignment with the true scene geometry. We propose an adaptive depth loss based on the gradient of color images improving depth estimation and novel view synthesis results over various baselines. Our simple yet effective regularization technique enables direct mesh extraction from the Gaussian representation yielding more physically accurate reconstructions of indoor scenes. Our code will be released in https://github.com/maturk/dn-splatter.

Automated airway segmentation in CT images is crucial for lung diseases' diagnosis. However manual annotation scarcity hinders supervised learning efficacy while unlimited intensities and sample imbalance lead to discontinuity and false-negative issues. To address these challenges we propose a novel airway segmentation model named Dynamical Multi-order Response Network (DMRN) integrating the unsupervised and supervised learning in parallel to alleviate the label scarcity of airway. In the unsupervised branch (1) we propose several novel strategies of Dynamic Mask-Ratio (DMR) to enable the model to perceive context information of varying sizes mimicking the laws of human learning vividly; (2) we present a novel target of Multi-Order Normalized Responses (MONR) exploiting the distinct order exponential operation of raw images and oriented gradients to enhance the textural representations of bronchioles. For the supervised branch we directly predict the final full segmentation map by the large-ratio cube-masked input instead of full input. Ultimately we verify the method performance and robustness by training on normal lung disease datasets while testing on lung cancer COVID-19 and Lung fibrosis datasets. All experimental results have proved that our method exceeds state-of-the-art methods significantly. Code will be released in the future.

Current multi-modal object re-identification approaches based on large-scale pre-trained backbones (i.e. ViT) have displayed remarkable progress and achieved excellent performance. However these methods usually adopt the standard full fine-tuning paradigm which requires the optimization of considerable backbone parameters causing extensive computational and storage requirements. In this work we propose an efficient prompt-tuning framework tailored for multi-modal object re-identification dubbed DMPT which freezes the main backbone and only optimizes several newly added decoupled modality-aware parameters. Specifically we explicitly decouple the visual prompts into modality-specific prompts which leverage prior modality knowledge from a powerful text encoder and modality-independent semantic prompts which extract semantic information from multi-modal inputs such as visible near-infrared and thermal-infrared. Built upon the extracted features we further design a Prompt Inverse Bind (PromptIBind) strategy that employs bind prompts as a medium to connect the semantic prompt tokens of different modalities and facilitates the exchange of complementary multi-modal information boosting final re-identification results. Experimental results on multiple common benchmarks demonstrate that our DMPT can achieve competitive results to existing state-of-the-art methods while requiring only 6.5% fine-tuning of the backbone parameters.

With the recent exhibited strength of generative diffusion models an open research question is if images generated by these models can be used to learn better visual representations. While this generative data expansion may suffice for easier visual tasks we explore its efficacy on a more difficult discriminative task: clothes-changing person re-identification (CC-ReID). CC-ReID aims to match people appearing in non-overlapping cameras even when they change their clothes across cameras. Not only are current CC-ReID models constrained by the limited diversity of clothing in current CC-ReID datasets but generating additional data that retains important personal features for accurate identification is a current challenge. To address this issue we propose DLCR a novel data expansion framework that leverages pre-trained diffusion and large language models (LLMs) to accurately generate diverse images of individuals in varied attire. We generate additional data for five benchmark CC-ReID datasets (PRCC CCVID LaST VC-Clothes and LTCC) and increase their clothing diversity by 10X totaling over 2.1M images generated. DLCR employs diffusion-based text-guided inpainting conditioned on clothing prompts constructed using LLMs to generate synthetic data that only modifies a subject's clothes while preserving their personally identifiable features. With this massive increase in data we introduce two novel strategies - progressive learning and test-time prediction refinement - that respectively reduce training time and further boosts CC-ReID performance. On the PRCC dataset we obtain a large top-1 accuracy improvement of 11.3% by training CAL a previous state of the art (SOTA) method with DLCR-generated data. We publicly release our code and generated data for each dataset here: https://github.com/CroitoruAlin/dlcr.

Scene-graph generation involves creating a structural representation of the relationships between objects in a scene by predicting subject-object-relation triplets from input data. Existing methods show poor performance in detecting triplets outside of a predefined set primarily due to their reliance on dependent feature learning. To address this issue we propose DDS- a decoupled dynamic scene-graph generation network- that consists of two independent branches that can disentangle extracted features. The key innovation of the current paper is the decoupling of the features representing the relationships from those of the objects which enables the detection of novel object-relationship combinations. The DDS model is evaluated on three datasets and outperforms previous methods by a significant margin especially in detecting previously unseen triplets.

Remote sensing change detection is crucial for understanding the dynamics of our planet's surface facilitating the monitoring of environmental changes evaluating human impact predicting future trends and supporting decision-making. In this work we introduce a novel approach for change detection that can leverage off-the-shelf unlabeled remote sensing images in the training process by pre-training a Denoising Diffusion Probabilistic Model (DDPM) - a class of generative models used in image synthesis. DDPMs learn the training data distribution by gradually converting training images into a Gaussian distribution using a Markov chain. During inference (i.e. sampling) they can generate a diverse set of samples closer to the training distribution starting from Gaussian noise achieving state-of-the-art image synthesis results. However in this work our focus is not on image synthesis but on utilizing it as a pre-trained feature extractor for the downstream application of change detection. Specifically we fine-tune a lightweight change classifier utilizing the feature representations produced by the pre-trained DDPM alongside change labels. Experiments conducted on the LEVIR-CD WHU-CD DSIFN-CD and CDD datasets demonstrate that the proposed DDPM-CD method significantly outperforms the existing self supervised state-of-the-art change detection methods in terms of F1 score IoU and overall accuracy highlighting the pivotal role of pre-trained DDPM as a feature extractor for downstream applications. Code and pre-trained models available at https://github.com/wgcban/ddpm-cd

Self-Supervised Semantic Segmentation aiming to leverage masses of unlabeled data for boosting semantic segmentation has been rapidly emerging as an active task in recent years. However existing self-supervised semantic segmentation approaches mainly focus on planar images leaving multiple distorted objects encountered in panoramic images unexplored due to the formidable challenge of handling heterogeneous degrees of distortions across different locations. In this paper we propose a novel Self-Supervised Panoramic Semantic Segmentation model termed DASC-SPT built upon the mainstream contrastive learning framework. Towards distortions in panoramic images we present two structures to better learn from distorted features by applying planar images. For the input images of self-supervision we design a Spherical Projection Transformation (SPT) strategy that involves randomly projecting planar images onto various locations of the sphere to introduce the distortions. For pixel-wise distorted features we construct a Deformation-aware Sampling Consistency (DASC) framework to further utilize the shared content and discrepancies caused by different distortions of paired views where the deformation-aware consistency can be quantified on pixel-wise features. Both of the two components facilitate the model to adapt to distortions and boost panoramic semantic segmentation. Extensive comprehensive experiments on three panoramic datasets demonstrate the effectiveness and superiority of DASC-SPT approach.

Test Time Adaptation (TTA) has emerged as a practical solution to mitigate the performance degradation of Deep Neural Networks (DNNs) in the presence of corrupti noise affecting inputs. Existing approaches in TTA continuously adapt the DNN leading to excessive resource consumption and performance degradation due to accumulation of error stemming from lack of supervision. In this work we propose Domain-Aware Real-Time Dynamic Adaptation (DARDA) to address such issues. Our key approach is to proactively learn latent representations of some corruption types each one associated with a sub-network state tailored to correctly classify inputs affected by that corruption. fter deployment DARDA adapts the DNN to previously unseen corruptions in an unsupervised fashion by (i) estimating the latent representation of the ongoing corruption; (ii) selecting the sub-network whose associated corruption is the closest in the latent space to the ongoing corruption; and (iii) adapting DNN state so that its representation matches the ongoing corruption. This way DARDA is more resource-efficient and can swiftly adapt to new distributions caused by different corruptions without requiring a large variety of input data. Through experiments with two popular mobile edge devices - Raspberry Pi and NVIDIA Jetson Nano - we show that DARDA reduces energy consumption and average cache memory footprint respectively by 1.74x and 2.64x with respect to the state of the art while increasing the performance by 10.4% 5.7% and 4.4% on CIFAR-10 CIFAR-100 and TinyImagenet.

We present a parameter-efficient method for continual video question-answering (VidQA) learning. Our method named DAM uses the proposed Dynamic Adapter Merging to (i) mitigate catastrophic forgetting (ii) enable efficient adaptation to continually arriving datasets (iii) handle inputs from unknown datasets during inference and (iv) enable knowledge sharing across similar dataset domains. Given a set of continually streaming VidQA datasets we sequentially train dataset-specific adapters for each dataset while freezing the parameters of a large pretrained video-language backbone. During inference given a video-question sample from an unknown domain our method first uses the proposed non-parametric router function to compute a probability for each adapter reflecting how relevant that adapter is to the current video-question input instance. Subsequently the proposed dynamic adapter merging scheme aggregates all the adapter weights into a new adapter instance tailored for that particular test sample to compute the final VidQA prediction mitigating the impact of inaccurate router predictions and facilitating knowledge sharing across domains. Our DAM model outperforms prior state-of-the-art continual learning approaches by 9.1% while exhibiting 1.9% less forgetting on 6 VidQA datasets spanning various domains. We further extend DAM to continual image classification and image QA and outperform prior methods by a large margin. The code will be publicly released.

Human parsing aims to segment human images into fine-grained semantic parts. Considering the underlying structure of the human body state-of-the-art methods typically depend on prior assumptions to represent intrinsic body relationships. However leveraging the same structural prior knowledge across various scenarios poses challenges in achieving stable prediction and requires additional network design efforts. To address these issues we introduce a novel method the Dynamic Dual Transformer for Parsing (D2FP) which dynamically learns the implicit prior structures of the human body. Specifically we derive input-dependent prior features from the learnable semantics of human images generating prior-embedded object queries accordingly before feeding them into the Transformer decoders. Our model includes three major components to effectively learn prior object queries: a prior extraction module a prior embedding module and a multi-scale dual Transformer decoder. Furthermore a novel prior enhancement strategy is introduced where the final decoded object queries provide structural clues to enhance initial prior features. Experimental results demonstrate the superiority and effectiveness of the proposed method across two well-known human parsing benchmarks: LIP and CIHP. Code and models are available at https://github.com/cvlab-yongin/D2FP.

Post-editing color in photographs is a crucial process for enhancing a photograph's aesthetic value. Traditionally this process has required a significant investment of time and manual effort. Previous color transfer algorithms achieved through encoder-decoder deep learning architectures have simplified this process. However these techniques may introduce artifacts and decrease image quality. Moreover previous approaches are not explainable making the method less user-friendly. In addition the computational requirements of these models limit their deployment across various devices. To address these challenges we introduce the Diffusion-based Look-Up Table (D-LUT). This approach is artifact-free explainable computationally efficient and does not require pretraining stage. It derives a 3D Look-Up Table (3D LUT) for transitioning between the color styles of different images. Specifically this 3D LUT is obtained using a score-matching algorithm followed by color distribution alignment through Langevin dynamics. Our proposed D-LUT approach has achieved state-of-the-art performance while requiring significantly less GPU memory than previous baselines. Importantly the 3D LUTs explicitly derived from the D-LUT algorithm enable color style transfer across broader visual modalities such as real-time color transfer for videos.