WACV 2025 论文列表

We propose a method for dense depth estimation from an event stream generated when sweeping the focal plane of the driving lens attached to an event camera. In this method a depth map is inferred from an "event focal stack" composed of the event stream using a convolutional neural network trained with synthesized event focal stacks. The synthesized event stream is created from a focal stack generated by Blender for any arbitrary 3D scene. This allows for training on scenes with diverse structures. Additionally we explored methods to eliminate the domain gap between real event streams and synthetic event streams. Our method demonstrates superior performance over a depth-from-defocus method in the image domain on synthetic and real datasets.

Advances in microscopy imaging enable researchers to visualize structures at the nanoscale level thereby unraveling intricate details of biological organization. However challenges such as image noise photobleaching of fluorophores and low tolerability of biological samples to high light doses remain restricting temporal resolutions and experiment durations. Reduced laser doses enable longer measurements at the cost of lower resolution and increased noise which hinders accurate downstream analyses. Here we train a denoising diffusion probabilistic model (DDPM) to predict high-resolution images by conditioning the model on low-resolution information. Additionally the probabilistic aspect of the DDPM allows for repeated generation of images that tend to further increase the signal-to-noise ratio. We show that our model achieves a performance that is better or similar to the previously best-performing methods across four highly diverse datasets. Importantly while any of the previous methods show competitive performance for some but not all datasets our method consistently achieves high performance across all four data sets suggesting high generalizability. Our code and datasets are available at https://github.com/kaschube-lab/ddpm_highres_microscopy https://doi.org/10.5281/zenodo.14178277 https://doi.org/10.5281/zenodo.14215837.

Neural Architecture Search (NAS) continues to serve a key roll in the design and development of neural networks for task specific deployment. Modern NAS techniques struggle to deal with ever increasing search space complexity and compute cost constraints. Existing approaches can be categorized into two buckets: fine-grained computational expensive NAS and coarse-grained low cost NAS. Our objective is to craft an algorithm with the capability to perform fine-grain NAS at a low cost. We propose projecting the problem to a lower dimensional space through predicting the difference in accuracy of a pair of similar networks. This paradigm shift allows for reducing computational complexity from exponential down to linear with respect to the size of the search space. We present a strong mathematical foundation for our algorithm in addition to extensive experimental results across a host of common NAS Benchmarks. Our methods significantly out performs existing works achieving better performance coupled with a significantly higher sample efficiency.

Semi-supervised learning (SSL) has achieved remarkable performance with a small fraction of labeled data by leveraging vast amounts of unlabeled data from the Internet. However this large pool of untrusted data is extremely vulnerable to data poisoning leading to potential backdoor attacks. Current backdoor defenses are not yet effective against such a vulnerability in SSL. In this study we propose a novel method Unlabeled Data Purification (UPure) to disrupt the association between trigger patterns and target classes by introducing perturbations in the frequency domain. By leveraging the Rate-Distortion-Perception (RDP) trade-off we further identify the frequency band where the perturbations are added and justify this selection. Notably UPure purifies poisoned unlabeled data without the need of extra clean labeled data. Extensive experiments on four benchmark datasets and five SSL algorithms demonstrate that UPure effectively reduces the attack success rate from 99.78% to 0% while maintaining model accuracy. Code is available here: https://github.com/chengyi-chris/UPure.

Deep supervision which involves extra supervisions to the intermediate features of a neural network was widely used in image classification in the early deep learning era since it significantly reduces the training difficulty and eases the optimization like avoiding gradient vanish over the vanilla training. Nevertheless with the emergence of normalization techniques and residual connection deep supervision in image classification was gradually phased out. In this paper we revisit deep supervision for masked image modeling (MIM) that pre-trains a Vision Transformer (ViT) via a mask-and-predict scheme. Experimentally we find that deep supervision drives the shallower layers to learn more meaningful representations accelerates model convergence and expands attention diversities. Our approach called DeepMIM significantly boosts the representation capability of each layer. In addition DeepMIM is compatible with many MIM models across a range of reconstruction targets. For instance using ViT-B DeepMIM on MAE achieves 84.2 top-1 accuracy on ImageNet outperforming MAE by +0.6. By combining DeepMIM with a stronger tokenizer CLIP our model achieves state-of-the-art performance on various downstream tasks including image classification (85.6 top-1 accuracy on ImageNet-1K outperforming MAE-CLIP by +0.8) object detection (52.8 AP^box on COCO) and semantic segmentation (53.1 mIoU on ADE20K). Code and models are available at https://github.com/OliverRensu/DeepMIM.

Cardiovascular diseases (CVDs) are the most common cause of death worldwide. Invasive x-ray coronary angiography (ICA) is one of the most important imaging modalities for the diagnosis of CVDs. ICA typically acquires only two 2D projections which makes the 3D geometry of coronary vessels difficult to interpret thus requiring 3D coronary artery tree reconstruction from two projections. State-of-the-art approaches require significant manual interactions and cannot correct the non-rigid cardiac and respiratory motions between non-simultaneous projections. In this study we propose a novel deep learning pipeline named DeepCA. We leverage the Wasserstein conditional generative adversarial network with gradient penalty latent convolutional transformer layers and a dynamic snake convolutional critic to implicitly compensate for the non-rigid motion and provide 3D coronary artery tree reconstruction. Through simulating projections from coronary computed tomography angiography (CCTA) we achieve the generalisation of 3D coronary tree reconstruction on real non-simultaneous ICA projections. We incorporate an application-specific evaluation metric to validate our proposed model on both a CCTA dataset and a real ICA dataset together with Chamfer l_2 distance. The results demonstrate promising performance of our DeepCA model in vessel topology preservation recovery of missing features and generalisation ability to real ICA data. To the best of our knowledge this is the first study that leverages deep learning to achieve 3D coronary tree reconstruction from two real non-simultaneous x-ray angiographic projections. The implementation of this work is available at: https://github.com/WangStephen/DeepCA.

Remote Sensing Change Detection (RS-CD) aims to detect relevant changes from Multi-Temporal Remote Sensing Images (MT-RSIs) which aids in various RS applications such as land cover land use human development analysis and disaster response. The performance of existing RS-CD methods is attributed to training on large annotated datasets. Furthermore most of these models are less transferable in the sense that the trained model often performs very poorly when there is a domain gap between training and test datasets. This paper proposes an unsupervised CD method based on deep metric learning that can deal with both of these issues. Given an MT-RSI the proposed method generates corresponding change probability map by iteratively optimizing an unsupervised CD loss without training it on a large dataset. Our unsupervised CD method consists of two interconnected deep networks namely Deep-Change Probability Generator (D-CPG) and Deep-Feature Extractor (D-FE). The D-CPG is designed to predict change and no change probability maps for a given MT-RSI while D-FE is used to extract deep features of MT-RSI that will be further used in the proposed unsupervised CD loss. We use transfer learning capability to initialize the parameters of D-FE. We iteratively optimize the parameters of D-CPG and D-FE for a given MT-RSI by minimizing the proposed unsupervised similarity-dissimilarity loss. This loss is motivated by the principle of metric learning where we simultaneously maximize the distance between change pair-wise pixels while minimizing the distance between no-change pair-wise pixels in bi-temporal image domain and their deep feature domain. The experiments conducted on three CD datasets show that our unsupervised CD method achieves significant improvements over the state-of-the-art supervised and unsupervised CD methods. Code and pre-trained models available at https://github.com/wgcban/Metric-CD

Low-light and blurring issues are prevalent when capturing photos at night often due to the use of long exposure to address dim environments. Addressing these joint problems can be challenging and error-prone if an end-to-end model is trained without incorporating an appropriate physical model. In this paper we introduce JUDE a Deep Joint Unrolling for Deblurring and Low-Light Image Enhancement inspired by the image physical model. Based on Retinex theory and the blurring model the low-light blurry input is iteratively deblurred and decomposed producing sharp low-light reflectance and illuminance through an unrolling mechanism. Additionally we incorporate various modules to estimate the initial blur kernel enhance brightness and eliminate noise in the final image. Comprehensive experiments on LOL-Blur and Real-LOL-Blur demonstrate that our method outperforms existing techniques both quantitatively and qualitatively.

To address the data scarcity associated with 3D assets 2D-lifting techniques such as Score Distillation Sampling (SDS) have become a widely adopted practice in text-to-3D generation pipelines. However the diffusion models used in these techniques are prone to viewpoint bias and thus lead to geometric inconsistencies such as the Janus problem. To counter this we introduce MT3D a text-to-3D generative model that leverages a high-fidelity 3D object to overcome viewpoint bias and explicitly infuse geometric understanding into the generation pipeline. Firstly we employ depth maps derived from a high-quality 3D model as control signals to guarantee that the generated 2D images preserve the fundamental shape and structure thereby reducing the inherent viewpoint bias. Next we utilize deep geometric moments to ensure geometric consistency in the 3D representation explicitly. By incorporating geometric details from a 3D asset MT3D enables the creation of diverse and geometrically consistent objects thereby improving the quality and usability of our 3D representations. Project page and code: https://moment-3d.github.io/

We introduce ViDSE a Video framework that Deduce and Selects visual Evidence for training-free video goal inference using language models. Unlike approaches that directly apply vision-language models (VLM) or combine VLM+LLM to process dense video visuals ViDSE explicitly selects relevant visual evidence (e.g. frames) based on the hypothesis deduced by the LLM. This approach not only improves accuracy but also reveals the logical process behind the model's decisions enhancing explainability. Our experiments demonstrate that this selection process significantly reduces ambiguity in the subsequent inference reasoning stage and outperforms VLM-only and VLM+LLM models on goal inference tasks such as CrossTask and COIN. We further validate ViDSE's generalizability and robustness on action recognition tasks such as ActivityNet and UCF-101 under training-free and open-vocabulary conditions. We observe that ViDSE easily generalizes to other video tasks (e.g. action recognition) requiring filtering of redundant and irrelevant information.

Open World Object Detection (OWOD) is a challenging computer vision task that extends standard object detection by (1) detecting and classifying unknown objects without supervision and (2) incrementally learning new object classes without forgetting previously learned ones. The absence of ground truths for unknown objects makes OWOD tasks particularly challenging. Many methods have addressed this by using pseudo-labels for unknown objects. The recently proposed Probabilistic Objectness transformer-based open-world detector (PROB) is a state-of-the-art model that does not require pseudo-labels for unknown objects as it predicts probabilistic objectness. However this method faces issues with learning conflicts between objectness and class predictions. To address this issue and further enhance performance we propose a novel model Decoupled PROB. Decoupled PROB introduces Early Termination of Objectness Prediction (ETOP) to stop objectness predictions at appropriate layers in the decoder resolving the learning conflicts between class and objectness predictions in PROB. Additionally we introduce Task-Decoupled Query Initialization (TDQI) which efficiently extracts features of known and unknown objects thereby improving performance. TDQI is a query initialization method that combines query selection and learnable queries and it is a module that can be easily integrated into existing DETR-based OWOD models. Extensive experiments on OWOD benchmarks demonstrate that Decoupled PROB surpasses all existing methods across several metrics significantly improving performance.

Dataset Condensation (DC) aims to reduce deep neural networks training efforts by synthesizing a small dataset such that it will be as effective as the original large dataset. Conventionally DC relies on a costly bi-level optimization which prohibits its practicality. Recent research formulates DC as a distribution matching problem which circumvents the costly bi-level optimization. However this efficiency sacrifices the DC performance. To investigate this performance degradation we decomposed the dataset distribution into content and style. Our observations indicate two major shortcomings of: 1) style discrepancy between original and condensed data and 2) limited intra-class diversity of condensed dataset. We present a simple yet effective method to match the style information between original and condensed data employing statistical moments of feature maps as well-established style indicators. Moreover we enhance the intra-class diversity by maximizing the Kullback-Leibler divergence within each synthetic class i.e. content. We demonstrate the efficacy of our method through experiments on diverse datasets of varying size and resolution achieving improvements of up to 4.1% on CIFAR10 4.2% on CIFAR100 4.3% on TinyImageNet 2.0% on ImageNet-1K 3.3% on ImageWoof 2.5% on ImageNette and 5.5% in continual learning accuracy.

In today's competitive marketing landscape effective complaint management is crucial for customer service and business success. Video complaints integrating text and image content offer invaluable insights by addressing customer grievances and delineating product benefits and drawbacks. However comprehending nuanced complaint aspects within vast daily multimodal financial data remains a formidable challenge. Addressing this gap we have curated a proprietary multimodal video complaint dataset comprising 433 publicly accessible instances. Each instance is meticulously annotated at the utterance level encompassing five distinct categories of financial aspects and their associated complaint labels. To support this endeavour we introduce Solution 3.0 a model designed for multimodal aspect-based complaint identification task. Solution 3.0 is tailored to perform three key tasks: 1) handling multimodal features (audio and video) 2) facilitating multilabel aspect classification and 3) conducting multitasking for aspect classifications and complaint identification parallelly. Solution 3.0 utilizes a CLIP-based dual frozen encoder with an integrated image segment encoder for global feature fusion enhanced by contextual attention (ISEC) to improve accuracy and efficiency. Our proposed framework surpasses current multimodal baselines exhibiting superior performance across nearly all metrics by opening new ways to strengthen appropriate customer care initiatives and effectively assisting individuals in resolving their problems.

Simplicity bias is a critical challenge in neural networks since it often leads to favoring simpler solutions and learning unintended decision rules captured by spurious correlations causing models to be biased and diminishing their generalizability. While existing solutions rely on human supervision obtaining annotations of the different bias attributes is often impractical. To tackle this we present Debiasify a novel self-distillation approach that works without any prior information about the nature of biases. Our method leverages a new distillation loss to distill knowledge within a network; from a deep layer where complex highly-predictive features reside to a shallow layer where simpler yet attribute-conditioned features are found in an unsupervised manner. In this way Debiasify learns robust debiased representations that generalize well across various biases and datasets enhancing worst-group performance and overall accuracy. Extensive experiments on computer vision and medical imaging benchmarks show the efficacy of our method significantly outperforming the previous unsupervised debiasing methods (e.g. a 10.13% improvement in worst-group accuracy on Wavy Hair classification in CelebA) while achieving comparable or superior performance to supervised methods. Our code is publicly available at the following link:Debiasify.

Generative models can create entirely new images but they can also partially modify real images in ways that are undetectable to the human eye. In this paper we address the challenge of automatically detecting such local manipulations. One of the most pressing problems in deepfake detection remains the ability of models to generalize to different classes of generators. In the case of fully manipulated images representations extracted from large self-supervised models (such as CLIP) provide a promising direction towards more robust detectors. Here we introduce DeCLIP-- a first attempt to leverage such large pretrained features for detecting local manipulations. We show that when combined with a reasonably large convolutional decoder pretrained self-supervised representations are able to perform localization and improve generalization capabilities over existing methods. Unlike previous work our approach is able to perform localization on the challenging case of latent diffusion models where the entire image is affected by the fingerprint of the generator. Moreover we observe that this type of data which combines local semantic information with a global fingerprint provides more stable generalization than other categories of generative methods

This paper proposes a dataset augmentation method by fine-tuning pre-trained diffusion models. Generating images using a pre-trained diffusion model with textual conditioning often results in domain discrepancy between real data and generated images. We propose a fine-tuning approach where we adapt the diffusion model by conditioning it with real images and novel text embeddings. We introduce a unique procedure called Mixing Visual Concepts (MVC) where we create novel text embeddings from image captions. The MVC enables us to generate multiple images which are diverse and yet similar to the real data enabling us to perform effective dataset augmentation. We perform comprehensive qualitative and quantitative evaluations with the proposed dataset augmentation approach showcasing both coarse-grained and fine-grained changes in generated images. Our approach outperforms state-of-the-art augmentation techniques on benchmark classification tasks. The code is available at https://github.com/rahatkutubi/MVC

Recent advances in deep learning have witnessed many successful medical image translation models that learn correspondences between two visual domains. However building robust mappings between domains is a significant challenge when handling misalignments caused by factors such as respiratory motion and anatomical changes. This issue is further exacerbated in scenarios with limited data availability leading to a significant degradation in translation quality. In this paper we introduce a novel data-efficient framework for aligning medical images via Reconfigurable Generative Network (Reconfig-MIT) for high-quality image translation. The key idea of Reconfig-MIT is to adaptively expand the generative network width within a Generative Adversarial Networks (GAN) architecture initially expanding rapidly to capture low-level features and then slowing to refine high-level complexities. This dynamic network adaptation mechanism allows to adaptively learn at different rates thus the model can better respond to deviations in the data caused by misalignments while maintaining an effective equilibrium with the discriminator (D). We also introduce the Recursive Cycle-Consistency Loss (R-CCL) which extends the cycle consistency loss to effectively preserve key anatomical structures and their spatial relationships improving translation quality. Extensive experiments show that Reconfig-MIT is a generic framework that enables easy integration with existing image translation methods including those incorporating registration networks used for correcting misalignments and provides robust and high-quality translation on paired and unpaired misaligned data in both data-rich and data-limited scenarios. https://github.com/IntellicentAI-Lab/Reconfig-MIT.

3D visual grounding aims to identify the target object within a 3D point cloud scene referred to by a natural language description. Previous works usually require significant data relating to point color and their descriptions to exploit the corresponding complicated verbo-visual relations. In our work we introduce Vigor a novel Data-Efficient 3D Visual Grounding framework via Order-aware Referring. Vigor leverages LLM to produce a desirable referential order from the input description for 3D visual grounding. With the proposed stacked object-referring blocks the predicted anchor objects in the above order allow one to locate the target object progressively without supervision on the identities of anchor objects or exact relations between anchor/target objects. We also present an order-aware warm-up training strategy which augments referential orders for pre-training the visual grounding framework allowing us to better capture the complex verbo-visual relations and benefit the desirable data-efficient learning scheme. Experimental results on the NR3D and ScanRefer datasets demonstrate our superiority in low-resource scenarios. In particular Vigor surpasses current state-of-the-art frameworks by 9.3% and 7.6% grounding accuracy under 1% data and 10% data settings on the NR3D dataset respectively.

Zero-shot quantization (ZSQ) using synthetic data is a key approach for post-training quantization (PTQ) under privacy and security constraints. However existing data generation methods often struggle to effectively generate data suitable for hardware-friendly quantization where all model layers are quantized. We analyze existing data generation methods based on batch normalization (BN) matching and identify several gaps between synthetic and real data: 1) Current generation algorithms do not optimize the entire synthetic dataset simultaneously; 2) Data augmentations applied during training are often overlooked; and 3) A distribution shift occurs in the final model layers due to the absence of BN in those layers. These gaps negatively impact ZSQ performance particularly in hardware-friendly quantization scenarios. In this work we propose Data Generation for Hardware-Friendly Quantization (DGH) a novel method that addresses these gaps. DGH jointly optimizes all generated images regardless of the image set size or GPU memory constraints. To address data augmentation mismatches DGH includes a preprocessing stage that mimics the augmentation process and enhances image quality by incorporating natural image priors. Finally we propose a new distribution-stretching loss that aligns the support of the feature map distribution between real and synthetic data. This loss is applied to the model's output and can be adapted to various tasks. DGH demonstrates significant improvements in quantization performance across multiple tasks achieving up to a 30% increase in accuracy for hardware-friendly ZSQ in both classification and object detection often performing on par with real data.