WACV 2025 论文列表

Glaucoma is a progressive optic neuropathy characterized by structural damage to the optic nerve head and functional changes in the visual field. Detecting glaucoma early is crucial to preventing loss of eyesight. However medical datasets often suffer from class imbalances making detection more difficult for deep-learning algorithms. We use a generative-based framework to enhance glaucoma diagnosis specifically addressing class imbalance through synthetic data generation. In addition we collected the largest national dataset for glaucoma detection to support our study. The imbalance between normal and glaucomatous cases leads to performance degradation of classifier models. By combining our proposed framework leveraging diffusion models with a pretraining approach we created a more robust classifier training process. This training process results in a better-performing classifier. The proposed approach shows promising results in improving the harmonic mean "sensitivity and specificity" and AUC for the roc for the glaucoma classifier. We report an improvement in the harmonic mean metric from 89.09% to 92.59% on the test set of our national dataset. We examine our method against other methods to overcome imbalance through extensive experiments. We report similar improvements on the AIROGS dataset. This study highlights that diffusion-based generation can be of great importance in tackling class imbalances in medical datasets to improve diagnostic performance.

Cone-Beam Computed Tomography (CBCT) has garnered significant attention due to lower radiation dosage and faster scanning time which has been widely used in clinical applications for decades. However its poor image quality is always challenging to clinical experts. To address this problem we propose our work DiffuCE a Diffusion model framework for CBCT Enhancement. The main contributions of our work are three-fold: (1) Increased Generalizability: Our training data exclusively comprises pixel space data eliminating the necessity for additional imaging machine settings. This emphasizes the model's ability to generalize effectively across diverse conditions. (2) Efficient Training: Rather than starting from scratch our approach fine-tunes from a well-established foundation model. This illustrates the viability of efficient training strategies for medical image restoration tasks optimizing resource utilization. (3) Competitive Performance: DiffuCE exhibits outstanding performance excelling in FID and LPIPS with 0.01 and 36.99 ahead of the second place in the private set. In the public dataset DiffuCE has a competitive performance compared to other SOTAs. Moreover in expert assessments DiffuCE achieves the highest score of 7.06 for overall satisfaction which is 1.38 ahead of the second place affirming its performance from a clinical standpoint. Codes are available at https://github.com/lzh107u/DiffuCE

Without access to the original training data data-free quantization (DFQ) aims to recover the performance loss induced by quantization. Most previous works have focused on using an original network to extract the train data information which is instilled into surrogate synthesized images. However existing DFQ methods do not take into account important aspects of quantization: the extent of a computational-cost-and-accuracy trade-off varies for each image depending on its task difficulty. To handle such varying trade-offs several efforts have been made to dynamically allocate bit-widths for each image. Such dynamic quantization however remains challenging and unexplored in the data-free domain because synthesized images of previous works fail to possess properties in natural test images that are crucial for learning the appropriate dynamic allocation policy: difficulty its diversity and its plausibility. By contrast we propose a data-free quantization framework that is dynamic-friendly by modeling varying extents of task difficulties with plausibility. We generate plausibly difficult images with soft labels whose probabilities are allocated to a group of similar classes. Images with diverse and plausible difficulties enable us to train the framework to dynamically handle the varying trade-offs. Consequently our framework achieves better accuracy-complexity Pareto front than existing data-free quantization approaches.

Human motion analysis offers significant potential for healthcare monitoring and early detection of diseases. The advent of radar-based sensing systems has captured the spotlight for they are able to operate without physical contact and they can integrate with pre-existing Wi-Fi networks. They are also seen as less privacy-invasive compared to camera-based systems. However recent research has shown high accuracy in recognizing subjects or gender from radar gait patterns raising privacy concerns. This study addresses these issues by investigating privacy vulnerabilities in radar-based Human Activity Recognition (HAR) systems and proposing a novel method for privacy preservation using Differential Privacy (DP) driven by attributions derived with Integrated Decision Gradient (IDG) algorithm. We investigate Black-box Membership Inference Attack (MIA) Models in HAR settings across various levels of attacker-accessible information. We extensively evaluated the effectiveness of the proposed IDG-DP method by designing a CNN-based HAR model and rigorously assessing its resilience against MIAs. Experimental results demonstrate the potential of IDG-DP in mitigating privacy attacks while maintaining utility across all settings particularly excelling against label-only and shadow model black-box MIA attacks. This work represents a crucial step towards balancing the need for effective radar-based HAR with robust privacy protection in healthcare environments.

Training data privacy is a fundamental problem in modern Artificial Intelligence (AI) applications such as face recognition recommendation systems language generation and many others as it may contain sensitive user information related to legal issues. To fundamentally understand how privacy mechanisms work in AI applications we study differential privacy (DP) in the Neural Tangent Kernel (NTK) regression setting where DP is one of the most powerful tools for measuring privacy under statistical learning and NTK is one of the most popular analysis frameworks for studying the learning mechanisms of deep neural networks. In our work we can show provable guarantees for both differential privacy and test accuracy of our NTK regression. Furthermore we conduct experiments on the basic image classification dataset CIFAR10 to demonstrate that NTK regression can preserve good accuracy under a modest privacy budget supporting the validity of our analysis. To our knowledge this is the first work to provide a DP guarantee for NTK regression.

With the success of Diffusion Models for image generation the technologies also have revolutionized the aesthetic Quick Response (QR) code generation. Despite significant improvements in visual attractiveness for the beautified codes their scannabilities are usually sacrificed and thus hinder their practical uses in real-world scenarios. To address this issue we propose a novel training-free Diffusion-based QR Code generator (DiffQRCoder) to effectively craft both scannable and visually pleasing QR codes. The proposed approach introduces Scanning-Robust Perceptual Guidance (SRPG) a new diffusion guidance for Diffusion Models to guarantee the generated aesthetic codes to obey the ground-truth QR codes while maintaining their attractiveness during the denoising process. Additionally we present another post-processing technique Scanning Robust Manifold Projected Gradient Descent (SR-MPGD) to further enhance their scanning robustness through iterative latent space optimization. With extensive experiments the results demonstrate that our approach not only outperforms other compared methods in Scanning Success Rate (SSR) with better or comparable CLIP aesthetic score (CLIP-aes.) but also significantly improves the SSR of the ControlNet-only approach from 60% to 99%. The subjective evaluation indicates that our approach achieves promising visual attractiveness to users as well. Finally even with different scanning angles and the most rigorous error tolerance settings our approach robustly achieves over 95% SSR demonstrating its capability for real-world applications. Our project page is available at https://jwliao1209.github.io/DiffQRCoder.

In the ever-evolving adversarial machine learning landscape developing effective defenses against patch attacks has become a critical challenge necessitating reliable solutions to safeguard real-world AI systems. Although diffusion models have shown remarkable capacity in image synthesis and have been recently utilized to counter l_p-norm bounded attacks their potential in mitigating localized patch attacks remains largely underexplored. In this work we propose DiffPAD a novel framework that harnesses the power of diffusion models for adversarial patch decontamination. DiffPAD first performs super-resolution restoration on downsampled input images then adopts binarization dynamic thresholding scheme and sliding window for effective localization of adversarial patches. Such a design is inspired by the theoretically derived correlation between patch size and diffusion restoration error that is generalized across diverse patch attack scenarios. Finally DiffPAD applies inpainting techniques to the original input images with the estimated patch region being masked. By integrating closed-form solutions for super-resolution restoration and image inpainting into the conditional reverse sampling process of a pre-trained diffusion model DiffPAD obviates the need for text guidance or fine-tuning. Through comprehensive experiments we demonstrate that DiffPAD not only achieves state-of-the-art adversarial robustness against patch attacks but also excels in recovering naturalistic images without patch remnants. The source code is available at https://github.com/JasonFu1998/DiffPAD.

Human mesh recovery (HMR) provides rich human body information for various real-world applications such as gaming human-computer interaction and virtual reality. While image-based HMR methods have achieved impressive results they often struggle to recover humans in dynamic scenarios leading to temporal inconsistencies and non-smooth 3D motion predictions due to the absence of human motion. In contrast video-based approaches leverage temporal information to mitigate this issue. In this paper we present DiffMesh an innovative motion-aware diffusion framework for video-based HMR. DiffMesh establishes a bridge between diffusion models and human motion efficiently generating accurate and smooth output mesh sequences by incorporating human motion within the forward process and reverse process in the diffusion model. Extensive experiments are conducted on the widely used datasets (Human3.6M and 3DPW) which demonstrate the effectiveness and efficiency of our DiffMesh. Visual comparisons in real-world scenarios further highlight DiffMesh's suitability for practical applications. The project webpage is: https://zczcwh.github.io/ diffmesh_page/

Diagnosing dementia particularly for Alzheimer's Disease (AD) and frontotemporal dementia (FTD) is complex due to overlapping symptoms. While magnetic resonance imaging (MRI) and positron emission tomography (PET) data are critical for the diagnosis integrating these modalities in deep learning faces challenges often resulting in suboptimal performance compared to using single modalities. Moreover the potential of multi-modal approaches in differential diagnosis which holds significant clinical importance remains largely unexplored. We propose a novel framework DiaMond to address these issues with vision Transformers to effectively integrate MRI and PET. DiaMond is equipped with self-attention and a novel bi-attention mechanism that synergistically combine MRI and PET alongside a multi-modal normalization to reduce redundant dependency thereby boosting the performance. DiaMond significantly outperforms existing multi-modal methods across various datasets achieving a balanced accuracy of 92.4% in AD diagnosis 65.2% for AD-MCI-CN classification and 76.5% in differential diagnosis of AD and FTD. We also validated the robustness of DiaMond in a comprehensive ablation study. The code is available at https://github.com/ai-med/DiaMond.

Diffusion Transformers (DiTs) have recently attracted significant interest from both industry and academia due to their enhanced capabilities in visual generation surpassing the performance of traditional diffusion models that employ U-Net. However the improved performance of DiTs comes at the expense of higher parameter counts and implementation costs which significantly limits their deployment on resource-constrained devices like mobile phones. We propose DiTAS a data-free post-training quantization (PTQ) method for efficient DiT inference. DiTAS relies on the proposed temporal-aggregated smoothing techniques to mitigate the impact of the channel-wise outliers within the input activations leading to much lower quantization error under extremely low bitwidth. To further enhance the performance of the quantized DiT we adopt the layer-wise grid search strategy to optimize the smoothing factor. Moreover we integrate a training-free LoRA module for weight quantization leveraging alternating optimization to minimize quantization errors without additional fine-tuning. Experimental results demonstrate that our approach enables 4-bit weight 8-bit activation (W4A8) quantization for DiTs while maintaining comparable performance as the full-precision model. Code is available at https://github.com/DZY122/DiTAS.

Although object detection models are widely used their predictive performance has been shown to deteriorate when faced with abnormal scenes. Such abnormalities can occur naturally (by partially occluded or out-of-distribution objects) or deliberately (in the case of an adversarial attack). Existing uncertainty quantification methods such as object detection evaluation metrics and label-uncertainty quantification techniques do not consider the abnormalities' effect on the model's internal decision-making process. Furthermore practical methods that consider the effects of abnormalities (such as abnormality detection and mitigation) are designed to deal with one type of abnormality. We present distinctive localization (DiL) an unsupervised practical and explainable metric that quantitatively interprets any type of abnormality and can be leveraged for preventive purposes. By utilizing XAI techniques (saliency maps) DiL maps the objectness of a given scene and captures the model's inner uncertainty regarding the identified (and missed) objects. DiL was evaluated across nine use cases including partially occluded and out-of-distribution objects as well as adversarial patches in both physical and digital spaces on benchmark datasets and our newly E-PO dataset (generated with DALL-E 2). Our results show that DiL: i) successfully interprets and quantifies an abnormality's effect on the model's decision-making process regardless of the abnormality type; and ii) can be leveraged to detect and mitigate this effect.

We introduce DiHuR a novel Diffusion-guided model for generalizable Human 3D Reconstruction and view synthesis from sparse minimally overlapping images. While existing generalizable human radiance fields excel at novel view synthesis they often struggle with comprehensive 3D reconstruction. Similarly directly optimizing implicit Signed Distance Function (SDF) fields from sparse-view images typically yields poor results due to limited overlap. To enhance 3D reconstruction quality we propose using learnable tokens associated with SMPL vertices to aggregate sparse view features and then to guide SDF prediction. These tokens learn a generalizable prior across different identities in training datasets leveraging the consistent projection of SMPL vertices onto similar semantic areas across various human identities. This consistency enables effective knowledge transfer to unseen identities during inference. Recognizing SMPL's limitations in capturing clothing details we incorporate a diffusion model as an additional prior to fill in missing information particularly for complex clothing geometries. Our method integrates two key priors in a coherent manner: the prior from generalizable feed-forward models and the 2D diffusion prior and it requires only multi-view image training without 3D supervision. DiHuR demonstrates superior performance in both within-dataset and cross-dataset generalization settings as validated on THuman ZJU-MoCap and HuMMan datasets compared to existing methods.

Aerial investigation is used for surveying damage and identifying post-disaster events through imagery data. However the challenge lies in detecting disaster-related areas within aerial or shipborne images as these can appear as minor regions making recognition difficult. To address this challenge we introduce the Detective Network (DeNet) designed to optimally mask images thereby shifting the attention of machine learning models towards these small yet crucial regions. Utilizing the concepts of patch and anchor box DeNet incorporates a masking candidate layer and a masking layer to facilitate optimal masking. Our experimental findings are compelling; by preprocessing images with DeNet before analysis using an image captioning model we achieved a remarkable accuracy of 92.91% in landslide detection from side-view image captions and 87.50% for shipborne view detection. The result demonstrates the efficacy of DeNet in enhancing the recognition of disaster-related areas in challenging imaging conditions.

Early detection of wildfires is essential to prevent large-scale fires resulting in extensive environmental structural and societal damage. Uncrewed aerial vehicles (UAVs) can cover large remote areas effectively with quick deployment requiring minimal infrastructure and equipping them with small cameras and computers enables autonomous real-time detection. In remote areas however detection methods are limited to onboard computation due to the lack of high-bandwidth mobile networks. For accurate camera-based localisation segmentation of the detected smoke is essential but training data for deep learning-based wildfire smoke segmentation is limited. This study shows how small specialised segmentation models can be trained using only bounding box labels leveraging zero-shot foundation model supervision. The method offers the advantages of needing only fairly easily obtainable bounding box labels and requiring training solely for the smaller student network. The proposed method achieved 63.3 percent mIoU on a manually annotated and diverse wildfire dataset. The used model can perform in real-time at 25 fps with a UAV-carried NVIDIA Jetson Orin NX computer while reliably recognising smoke as demonstrated at real-world forest burning events. Code is available at: https://gitlab.com/fgi_nls/public/wildfire-real-time-segmentation

Modern text-to-image (T2I) diffusion models can generate images with remarkable realism and creativity. These advancements have sparked research in fake image detection and attribution yet prior studies have not fully explored the practical and scientific dimensions of this task. In addition to attributing images to 12 state-of-the-art T2I generators we provide extensive analyses on what inference stage hyperparameters and image modifications are discernible. Our experiments reveal that initialization seeds are highly detectable along with other subtle variations in the image generation process to some extent. We further investigate what visual traces are leveraged in image attribution by perturbing high-frequency details and employing mid-level representations of image style and structure. Notably altering high-frequency information causes only slight reductions in accuracy and training an attributor on style representations outperforms training on RGB images. Our analyses underscore that fake images are detectable and attributable at various levels of visual granularity.

Graphic designs are an effective medium for visual communication. They range from greeting cards to corporate flyers and beyond. Off-late machine learning techniques are able to generate such designs which accelerates the rate of content production. An automated way of evaluating their quality becomes critical. Towards this end we introduce Design-o-meter a data-driven methodology to quantify the goodness of graphic designs. Further our approach can suggest modifications to these designs to improve its visual appeal. To the best of our knowledge Design-o-meter is the first approach that scores and refines designs in a unified framework despite the inherent subjectivity and ambiguity of the setting. Our exhaustive quantitative and qualitative analysis of our approach against baselines adapted for the task (including recent Multimodal LLM based approaches) brings out the efficacy of our methodology. We hope our work will usher more interest in this important and pragmatic problem setting. Project Page: https://sahilg06.github.io/Design-o-meter/

Recent advancements in image denoising have leveraged neural networks to enhance performance particularly in scenarios where clean-noisy image pairs are unavailable. In this context self-supervised image denoising methods have gained prominence centered around the principle of J-invariance -- ensuring that the output pixel is not influenced by its corresponding input pixel. Traditionally enforcing J-invariance has constrained blind spot network (BSN) designs requiring even core operations such as upsampling or downsampling to follow complex rules. This limitation has led to the exclusion of efficient multi-resolution architectures such as U-net increasing computational complexity. To address these constraints we introduce generalized design principles for multi-scale J-invariant networks that allow for the flexible incorporation of nearly any architectural elements. This approach challenges the prevailing notion that J-invariance must be maintained throughout the entire process. Based on our design principles we present U-BSN a novel J-invariant network design that utilizes the versatile U-Net architecture adapting it to accommodate self-supervised learning effectively. We also propose randomized PD an advanced technique that enhances denoising of real-world images with structured noise. Experimental results validate that U-BSN surpasses existing BSNs in handling real-world noise scenarios and achieves the lowest computational complexity among comparable networks thus confirming the effectiveness of our design principles and proposed methodologies.

We demonstrate an image dequantizing diffusion model that enables novel edits on natural images. We propose operating on quantized images because they offer easy abstraction for patch-based edits and palette transfer. In particular, we show that color palettes can make the output of the diffusion model easier to control and interpret. We first establish that existing image restoration methods are not sufficient, such as JPEG noise reduction models. We then demonstrate that our model can generate natural images that respect the color palette the user asked for. For palette transfer, we propose a method based on weighted bipartite matching. We then show that our model generates plausible images even after extreme palette transfers, respecting user query. Our method can optionally condition on the source texture in part or all of the image. In doing so, we overcome a common problem in existing image colorization methods that are unable to produce colors with a different luminance than the input. We evaluate several possibilities for texture conditioning and their trade-offs, including luminance, image gradients, and thresholded gradients, the latter of which performed best in maintaining texture and color control simultaneously. Our method can be usefully extended to another practical edit: recoloring patches of an image while respecting the source texture. Our procedure is supported by several qualitative and quantitative evaluations.

The task of 3D semantic scene completion using monocular cameras is gaining significant attention in the field of autonomous driving. This task aims to predict the occupancy status and semantic labels of each voxel in a 3D scene from partial image inputs. Despite numerous existing methods many face challenges such as inaccurately predicting object shapes and misclassifying object boundaries. To address these issues we propose DepthSSC an advanced method for semantic scene completion using only monocular cameras. DepthSSC integrates the Spatial Transformation Graph Fusion (ST-GF) module with Geometric-Aware Voxelization (GAV) enabling dynamic adjustment of voxel resolution to accommodate the geometric complexity of 3D space. This ensures precise alignment between spatial and depth information effectively mitigating issues such as object boundary distortion and incorrect depth perception found in previous methods. Evaluations on the SemanticKITTI and SSCBench-KITTI-360 dataset demonstrate that DepthSSC not only captures intricate 3D structural details effectively but also achieves state-of-the-art performance.