WACV 2025 论文列表

The discovery of unique treatment candidates for complex diseases is a challenging task for current drug discovery programs. Biopharma research has developed automated and scalable screening assays of cell culture models to screen thousands of drug candidates in parallel e.g. by considering bio-image based assays. However the large amount of data hinders a systematic review by human experts to distinguish between different disease and healthy phenotypes. A prevalent approach to uncover phenotypic endpoints in a dataset is based on the concept of archetypal analysis which seeks for extremal points in a dataset. State-of-the-art non-linear archetypal methods based on variational autoencoders require k - 1 latent dimensions to encode k archetypes. However in high content imaging we frequently require a significantly larger number of latent dimensions than archetypes to encode HCIs which results in weak latent representations and ambiguous archetypes. To overcome this limitation we propose to relax the simplex constraint in the latent space to a unit hypersphere and learn the respective archetypes based on online dictionary learning. Extensive experiments on two industry-relevant assays and a synthetic MNIST example demonstrate that our method outperforms state-of-the-art deep archetypal analysis approaches.

In this paper we introduce DiQP; a novel Transformer-Diffusion model for restoring 8K video quality degraded by codec compression. To the best of our knowledge our model is the first to consider restoring the artifacts introduced by various codecs (AV1 HEVC) by Denoising Diffusion without considering additional noise. This approach allows us to model the complex non-Gaussian nature of compression artifacts effectively learning to reverse the degradation. Our architecture combines the power of Transformers to capture long-range dependencies with an enhanced windowed mechanism that preserves spatio-temporal context within groups of pixels across frames. To further enhance restoration the model incorporates auxiliary "Look Ahead" and "Look Around" modules providing both future and surrounding frame information to aid in reconstructing fine details and enhancing overall visual quality. Extensive experiments on different datasets demonstrate that our model outperforms state-of-the-art methods particularly for high-resolution videos such as 4K and 8K showcasing its effectiveness in restoring perceptually pleasing videos from highly compressed sources

The growing interest in generating recipes from food images has drawn substantial research attention in recent years. Existing works for recipe generation primarily utilize a two-stage training method--first predicting ingredients from a food image and then generating instructions from both the image and ingredients. Large Multi-modal Models (LMMs) which have achieved notable success across a variety of vision and language tasks shed light on generating both ingredients and instructions directly from images. Nevertheless LMMs still face the common issue of hallucinations during recipe generation leading to suboptimal performance. To tackle this issue we propose a retrieval augmented large multimodal model for recipe generation. We first introduce Stochastic Diversified Retrieval Augmentation (SDRA) to retrieve recipes semantically related to the image from an existing datastore as a supplement integrating them into the prompt to add diverse and rich context to the input image. Additionally Self-Consistency Ensemble Voting mechanism is proposed to determine the most confident prediction recipes as the final output. It calculates the consistency among generated recipe candidates which use different retrieval recipes as context for generation. Extensive experiments validate the effectiveness of our proposed method which demonstrates state-of-the-art (SOTA) performance in recipe generation on the Recipe1M dataset.

Low-rank adaptation (LoRA) has shifted the paradigm of adapting pre-trained Vision Transformers (ViT) achieving great efficiency by updating only a subset of tailored parameters to approximate weight updates. However the multi-head design of the self-attention mechanism with the heads working in parallel in the computation flow exhibiting similar visual patterns and requiring update over all of them incurs unnecessary storage and computational overhead. In this paper we propose Head-level responsiveness tuning for low-rank adaptation (Heart-LoRA). The proposed method explores redundancy among the heads and selectively activates task-responsive heads thus enabling fine-grained head-level tuning. Additionally given the different responsiveness of heads to diverse visual tasks our proposed method dynamically activates a subset of the approximated heads that are tailored to the current task. Experimental results show that Heart-LoRA yields superior performance over state-of-the-art PETL approaches on visual adaptation benchmark datasets.

Diffusion-based image generation models can enhance image quality when conditioned on ground truth labels. Here we conduct a comprehensive experimental study on image-level conditioning for diffusion models using cluster assignments. We investigate how individual clustering determinants such as the number of clusters and the clustering method impact image synthesis across three different datasets. Given the optimal number of clusters with respect to image synthesis we show that cluster-conditioning can achieve state-of-the-art performance with an FID of 1.67 for CIFAR10 and 2.17 for CIFAR100 along with a strong increase in training sample efficiency. We further propose a novel empirical method to estimate an upper bound for the optimal number of clusters. Unlike existing approaches we find no significant association between clustering performance and the corresponding cluster-conditional FID scores. Code is available at https://github.com/ HHU-MMBS/cedm-official-wavc2025

The advancement of vision-language models particularly the Contrastive Language-Image Pre-training (CLIP) model has revolutionized the field of machine learning by enabling robust zero-shot learning capabilities. These capabilities allow models to understand and respond to previously unseen data without task-specific training. However adapting CLIP to integrate specialized knowledge from various domains while retaining its zero-shot capabilities remains a significant challenge. To address this we introduce a novel prompt ensemble learning approach called Group-wise Prompt Ensemble (GPE). This method aims to enhance CLIP's zero-shot capabilities by incorporating new domain knowledge while improving its adaptability and robustness against data distribution shifts. Our approach hinges on three main strategies: prompt grouping with masked attention to optimize CLIP's adaptability while safeguarding its zero-shot capabilities; the incorporation of auxiliary prompts for the seamless integration of new domain insights without disrupting the original model's representation; and an ensemble learning strategy that effectively merges original and new knowledge. Through rigorous experimentation including more challenging cross-dataset transfer evaluations our GPE method redefines the benchmarks for the adaptability and efficiency of vision-language models surpassing existing models across various scenarios.

Bird's Eye View (BEV) semantic maps have recently garnered a lot of attention as a useful representation of the environment to tackle assisted and autonomous driving tasks. However most of the existing work focuses on the fully supervised setting training networks on large annotated datasets. In this work we present RendBEV a new method for the self-supervised training of BEV semantic segmentation networks leveraging differentiable volumetric rendering to receive supervision from semantic perspective views computed by a 2D semantic segmentation model. Our method enables zero-shot BEV semantic segmentation and already delivers competitive results in this challenging setting. When used as pretraining to then fine-tune on labeled BEV ground truth our method significantly boosts performance in low-annotation regimes and sets a new state of the art when fine-tuning on all available labels.

One-class anomaly detection aims to detect objects that do not belong to a predefined normal class. In practice training data lack those anomalous samples; hence state-of-the-art methods are trained to discriminate between normal and synthetically-generated pseudo-anomalous data. Most methods use data augmentation techniques on normal images to simulate anomalies. However the best-performing ones implicitly leverage a geometric bias present in the benchmarking datasets. This limits their usability in more general conditions. Others are relying on basic noising schemes that may be suboptimal in capturing the underlying structure of normal data. In addition most still favour the image domain to generate pseudo-anomalies training models end-to-end from only the normal class and overlooking richer representations of the information. To overcome these limitations we consider frozen yet rich feature spaces given by pretrained models and create pseudo-anomalous features with a novel adaptive linear feature perturbation technique. It adapts the noise distribution to each sample applies decaying linear perturbations to feature vectors and further guides the classification process using a contrastive learning objective. Experimental evaluation conducted on both standard and geometric bias-free datasets demonstrates the superiority of our approach with respect to comparable baselines. The codebase is accessible via our public repository.

Blood pressure (BP) monitoring is crucial for health assessment but existing contact-based methods face cost and comfort barriers. Remote photoplethysmography (rPPG) offers a promising contactless solution yet research is hampered by limited rPPG datasets with corresponding BP labels. This paper presents a transfer learning methodology for BP measurement. This approach involves utilizing a base dataset comprising signals produced by PPG (PPG-signals) to acquire knowledge that can be transferred to a target dataset containing signals generated by rPPG (rPPG-signals). In our study we trained diverse deep learning models using publicly available datasets containing PPG-signals. Subsequently these models were fine-tuned and evaluated using a public dataset that specifically consists of rPPG-signals. Additionally we explored the relationship between BP and heart rate and examined different loss functions and normalization approaches to optimize the performance of the deep learning models. The findings of our study demonstrate that our best model achieved a better performance than the state-of-the-art model with mean absolute error (MAE) of 8.721 (reduced by 4.879) mmHg and 8.653 (reduced by 1.647) mmHg for systolic blood pressure (SBP) and diastolic blood pressure (DBP) in a dataset with clinical settings showing promising potential for remote BP estimation.

By aligning paired image and caption embeddings as input contrastive vision-language representation learning has witnessed significant advances as illustrated by CLIP allowing visual encoders to learn from textual supervision and vice versa. Benefiting from millions of image-caption pairs collected from the Internet CLIP-like models show competitive performances against fully supervised baselines. However the learned visual representations are still undermined due to the binary constraint as most contrastive learning frameworks follow strict one-to-one correspondence for the input pairs of data and optimize the models using the InfoNCE loss function. The embeddings of the paired image-text are aligned while the unpaired image-text are pushed away from each other. In fact there are naturally many "false negatives" among these negative pairs since unpaired data can also have a high similarity. In this work we aim to overcome the impact of false negatives in vision-language representation learning by introducing soft targets for estimating the similarity between unpaired images and texts using external semantic knowledge structured in the form of graphs. The interest of such a method is demonstrated in the application context of medical imaging.

Image copy detection is the task of detecting edited copies of any image within a reference database. While previous approaches have shown remarkable progress the large size of their networks and descriptors remains a dis-advantage complicating their practical application. In this paper we propose a novel method that achieves competitive performance by using a lightweight network and compact descriptors. By utilizing relational self-supervised distillation to transfer knowledge from a large network to a small network we enable the training of lightweight networks with smaller descriptor sizes. We introduce relational self-supervised distillation for flexible representation in a smaller feature space and apply contrastive learning with a hard negative loss to prevent dimensional collapse. For the DISC2021 benchmark ResNet-50 and EfficientNet-B0 are used as the teacher and student models respectively with micro average precision improving by 5.0%/4.9%/5.9% for 64/128/256 descriptor sizes compared to the baseline method. The code is available at https://github.com/juntae9926/RDCD.

Generating human motion from textual descriptions is a challenging task. Existing methods either struggle with physical credibility or are limited by the complexities of physics simulations. In this paper we present ReinDiffuse that combines reinforcement learning with motion diffusion model to generate physically credible human motions that align with textual descriptions. Our method adapts Motion Diffusion Model to output a parameterized distribution of actions making them compatible with reinforcement learning paradigms. We employ reinforcement learning with the objective of maximizing physically plausible rewards to optimize motion generation for physical fidelity. Our approach outperforms existing state-of-the-art models on two major datasets HumanML3D and KIT-ML achieving significant improvements in physical plausibility and motion quality. Project: https://reindiffuse.github.io/

The rise of the generative models quality during the past years enabled the generation of edited variations of images at an important scale. To counter the harmful effects of such technology the Image Difference Captioning (IDC) task aims to describe the differences between two images. While this task is successfully handled for simple 3D rendered images it struggles on real-world images. The reason is twofold: the training data-scarcity and the difficulty to capture fine-grained differences between complex images. To address those issues we propose in this paper a simple yet effective framework to both adapt existing image captioning models to the IDC task and augment IDC datasets. We introduce BLIP2IDC an adaptation of BLIP2 to the IDC task at low computational cost and show it outperforms two-streams approaches by a significant margin on real-world IDC datasets. We also propose to use synthetic augmentation to improve the performance of IDC models in an agnostic fashion. We show that our synthetic augmentation strategy provides high quality data leading to a challenging new dataset well-suited for IDC named Syned. The code weights and dataset are available here https://github.com/gautierevn/BLIP2IDC.

Applying pseudo labeling techniques has been found to be advantageous in semi-supervised 3D object detection (SSOD) in Bird's-Eye-View (BEV) for autonomous driving particularly where labeled data is limited. In the literature Exponential Moving Average (EMA) has been used for adjustments of the weights of teacher network by the student network. However the same induces catastrophic forgetting in the teacher network. In this work we address this issue by introducing a novel concept of Reflective Teacher where the student is trained by both labeled and pseudo labeled data while its knowledge is progressively passed to the teacher through a regularizer to ensure retention of previous knowledge. Additionally we propose Geometry Aware BEV Fusion (GA-BEVFusion) for efficient alignment of multi-modal BEV features thus reducing the disparity between the modalities - camera and LiDAR. This helps to map the precise geometric information embedded among LiDAR points reliably with the spatial priors for extraction of semantic information from camera images. Our experiments on the nuScenes and Waymo datasets demonstrate: 1) improved performance over state-of-the-art methods in both fully supervised and semi-supervised settings; 2) Reflective Teacher achieves equivalent performance with only 25% and 22% of labeled data for nuScenes and Waymo datasets respectively in contrast to other fully supervised methods that utilize the full labeled dataset.

Over the past few years Text-to-Image (T2I) generation approaches based on diffusion models have gained significant attention. However vanilla diffusion models often suffer from spelling inaccuracies in the text displayed within the generated images. The capability to generate visual text is crucial offering both academic interest and a wide range of practical applications. To produce accurate visual text images state-of-the-art techniques adopt a glyph-controlled image generation approach consisting of a text layout generator followed by an image generator that is conditioned on the generated text layout. Nevertheless our study reveals that these models still face three primary challenges prompting us to develop a testbed to facilitate future research. We introduce a benchmark LenCom-Eval specifically designed for testing models capability in generating images with Lengthy and Complex visual text. Subsequently we introduce a training-free framework to enhance the two-stage generation approaches. We examine the effectiveness of our approach on both LenCom-Eval and MARIO-Eval benchmarks and demonstrate notable improvements across a range of evaluation metrics including CLIPScore OCR precision recall F1 score accuracy and edit distance scores. For instance our proposed framework improves the backbone model TextDiffuser by more than 23% and 13.5% in terms of OCR word F1 on LenCom-Eval and MARIO-Eval respectively. Our work makes a unique contribution to the field by focusing on generating images with long and rare text sequences a niche previously unexplored by existing literature.

Smartphones with multi-camera systems featuring cameras with varying field-of-views (FoVs) are increasingly common. This variation in FoVs results in content differences across videos paving the way for an innovative approach to video super-resolution (VSR). This method enhances the VSR performance of lower resolution (LR) videos by leveraging higher resolution reference (Ref) videos. Previous works which operate on this principle generally expand on traditional VSR models by combining LR and Ref inputs over time into a unified stream. However we can expect that better results are obtained by independently aggregating these Ref image sequences temporally. Therefore we introduce an improved method RefVSR++ which performs the parallel aggregation of LR and Ref images in the temporal direction aiming to optimize the use of the available data. RefVSR++ also incorporates improved mechanisms for aligning image features over time crucial for effective VSR. Our experiments demonstrate that RefVSR++ outperforms previous works by over 1dB in PSNR setting a new benchmark in the field.

Identifying AI-generated content is critical for the safe and ethical use of generative AI. Recent research has focused on developing detectors that generalize to unknown generators with popular methods relying either on high-level features or low-level fingerprints. However these methods have clear limitations: biased towards unseen content or vulnerable to common image degradations such as JPEG compression. To address these issues we propose a novel approach SFLD which incorporates PatchShuffle to integrate high-level semantic and low-level textural information. SFLD applies PatchShuffle at multiple levels improving robustness and generalization across various generative models. Additionally current benchmarks face challenges such as low image quality insufficient content preservation and limited class diversity. In response we introduce TwinSynths a new benchmark generation methodology that constructs visually near-identical pairs of real and synthetic images to ensure high quality and content preservation. Our extensive experiments and analysis show that SFLD outperforms existing methods on detecting a wide variety of fake images sourced from GANs diffusion models and TwinSynths demonstrating the state-of-the-art performance and generalization capabilities to novel generative models.

Classical approaches to Vanishing Point Detection (VPD) rely solely on the presence of explicit straight lines in images. Recent supervised deep learning methods rely on learned filters from labeled datasets. We propose an alternative unsupervised approach: Recurrence-based Vanishing Point Detection (R-VPD) that uses implicit lines derived from discovered recurring correspondences in addition to explicit lines. Furthermore we contribute two new VPD datasets: 1) a Synthetic Image dataset with 3200 ground truth vanishing points and 2) a Real-World Image dataset with 1400 human annotated vanishing points. We compare our method with two classical methods and two state-of-the-art deep learning-based VPD methods. We demonstrate that our unsupervised approach outperforms all the methods on the synthetic images dataset outperforms the classical methods and is on par with the supervised learning approaches on real-world images. Code and data can be found here: http://vision.cse.psu.edu/data/data.shtml

Human pose estimation (HPE) is crucial for various applications. However deploying HPE algorithms in surveillance contexts raises significant privacy concerns due to the potential leakage of sensitive personal information (SPI) such as facial features and ethnicity. Existing privacy-enhancing methods often compromise either privacy or performance or they require costly additional modalities. We propose a novel privacy-enhancing system that generates privacy-enhanced portraits while maintaining high HPE performance. Our key innovations include the reversible recovery of SPI for authorized personnel and the preservation of contextual information. By jointly optimizing a privacy-enhancing module a privacy recovery module and a pose estimator our system ensures robust privacy protection efficient SPI recovery and high-performance HPE. Experimental results demonstrate the system's robust performance in privacy enhancement SPI recovery and HPE. The code associated with this study can be found at this URL.