WACV 2025 论文列表

Positron Emission Tomography (PET) and Computed Tomography (CT) are essential for diagnosing staging and monitoring various diseases particularly cancer. Despite their importance the use of PET/CT systems is limited by the necessity for radioactive materials the scarcity of PET scanners and the high cost associated with PET imaging. In contrast CT scanners are more widely available and significantly less expensive. In response to these challenges our study addresses the issue of generating PET images from CT images aiming to reduce both the medical examination cost and the associated health risks for patients. Our contributions are twofold: First we introduce a conditional diffusion model named CPDM which to our knowledge is one of the initial attempts to employ a diffusion model for translating from CT to PET images. Second we provide the largest CT-PET dataset to date comprising 2028628 paired CT-PET images which facilitates the training and evaluation of CT-to-PET translation models. For the CPDM model we incorporate domain knowledge to develop two conditional maps: the Attention map and the Attenuation map. The former helps the diffusion process focus on areas of interest while the latter improves PET data correction and ensures accurate diagnostic information. Experimental evaluations across various benchmarks demonstrate that CPDM surpasses existing methods in generating high-quality PET images in terms of multiple metrics. The source code and data samples are available at https://github.com/thanhhff/CPDM.

For designing a wide range of everyday objects the design process should be aware of both the human body and the underlying semantics of the design specification. However these two objectives present significant challenges to the current AI-based designing tools. In this work we present a method to synthesize body-aware 3D objects from a base mesh given an input body geometry and either text or image as guidance. The generated objects can be simulated on virtual characters or fabricated for real-world use. We propose to use a mesh deformation procedure that optimizes for both semantic alignment as well as contact and penetration losses. Using our method users can generate both virtual or real-world objects from text image or sketch without the need for manual artist intervention. We present both qualitative and quantitative results on various object categories demonstrating the effectiveness of our approach.

Out-of-distribution (OOD) detection remains a key challenge preventing the rollout of key AI technologies like autonomous vehicles into the mainstream as classifiers trained on in-distribution (ID) data are unable to gracefully handle OOD data. While OOD detection remains an active area of research, current post-hoc methods often suffer from limited separability between ID and OOD, and outlier exposure-based methods lack generalisation to unseen outlier types. We present CRAFT, a fine-tuning approach for arming pre-trained classifiers against OOD inputs without requiring access to outliers. The key insight that underpins our approach is that during pre-training, classifiers implicitly learn a ranking across the ID classes that is not respected by OOD data. Therefore, a form of fine-tuning without outliers of a pre-trained classifier can sharpen the rank order of the classes, making them sensitive to the presence of OOD data. Furthermore, the fine-tuned model does not impact the ability of the classifier to correctly classify ID inputs to their respective classes. Experiments on CIFAR-10, CIFAR-100, and ImageNet-200 demonstrate that CRAFT outperforms 33 existing methods, particularly in the more challenging near-OOD detection, as well as in overall OOD detection consistency and ID classification accuracy.

In annotating real-life large noisy and domain-specific images for digitising infrastructure substantial human effort persists despite past advancements. This research provides practical and interpretable scores for human annotators enabling flexible annotation strategies improving automation and reducing the effort required to create and correct image labels. The authors present the CRAAC solution: Consistency Regularised Active learning and Automatic Corrections which builds on Mask R-CNN with three additional modules: consistency regularisation scoring modules for active learning and automatic corrections. Experiments on our pavement image dataset recorded with a low silhouette score of 0.146 and qualitative annotation inconsistencies reduce the human effort of mouse clicks by 5-11% and improve the quality metrics of mAP and AR by approx. 40% from the original Mask R-CNN. The automatic correction further reduces the performance variation.

Automated waste recycling aims to efficiently separate the recyclable objects from the waste by employing vision-based systems. However the presence of varying shaped objects having different material types makes it a challenging problem especially in cluttered environments. Existing segmentation methods perform reasonably on many semantic segmentation datasets by employing multi-contextual representations however their performance is degraded when utilized for waste object segmentation in cluttered scenarios. In addition plastic objects further increase the complexity of the problem due to their translucent nature. To address these limitations we introduce an efficacious segmentation network named COSNet that uses boundary cues along with multi-contextual information to accurately segment the objects in cluttered scenes. COSNet introduces novel components including feature sharpening block (FSB) and boundary enhancement module (BEM) for enhancing the features and highlighting the boundary information of irregular waste objects in cluttered environment. Extensive experiments on three challenging datasets including ZeroWaste-f SpectralWaste and ADE20K demonstrate the effectiveness of the proposed method. Our COSNet achieves a significant gain of 1.8% on ZeroWaste-f and 2.1% on SpectralWaste datasets respectively in terms of mIoU metric.

Challenges in cross-learning involve inhomogeneous or even inadequate amount of training data and lack of resources for retraining large pretrained models. Inspired by transfer learning techniques in NLP adapters and prefix tuning this paper presents a new model-agnostic plugin architecture for cross-learning called CM3T that allows transformer-based models to be able adapt to new or missing information. We introduce two adapter blocks: multi-head vision adapters for transfer learning and cross-attention adapters for multimodal learning. Training becomes substantially efficient as the backbone and other plugins do not need to be finetuned along with these additions. Experiments and ablation studies on three datasets - Epic-Kitchens-100 MPIIGroupInteraction and UDIVA v0.5 - show the efficacy of this framework on different recording settings and tasks. With only 12.8% trainable parameters compared to the backbone to process video input and 22.3% trainable parameters for two additional modalities we achieve comparable and even better results than the state-of-the-art. Compared to similar methods our work achieves this result without any specific requirements for training or pretraining and is a step towards bridging the gap between a general model and specific practical applications in the field of video classification.

Pedestrian Attribute Recognition (PAR) serves as a fundamental task in computer vision and is crucial for upgrading security systems. It helps in precisely identifying and characterizing various attributes of pedestrians. However current PAR datasets have certain issues in representing a wide range of attributes correctly which makes the existing PAR methods less effective in real-world scenarios. Addressing this limitation this paper introduces PEARL a comprehensive dataset comprising of diverse pedestrian images annotated with 146 attributes. These samples have been sourced from surveillance videos across twelve countries. This paper also formulates an image-based PAR using language-image fusion strategy and utilizes CLIP as a new evaluation baseline. Specifically we leverage textual information by transforming sets of attributes into meaningful sentences. Addressing the inherent data imbalance in PAR we provide three types of prompt settings to optimize the training of the CLIP model. Our evaluation encompasses a thorough assessment of the proposed baseline model across various datasets including PEARL dataset as well as established PAR benchmarks such as PA100K RAP and PETA.

Detection of out-of-distribution (OOD) samples is crucial for safe real-world deployment of machine learning models. Recent advances in vision language foundation models have made them capable of detecting OOD samples without requiring in-distribution (ID) images. However these zero-shot methods often underperform as they do not adequately consider ID class likelihoods in their detection confidence scoring. Hence we introduce CLIPScope a zero-shot OOD detection approach that normalizes the confidence score of a sample by class likelihoods akin to a Bayesian posterior update. Furthermore CLIPScope incorporates a novel strategy to mine OOD classes from a large lexical database. It selects class labels that are farthest and nearest to ID classes in terms of CLIP embedding distance to maximize coverage of OOD samples. We conduct extensive ablation studies and empirical evaluations demonstrating state of the art performance of CLIPScope across various OOD detection benchmarks.

Pre-trained vision-language models (VLMs) exemplified by CLIP demonstrate remarkable adaptability across zero-shot classification tasks without additional training. However their performance diminishes in the presence of domain shifts. In this study we introduce CLIP Adaptation duRing Test-Time (CLIPArTT) a fully test-time adaptation (TTA) approach for CLIP which involves automatic text prompts construction during inference for their use as text supervision. Our method employs a unique minimally invasive text prompt tuning process wherein multiple predicted classes are aggregated into a single new text prompt used as pseudo label to re-classify inputs in a transductive manner. Additionally we pioneer the standardization of TTA benchmarks (e.g. TENT) in the realm of VLMs. Our findings demonstrate that without requiring additional transformations nor new trainable modules CLIPArTT enhances performance dynamically across non-corrupted datasets such as CIFAR-100 corrupted datasets like CIFAR-100-C and ImageNet-C alongside synthetic datasets such as VisDA-C. This research underscores the potential for improving VLMs' adaptability through novel test-time strategies offering insights for robust performance across varied datasets and environments. The code can be found at: https://github.com/dosowiechi/CLIPArTT.git

Video frame interpolation is an ill-posed problem and a wide variety of methods have been proposed ranging from more traditional computer vision strategies to the most recent developments with neural network models. While there are many methods to interpolate video frames quality assessment regarding the resulting artifacts from these methods remains dependent on off-the-shelf methods. Although these methods can make accurate quality predictions for many visual artifacts such as compression blurring and banding their performance is mediocre for video frame interpolation artifacts due to the unique spatio-temporal qualities of such artifacts. To address this we aim to leverage semantic feature extraction capabilities of the pre-trained visual backbone of CLIP. Specifically we adapt its multi-scale approach to our feature extraction network and combine it with the spatio-temporal attention mechanism of the Video Swin Transformer. This allows our model to detect interpolation-related artifacts across frames and predict the relevant differential mean opinion score. Our model outperforms existing state-of-the-art quality metrics for assessing the quality of interpolated frames in both full-reference and no-reference settings.

An important aspect of deploying face recognition (FR) algorithms in real-world applications is their ability to learn new face identities from a continuous data stream. However the online training of existing deep neural network-based FR algorithms which are pre-trained offline on large-scale stationary datasets encounter two major challenges: 1) catastrophic forgetting of previously learned identities and 2) the need to store past data for complete retraining from scratch leading to significant storage constraints and privacy concerns. In this paper we introduce CLFace a continual learning framework designed to preserve and incrementally extend the learned knowledge. CLFace eliminates the classification layer resulting in a resource-efficient FR model that remains fixed throughout lifelong learning and provides label-free supervision to a student model making it suitable for open-set face recognition during incremental steps. We introduce an objective function that employs feature-level distillation to reduce drift between feature maps of the student and teacher models across multiple stages. Additionally it incorporates a geometry-preserving distillation scheme to maintain the orientation of the teacher model's feature embedding. Furthermore a contrastive knowledge distillation is incorporated to continually enhance the discriminative power of the feature representation by matching similarities between new identities. Experiments on several benchmark FR datasets demonstrate that CLFace outperforms baseline approaches and state-of-the-art methods on unseen identities using both in-domain and out-of-domain datasets.

We propose CLASS; a novel unified model for the synthesis and search for design layouts two tasks that are often handled separately by prior works. We propose to learn a compact and coherent latent feature of a layout supporting joint search and synthesis. This allows various operations such style-conditioned layout generation latent space manipulation and provides seamless integration of search and synthesis for an effective design workflow. We train CLASS with a dual decoder: a new transformer-based layout-conditioned decoder and a CNN-based raster decoder. The latent-conditioned decoder explicitly conditions upon a latent vector while generating a layout in an auto-regressive fashion. We train CLASS under variational framework which in conjunction with a raster-decoder enhances the latent representation improving both generation and retrieval performances. We show the effectiveness of CLASS on the RICO and PubLayNet benchmarks and demonstrate that CLASS is capable of high-quality synthesis from scratch as well as performing self-completion interpolation project between design layouts whilst achieving close to or better than state-of-the-art search performance.

Visual Question Answering (VQA) systems witnessed a significant advance in recent years due to the development of large-scale Vision-Language Pre-trained Models (VLPMs). As the application scenario and user demand change over time an advanced VQA system is expected to be capable of continuously expanding its knowledge and capabilities over time not only to handle new tasks (i.e. new question types or visual scenes) but also to answer questions in new specialized domains without forgetting previously acquired knowledge and skills. Existing works studying CL on VQA tasks primarily consider answer- and question-type incremental learning or scene- and function-incremental learning whereas how VQA systems perform when they encounter new domains and increasing user demands has not been studied. Motivated by this we introduce CL-CrossVQA a rigorous Continual Learning benchmark for Cross-domain Visual Question Answering through which we conduct extensive experiments on 4 VLPMs 5 CL approaches and 5 VQA datasets from different domains. In addition by probing the forgetting phenomenon of the intermediate layers we provide insights into how model architecture affects CL performance why CL approaches can help mitigate forgetting in VLPMs and how to design CL approaches suitable for VLPMs in this challenging continual learning environment. To facilitate future work on developing an advanced All-in-One VQA system we will release our datasets and code.

Reproducibility and replicability are critical pillars of empirical research particularly in machine learning where they depend not only on the availability of models but also on the datasets used to train and evaluate those models. In this paper we introduce the Construction Industry Steel Ordering List (CISOL) dataset which was developed with a focus on transparency to ensure reproducibility replicability and extensibility. CISOL provides a valuable new research resource and highlights the importance of having diverse datasets even in niche application domains such as table extraction in civil engineering. CISOL is unique in that it contains real-world civil engineering documents from industry making it a distinctive contribution to the field. The dataset contains more than 120000 annotated instances in over 800 document images positioning it as a medium-sized dataset that provides a robust foundation for Table Structure Recognition (TSR) and Table Detection (TD) tasks. Benchmarking results show that CISOL achieves 67.22 mAP@0.5:0.95:0.05 using the YOLOv8 model outperforming the TSR-specific TATR model. This highlights the effectiveness of CISOL as a benchmark for advancing TSR especially in specialized domains.

Camouflaged object detection (COD) the task of identifying objects concealed within their surroundings is often quite challenging due to the similarity that exists between the foreground and background. By incorporating an additional referring image where the target object is clearly visible we can leverage the similarities between the two images to detect the camouflaged object. In this paper we propose a novel problem setup: referring camouflaged object discovery (RCOD). In RCOD segmentation occurs only when the object in the referring image is also present in the camouflaged image; otherwise a blank mask is returned. This setup is particularly valuable when searching for specific camouflaged objects. Current COD methods are often generic leading to numerous false positives in applications focused on specific objects. To address this we introduce a new framework called Co-Saliency Inspired Referring Camouflaged Object Discovery (CIRCOD). Our approach consists of two main components: Co-Saliency-Aware Image Transformation (CAIT) and Co-Salient Object Discovery (CSOD). The CAIT module reduces the appearance and structural variations between the camouflaged and referring images while the CSOD module utilizes the similarities between them to segment the camouflaged object provided the images are semantically similar. Covering all semantic categories in current COD benchmark datasets we collected over 1000 referring images to validate our approach. Our extensive experiments demonstrate the effectiveness of our method and show that it achieves superior results compared to existing methods. Code is available at https://github.com/avigupta2798/CIRCOD/

Multiple Instance Learning (MIL) has shown potential for analyzing Whole Slide Images (WSIs) in digital pathology but it faces challenges related to redundant information learning and generalization due to limited supervision and the computational complexity of Gigapixel WSIs. Many MIL-based methods apply a small weight matrix to all WSI patches. In this study we focus on developing computationally efficient models that improve MIL-based WSI classification by processing fewer patches while improving performance. We propose an attention-based approach using knowledge distillation where a compute-intensive "instructor" model analyzes all WSI patches to train a resource-efficient "learner" model which considers only a subset of patches. Comprehensive evaluations on four cancer subtype datasets--TCGA-BRCA TCGA-NSCLC TCGA-RCC and PANDA--demonstrate that an "observe-everything" instructor can effectively train an "observe-minimally" learner network. Overall our proposed learner network enhances performance by 4% compared to the state-of-the-art while reducing inference time by 45% and FLOPs by approximately 88%.

Unmanned underwater image analysis for marine monitoring faces two key challenges: (i) degraded image quality due to light attenuation and (ii) hardware storage constraints limiting high-resolution image collection. Existing methods primarily address image enhancement with approaches that hinge on storing the full-size input. In contrast we introduce the Capsule Enhanced Variational AutoEncoder (CE-VAE) a novel architecture designed to efficiently compress and enhance degraded underwater images. Our attention-aware image encoder can project the input image onto a latent space representation while being able to run online on a remote device. The only information that needs to be stored on the device or sent to a beacon is a compressed representation. There is a dual-decoder module that performs offline full-size enhanced image generation. One branch reconstructs spatial details from the compressed latent space while the second branch utilizes a capsule-clustering layer to capture entity-level structures and complex spatial relationships. This parallel decoding strategy enables the model to balance fine-detail preservation with context-aware enhancements. CE-VAE achieves state-of-the-art performance in underwater image enhancement on six benchmark datasets providing up to 3x higher compression efficiency than existing approaches. Code available at https://github.com/iN1k1/ce-vae-underwater-image-enhancement.

Capturing multi-scale context within feature maps is crucial for semantic segmentation. With the success of the Vision Transformer (ViT) recent models have been designed with transformer decoders to capture it. However these models face limitations in utilizing diverse contextual information due to the inherent nature of the attention mechanism and structural constraints. Typically multi-head attention which leads to similar receptive fields for each token feture is achieved at the expense of significantly increased computational cost. The nature of the structure can cause inconsistent combination of the information across different levels. To address this issue in this paper we propose a novel and effective decoding scheme CCASeg which is based on convolutional cross-attention (CCA). The proposed CCA along with the decoding structure is devised not only to capture both local and global context through convolutional kernels of various sizes but also to achieve high efficiency by effective utilization of the cheap convolution operations. Moreover the decoding structure which ensures the successive combination of information across various levels facilitates understanding of diverse contexts. Consequently this novel decoding scheme enables feature maps to effectively learn the relationships between objects of different sizes. In this way our proposed CCASeg outperforms previous state-of-the-art methods on popular semantic segmentation benchmarks including ADE20K Cityscapes COCO-stuff and iSAID.

Foundation Models (FMs) have been successful in various computer vision tasks like image classification object detection and image segmentation. However these tasks remain challenging when these models are tested on datasets with different distributions from the training dataset a problem known as domain shift. This is especially problematic for recognizing animal species in camera-trap images where we have variability in factors like lighting camoulage and occlusions. In this paper we propose the Camera Trap Language-guided Contrastive Learning (CATALOG) model to address these issues. Our approach combines multiple FMs to extract visual and textual features from camera-trap data and uses a contrastive loss function to train the model. We evaluate CATALOG on two benchmark datasets and show that it outperforms previous state-of-the-art methods in camera-trap image recognition especially when the training and testing data have different animal species or come from different geographical areas. Our approach demonstrates the potential of using FMs in combination with multi-modal fusion and contrastive learning for addressing domain shifts in camera-trap image recognition. The code of CATALOG is publicly available at https://github.com/Julian075/CATALOG.