WACV 2025 论文列表

Identification of gender is critical not only for human-computer interaction but also for scrutinizing the search space in which an identity needs to be determined. Traditionally "real" facial images are employed for gender identification by computer vision algorithms. Due to the tremendous rise of privacy and advancement in generative networks synthetic face images are heavily developed and can be used for several face-related studies including gender classification. However their effectiveness compared to real images is still unexplored for gender classification. In response this study explores the effectiveness of gender classification networks trained on real and synthetic face images offering novel insights into the effectiveness of these two data distributions. For that we implemented several state-of-the-art gender classification architectures covering convolutional neural networks (CNNs) and vision transformers (ViT). Our research builds on the rigorous evaluation of 8 Deep Neural Networks (DNNs) across 4 diverse datasets and 6 types of image corruptions. To make the research interpretable we have also used several explainable mechanisms including Grad-CAM and t-SNE visualizations. In brief the impact of the proposed research is multifold: (i) understand the effectiveness of real vs. synthetic data distributions in network training and (ii) whether the synthetic models reflect the true physical world distribution to ensure that the models trained on them are resilient against image perturbations.

The bidirectional reflectance distribution function (BRDF) is an essential tool to capture the complex interaction of light and matter. Recently several works have employed neural methods for BRDF modeling following various strategies ranging from utilizing existing parametric models to purely neural parametrizations. While all methods yield impressive results a comprehensive comparison of the different approaches is missing in the literature. In this work we present a thorough evaluation of several approaches including results for qualitative and quantitative reconstruction quality and an analysis of reciprocity and energy conservation. Moreover we propose two extensions that can be added to existing approaches: A novel additive combination strategy for neural BRDFs that split the reflectance into a diffuse and a specular part and an input mapping that ensures reciprocity exactly by construction while previous approaches only ensure it by soft constraints.

Knowledge distillation (KD) remains challenging due to the opaque nature of the knowledge transfer process from a Teacher to a Student making it difficult to address certain issues related to KD. To address this we proposed UniCAM a novel gradient-based visual explanation method which effectively interprets the knowledge learned during KD. Our experimental results demonstrate that with the guidance of the Teacher's knowledge the Student model becomes more efficient learning more relevant features while discarding those that are not relevant. We refer to the features learned with the Teacher's guidance as distilled features and the features irrelevant to the task and ignored by the Student as residual features. Distilled features focus on key aspects of the input such as textures and parts of objects. In contrast residual features demonstrate more diffused attention often targeting irrelevant areas including the backgrounds of the target objects. In addition we proposed two novel metrics: the feature similarity score (FSS) and the relevance score (RS) which quantify the relevance of the distilled knowledge. Experiments on the CIFAR10 ASIRRA and Plant Disease datasets demonstrate that UniCAM and the two metrics offer valuable insights to explain the KD process.

Photorealistic reconstruction relying on 3D Gaussian Splatting has shown promising potential in various domains. However the current 3D Gaussian Splatting system only supports radiance field reconstruction using undistorted perspective images. In this paper we present OmniGS a novel omnidirectional Gaussian splatting system to take advantage of omnidirectional images for fast radiance field reconstruction. Specifically we conduct a theoretical analysis of spherical camera model derivatives in 3D Gaussian Splatting. According to the derivatives we then implement a new GPU-accelerated omnidirectional rasterizer that directly splats 3D Gaussians onto the equirectangular screen space for omnidirectional image rendering. We realize differentiable optimization of the omnidirectional radiance field without the requirement of cube-map rectification or tangent-plane approximation. Extensive experiments conducted in egocentric and roaming scenarios demonstrate that our method achieves state-of-the-art reconstruction quality and high rendering speed using omnidirectional images. The code will be publicly available at https://github.com/liquorleaf/OmniGS.

Depth estimation is the fundamental computer vision task for scene analysis. With the emergence of the deep learning era supervised monocular image depth estimation (MDE) became a popular choice for the task. Predominantly MDE methods utilize 360 images as ideal input due to their comprehensive field of view scene content compared to perspective images but they suffer from distortions in polar regions making it a more challenging ill-posed problem to date. Over the years methods using CNNs and/or large transformers taking 360 and/or projected perspective patch inputs have been proposed to solve the 360 MDE problem by formulating it as a regression or a classification task. Nevertheless their performance still suffers from global discrepancy inaccuracy poor details and generalizability. Lately diffusion-generating models have shown state-of-the-art performance in image synthesis that captures exceptionally rich knowledge of the visual world. However their ability to perform omnidirectional perception tasks is still unexplored. In this paper we explore a new approach called OmniDiffusion that reformulates the 360 MDE task as a diffusion denoising process. We present a diffusion-based framework to learn an iterative denoising process that denoises random depth distribution into the required depths. The diffusion process is performed in the latent space and uses the guidance of encoded RGB image visual as a condition. Furthermore to advance the image latent in a geometrically meaningful direction we leverage semantic segmentation and surface normal information to provide a more detailed contextual assistance to the denoising process. The performed experiments on the multiple real-world datasets show that our diffusion-denoising approach with the proposed conditions more appropriately refines depths outperforming the existing MDE and diffusion-based methods with state-of-the-art generalization ability while generating more accurate high-quality and detailed 360 depths.

3D semantic occupancy prediction is crucial for finely representing the surrounding environment which is essential for ensuring the safety in autonomous driving. Existing fusion-based occupancy methods typically involve performing a 2D-to-3D view transformation on image features followed by computationally intensive 3D operations to fuse these with LiDAR features leading to high computational costs and reduced accuracy. Moreover current research on occupancy prediction predominantly focuses on designing specific network architectures often tailored to particular models with limited attention given to the more fundamental aspect of semantic feature learning. This gap hinders the development of more transferable methods that could enhance the performance of various occupancy models. To address these challenges we propose OccLoff a framework that Learns to Optimize Feature Fusion for 3D occupancy prediction. Specifically we introduce a sparse fusion encoder with entropy masks that directly fuses 3D and 2D features improving model accuracy while reducing computational overhead. Additionally we propose a transferable proxy-based loss function and an adaptive hard sample weighting algorithm which enhance the performance of several state-of-the-art methods. Extensive evaluations on the nuScenes and SemanticKITTI benchmarks demonstrate the superiority of our framework and ablation studies confirm the effectiveness of each proposed module.

Semantic occupancy has recently gained significant traction as a prominent 3D scene representation. However most existing camera-based methods rely on large and costly datasets with fine-grained 3D voxel labels for training which limits their practicality and scalability. Furthermore approaches in this domain lack the modelling of scene dynamics. In this work we present a novel approach to occupancy estimation inspired by neural radiance field (NeRF) using supervision in 2D based on 3D labels provided by LiDAR that offers a more natural way of supervision than voxel labels. In particular we employ differentiable volumetric rendering to predict depth and semantic maps and train a 3D network based on supervision in 2D space only. To enhance geometric accuracy and increase the supervisory signal we introduce temporal rendering of adjacent time steps. Additionally we introduce occupancy flow as a mechanism to handle dynamic objects in the scene and ensure their temporal consistency. Through extensive experimentation we demonstrate that supervision in 2D with LiDAR can achieve state-of-the-art performance compared to methods using voxel labels and when combining it with voxel supervision in 3D temporal rendering and occupancy flow we outperform all previous occupancy estimation models significantly. We conclude that the proposed rendering supervision and occupancy flow advances occupancy estimation.

Self-supervised learning (SSL) has emerged as a promising technique for analyzing medical modalities such as X-rays due to its ability to learn without annotations. However conventional SSL methods face challenges in achieving semantic alignment and capturing subtle details which limits their ability to accurately represent the underlying anatomical structures and pathological features. To address these limitations we propose OTCXR a novel SSL framework that leverages optimal transport (OT) to learn dense semantic invariance. By integrating OT with our innovative Cross-Viewpoint Semantics Infusion Module (CV-SIM) OTCXR enhances the model's ability to capture not only local spatial features but also global contextual dependencies across different viewpoints. This approach enriches the effectiveness of SSL in the context of chest radiographs. Furthermore OTCXR incorporates variance and covariance regularizations within the OT framework to prioritize clinically relevant information while suppressing less informative features. This ensures that the learned representations are comprehensive and discriminative particularly beneficial for tasks such as thoracic disease diagnosis. We validate OTCXR's efficacy through comprehensive experiments on three publicly available chest X-ray datasets. Our empirical results demonstrate the superiority of OTCXR over state-of-the-art methods across all evaluated tasks confirming its capability to learn semantically rich representations.

Vision Transformers (ViTs) have demonstrated remarkable capabilities in learning representations but their performance is compromised when applied to unseen domains. Previous methods either engage in prompt learning during the training phase or modify model parameters at test time through entropy minimization. The former often overlooks unlabeled target data while the latter doesn't fully address domain shifts. In this work our approach Optimal Transport-guided Test-Time Visual Prompting (OT-VP) handles these problems by leveraging prompt learning at test time to align the target and source domains without accessing the training process or altering pre-trained model parameters. This method involves learning a universal visual prompt for the target domain by optimizing the Optimal Transport distance. With only four learned prompt tokens OT-VP exceeds state-of-the-art performance across three stylistic datasets--PACS VLCS OfficeHome and one corrupted dataset ImageNet-C. Additionally OT-VP operates efficiently both in terms of memory and computation and is adaptable for extension to online settings. The code is available at https://github.com/zybeich/OT-VP.

The objective of Radiology Report Generation (RRG) is to automatically generate coherent textual analyses of diseases based on radiological images thereby alleviating the workload of radiologists. Current AI-based methods for RRG primarily focus on modifications to the encoder-decoder model architecture. To advance these approaches this paper introduces an Organ-Regional Information Driven (ORID) framework which can effectively integrate multi-modal information and reduce the influence of noise from unrelated organs. Specifically based on the LLaVA-Med we first construct an RRG-related instruction dataset to improve organ-regional diagnosis description ability and get the LLaVA-Med-RRG. After that we propose an organ-based cross-modal fusion module to effectively combine the information from the organ-regional diagnosis description and radiology image. To further reduce the influence of noise from unrelated organs on the radiology report generation we introduce an organ importance coefficient analysis module which leverages Graph Neural Network (GNN) to examine the interconnections of the cross-modal information of each organ region. Extensive experiments and comparisons with state-of-the-art methods across various evaluation metrics demonstrate the superior performance of our proposed method.

Although facial landmark detection (FLD) has gained significant progress existing FLD methods still suffer from performance drops on partially non-visible faces such as faces with occlusions or under extreme lighting conditions or poses. To address this issue we introduce ORFormer a novel transformer-based method that can detect non-visible regions and recover their missing features from visible parts. Specifically ORFormer associates each image patch token with one additional learnable token called the messenger token. The messenger token aggregates features from all but its patch. This way the consensus between a patch and other patches can be assessed by referring to the similarity between its regular and messenger embeddings enabling non-visible region identification. Our method then recovers occluded patches with features aggregated by the messenger tokens. Leveraging the recovered features ORFormer compiles high-quality heatmaps for the downstream FLD task. Extensive experiments show that our method generates heatmaps resilient to partial occlusions. By integrating the resultant heatmaps into existing FLD methods our method performs favorably against the state of the arts on challenging datasets such as WFLW and COFW.

In the face of pressing environmental issues in the 21st century monitoring surface changes on Earth is more important than ever. Large-scale remote sensing such as satellite imagery is an important tool for this task. However using supervised methods to detect changes is difficult because of the lack of satellite data annotated with change labels especially for rare categories of change. Annotation proves challenging due to the sparse occurrence of changes in satellite images. Even within a vast collection of images only a small fraction may exhibit persistent changes of interest. To address this challenge we introduce OPTIMUS a self-supervised learning method based on an intuitive principle: if a model can recover information about the relative order of images in the time series then that implies that there are long-lasting changes in the images. OPTIMUS demonstrates this principle by using change point detection methods on model outputs in a time series. We demonstrate that OPTIMUS can directly detect interesting changes in satellite images achieving an improvement in AUROC score from 56.3% to 87.6% at distinguishing changed time series from unchanged ones compared to baselines. Our code and dataset are available at https://huggingface.co/datasets/optimus-change/optimus-dataset/.

Audio Description (AD) plays a pivotal role as an application system aimed at guaranteeing accessibility in multimedia content which provides additional narrations at suitable intervals to describe visual elements catering specifically to the needs of visually impaired audiences. In this paper we introduce CA3D the pioneering unified Context-Aware Automatic Audio Description system that provides AD event scripts with precise locations in the long cinematic content. Specifically CA3D system consists of: 1) a Temporal Feature Enhancement Module to efficiently capture longer term dependencies 2) an anchor-based AD event detector with feature suppression module that localizes the AD events and extracts discriminative feature for AD generation and 3) a self-refinement module that leverages the generated output to tweak AD event boundaries from coarse to fine. Unlike conventional methods which rely on metadata and ground truth AD timestamp for AD detection and generation tasks the proposed CA3D is the first end-to-end trainable system that only uses visual cue. Extensive experiments demonstrate that the proposed CA3D improves existing architectures for both AD event detection and script generation metrics establishing the new state-of-the-art performances in the AD automation.

In diffusion models deviations from a straight generative flow are a common issue resulting in semantic inconsistencies and suboptimal generations. To address this challenge we introduce `Non-Cross Diffusion' an innovative approach in generative modeling for learning ordinary differential equation (ODE) models. Our methodology strategically incorporates an ascending dimension of input to effectively connect points sampled from two distributions with uncrossed paths. This design is pivotal in ensuring enhanced semantic consistency throughout the inference process which is especially critical for applications reliant on consistent generative flows including various distillation methods and deterministic sampling which are fundamental in image editing and interpolation tasks. Our empirical results demonstrate the effectiveness of Non-Cross Diffusion showing a substantial improvements in semantic consistencies at various inference steps and enhancing the overall performance of diffusion models.

Supervised object detection requires annotated datasets for training and evaluation purposes. However human annotation of large datasets is error-prone and frequent mistakes are erroneous labels missing objects and imprecise bounding boxes. The main goals of this work are to quantify the extent of annotation noise in terms of corner-wise discrepancies assess how it impacts evaluation metrics for object detection and propose noise-aware alternatives that serve as upper and lower bounds for a baseline metric. We focus our analysis on the Microsoft COCO dataset and re-evaluate several state-of-the-art object detectors using the proposed metrics. We show that the Average Precision (AP) metric might be considerably over or under-estimated particularly for small objects and restrictive IoU acceptance thresholds. Our code is available at https://github.com/Artcs1/Error-Aware.

Object detection in art is a valuable tool for the digital humanities as it allows for faster identification of objects in artistic and historical images compared to humans. However annotating such images poses significant challenges due to the need for specialized domain expertise. We present NADA (no annotations for detection in art) a pipeline that leverages diffusion models' art-related knowledge for object detection in paintings without the need for full bounding box supervision. Our method which supports both weakly-supervised and zero-shot scenarios and does not require any fine-tuning of its pretrained components consists of a class proposer based on large vision-language models and a class-conditioned detector based on Stable Diffusion. NADA is evaluated on two artwork datasets ArtDL 2.0 and IconArt outperforming prior work in weakly-supervised detection while being the first work for zero-shot object detection in art. Code is available at https://github.com/patrick-john-ramos/nada

Spatio-Temporal Video Grounding (STVG) - the task of identifying the target object in the field-of-view that the language instruction refers to - is a fundamental vision-language task. Current STVG approaches typically utilize feeds from an RGB camera that is assumed to be always-on and process the video frames using complex neural network pipelines. As a result they often impose prohibitive system overheads (energy latency) on pervasive devices. To address this we propose NeuroViG with two key innovations: (a) leveraging on event streams from a low-power neuromorphic event camera sensor to perform selective triggering of the more energy-hungry RGB camera for STVG and (b) augmenting the STVG model with a lightweight Adaptive Frame Selector (AFS) that bypasses complex transformer-based operations for a majority of video frames thereby enabling its execution on a pervasive Jetson AGX device. We have also introduced modifications to the neural network processing pipeline such that the system can offer tunable tradeoffs between accuracy and energy/latency. Our proposed NeuroViG system allows us to reduce the STVG energy overhead and latency by 4x and 3.8x respectively for less than 1% loss in accuracy.

Among various active 3D measurement techniques Structured Light (SL) is one of the most popular method for its robustness and high accuracy. Ordinary SL system consists of a camera and a projector and by projecting a pre-defined pattern we can obtain pixel-to-pixel correspondences between the camera and the projector for triangulation. However if we lack the knowledge of the projected pattern for some reason e.g. the projected pattern is not as expected due to lens distortion inaccurate calibration undesired optical phenomenon like inter-reflection and so on the accuracy of conventional SL is severally degraded. As a remedy we propose unsupervised structured light (USSL) which does not explicitly use prior knowledge on the pattern. Inspired by the fact that human can recognize the scene structure illuminated by unknown light source (e.g. rotating mirror ball) and some prior work have succeeded in novel-view-synthesis under unknown illumination conditions we implement USSL on Neural Signed Distance Fields (Neural SDF) pipeline with implicit reflection module powered by neural network. Additionally since every SL method causes occlusion (shadow) by pattern projection we must take it into account for accurate shape reconstruction. To this end we integrate shadow volume rendering into the proposed pipeline. Experiments with synthetic and real dataset are conducted to confirm the feasibility of the proposed method.

Neural field-based SLAM methods typically employ a single monolithic field as their scene representation. This prevents efficient incorporation of loop closure constraints and limits scalability. To address these shortcomings we propose a novel RGB-D neural mapping framework in which the scene is represented by a collection of lightweight neural fields which are dynamically anchored to the pose graph of a sparse visual SLAM system. Our approach shows the ability to integrate large-scale loop closures while requiring only minimal reintegration. Furthermore we verify the scalability of our approach by demonstrating successful building-scale mapping taking multiple loop closures into account during the optimization and show that our method outperforms existing state-of-the-art approaches on large scenes in terms of quality and runtime. Our code is available open-source at https://github.com/KTH-RPL/neural_graph_mapping.