WACV 2025 Papers

Document images are often degraded by various stains significantly impacting their readability and hindering downstream applications such as document digitization and analysis. The absence of a comprehensive stained document dataset has limited the effectiveness of existing document enhancement methods in removing stains while preserving fine-grained details. To address this challenge we construct StainDoc the first large-scale high-resolution (2145x2245) dataset specifically designed for document stain removal. StainDoc comprises over 5000 pairs of stained and clean document images across multiple scenes. This dataset encompasses a diverse range of stain types severities and document backgrounds facilitating robust training and evaluation of document stain removal algorithms. Furthermore we propose StainRestorer a Transformer-based document stain removal approach. StainRestorer employs a memory-augmented Transformer architecture that captures hierarchical stain representations at part instance and semantic levels via the DocMemory module. The Stain Removal Transformer (SRTransformer) leverages these feature representations through a dual attention mechanism: an enhanced spatial attention with an expanded receptive field and a channel attention captures channel-wise feature importance. This combination enables precise stain removal while preserving document content integrity. Extensive experiments demonstrate StainRestorer's superior performance over state-of-the-art methods on the StainDoc dataset and its variants StainDoc_Mark and StainDoc_Seal establishing a new benchmark for document stain removal. Our work highlights the potential of memory-augmented Transformers for this task and contributes a valuable dataset to advance future research.

Accurate trajectory forecasting is crucial for the performance of various systems such as advanced driver-assistance systems and self-driving vehicles. These forecasts allow us to anticipate events that lead to collisions and therefore to mitigate them. Deep Neural Networks have excelled in motion forecasting but overconfidence and weak uncertainty quantification persist. Deep Ensembles address these concerns yet applying them to multimodal distributions remains challenging. In this paper we propose a novel approach named Hierarchical Light Transformer Ensembles (HLT-Ens) aimed at efficiently training an ensemble of Transformer architectures using a novel hierarchical loss function. HLT-Ens leverages grouped fully connected layers inspired by grouped convolution techniques to capture multimodal distributions effectively. We demonstrate that HLT-Ens achieves state-of-the-art performance levels through extensive experimentation offering a promising avenue for improving trajectory forecasting techniques. We make our code available at github.com/alafage/hlt-ens.

Despite the latest remarkable advances in generative modeling efficient generation of high-quality 3D objects from textual prompts remains a difficult task. A key challenge lies in data scarcity: the most extensive 3D datasets encompass merely millions of samples while their 2D counterparts contain billions of text-image pairs. To address this we propose a novel approach which harnesses the power of large pretrained 2D diffusion models. More specifically our approach HexaGen3D fine-tunes a pretrained text-to-image model to jointly predict 6 orthographic projections and the corresponding 3D latent. We then decode these latents to generate a textured mesh. HexaGen3D does not require per-sample optimization and can infer high-quality and diverse objects from textual prompts in 7 seconds offering significantly better quality-to-latency trade-offs than existing approaches. Furthermore HexaGen3D demonstrates strong generalization to new objects or compositions.

Unsupervised anomaly detection (AD) is a critical task in various domains from manufacturing to infrastructure monitoring. To advance this field we introduce two novel datasets: CARS-AD and ROADS-AD designed to challenge existing unsupervised AD methods with their diverse and heterogeneous image content. CARS-AD comprises real images of cars with various defects while ROADS-AD contains images of roads from multiple countries each presenting unique challenges in anomaly detection. These datasets provide ground truth pixel-wise masks and image-level ground truth labels enabling detailed evaluation and benchmarking of AD algorithms. We evaluate state-of-the-art unsupervised AD methods on both datasets using the AUROC metric to assess detection and localization performance. Our results reveal significant room for improvement underscoring the complexity of the datasets and the need for robust AD techniques. Notably Csflow and U-Flow demonstrate superior performance on the CARS-AD Dataset leveraging their ability to process multi-scale features effectively. Conversely Reverse Distillation excels in anomaly localization on the ROADS-AD Dataset showcasing the importance of nuanced approaches for diverse anomaly types. Our findings underscore the importance of addressing the challenges posed by heterogeneous datasets in unsupervised AD. We hope that the introduction of CARS-AD and ROADS-AD will inspire further research in this field driving the development of innovative AD methods capable of handling real-world anomalies with greater accuracy and reliability. CARS-AD Dataset and ROADS-AD Dataset are publicly available at https://github.com/juanb09111/heterogeneousAD.

Recent advances in high-definition (HD) map construction from surround-view images have highlighted their cost-effectiveness in deployment. However prevailing techniques often fall short in accurately extracting and utilizing road features as well as in the implementation of view transformation. In response we introduce HeightMapNet a novel framework that establishes a dynamic relationship between image features and road surface height distributions. By integrating height priors our approach refines the accuracy of Bird's-Eye-View (BEV) features beyond conventional methods. HeightMapNet also introduces a foreground-background separation network that sharply distinguishes between critical road elements and extraneous background components enabling precise focus on detailed road micro-features. Additionally our method leverages multi-scale features within the BEV space optimally utilizing spatial geometric information to boost model performance. HeightMapNet has shown exceptional results on the challenging nuScenes and Argoverse 2 datasets outperforming several widely recognized approaches. The code will be available at https://github.com/adasfag/HeightMapNet/.

Accurate 3D lane detection from monocular images presents significant challenges due to depth ambiguity and imperfect ground modeling. Previous attempts to model the ground have often used a planar ground assumption with limited degrees of freedom making them unsuitable for complex road environments with varying slopes. Our study introduces HeightLane an innovative method that predicts a height map from monocular images by creating anchors based on a multi-slope assumption. This approach provides a detailed and accurate representation of the ground. HeightLane employs the predicted heightmap along with a deformable attention-based spatial feature transform framework to efficiently convert 2D image features into 3D bird's eye view (BEV) features enhancing spatial understanding and lane structure recognition. Additionally the heightmap is used for the positional encoding of BEV features further improving their spatial accuracy. This explicit view transformation bridges the gap between front-view perceptions and spatially accurate BEV representations significantly improving detection performance. To address the lack of the necessary ground truth height map in the original OpenLane dataset we leverage the Waymo dataset and accumulate its LiDAR data to generate a height map for the drivable area of each scene. The GT heightmaps are used to train the heightmap extraction module from monocular images. Extensive experiments on the OpenLane validation set show that HeightLane achieves state-of-the-art performance in terms of F-score highlighting its potential in real-world applications.

Detection Transformers (DETR) have recently set new benchmarks in object detection. However their performance in detecting rotated objects lags behind established oriented object detectors. Our analysis identifies a key observation: the boundary discontinuity and square-like problem in bipartite matching poses an issue with assigning appropriate ground truths to predictions leading to du plicate low-confidence predictions. To address this we introduce a Hausdorff distance-based cost for bipartite matching which more accurately quantifies the discrepancy between predictions and ground truths. Additionally we find that a static denoising approach impedes the training of rotated DETR especially as the quality of the detector's predictions begins to exceed that of the noised ground truths. To overcome this we propose an adaptive query denoising method that employs bipartite matching to selectively eliminate noised queries that detract from model improvement. When compared to models adopting a ResNet-50 backbone our proposed model yields remarkable improvements achieving +4.18 AP50 +4.59 AP50 and +4.99 AP50 on DOTA-v2.0 DOTA-v1.5 and DIOR-R respectively.

Creating images where surface patterns of one object - such as cracks holes or grooves - are precisely transferred onto objects made of different materials remains a challenging task in computer graphics. For example recreating the exact pattern of wood grain cracks on a metallic surface while maintaining the realistic metallic texture requires sophisticated technical solutions. In this study we introduce Harmonizing Attention a new method that can automatically extract these surface patterns from photographs and recreate them with different materials while preserving natural-looking textures. Our approach achieves this through a novel attention mechanism that can process multiple reference images simultaneously without requiring additional training. This makes the method both practical and efficient for real-world applications opening up new possibilities in augmented reality image editing and beyond.

Generative text-to-image models such as Stable Diffusion have demonstrated a remarkable ability to generate diverse high-quality images. However they are surprisingly inept when it comes to rendering human hands which are often anatomically incorrect or reside in the "uncanny valley". In this paper we propose a method HandCraft for restoring such malformed hands. This is achieved by automatically constructing masks and depth images for hands as conditioning signals using a parametric model allowing a diffusion-based image editor to fix the hand's anatomy and adjust its pose while seamlessly integrating the changes into the original image preserving pose color and style. Our plug-and-play hand restoration solution is compatible with existing pretrained diffusion models and the restoration process facilitates adoption by eschewing any fine-tuning or training requirements for the diffusion models. We also contribute MalHand datasets that contain generated images with a wide variety of malformed hands in several styles for hand detector training and hand restoration benchmarking and demonstrate through qualitative and quantitative evaluation that HandCraft not only restores anatomical correctness but also maintains the integrity of the overall image.

Autonomous driving has garnered significant attention in recent research and Bird's-Eye-View (BEV) map segmentation plays a vital role in the field providing the basis for safe and reliable operation. While data augmentation is a commonly used technique for improving BEV map segmentation networks existing approaches predominantly focus on manipulating spatial domain representations. In this work we investigate the potential of frequency domain data augmentation for camera-based BEV map segmentation. We observe that high-frequency information in camera images is particularly crucial for accurate segmentation. Based on this insight we propose High-frequency Shuffle Data Augmentation (HSDA) a novel data augmentation strategy that enhances a network's ability to interpret high-frequency image content. This approach encourages the network to distinguish relevant high-frequency information from noise leading to improved segmentation results for small and intricate image regions as well as sharper edge and detail perception. Evaluated on the nuScenes dataset our method demonstrates broad applicability across various BEV map segmentation networks achieving a new state-of-the-art mean Intersection over Union (mIoU) of 61.3% for camera-only systems. This significant improvement underscores the potential of frequency domain data augmentation for advancing the field of autonomous driving perception. Code has been released: https://github.com/Zarhult/HSDA

Compositional Zero-Shot Learning (CZSL) has emerged as an essential paradigm in machine learning aiming to overcome the constraints of traditional zero-shot learning by incorporating compositional thinking into its methodology. Conventional zero-shot learning has difficulty managing unfamiliar combinations of seen and unseen classes because it depends on pre-defined class embeddings. In contrast Compositional Zero-Shot Learning leverages the inherent hierarchies and structural connections among classes creating new class representations by combining attributes components or other semantic elements. In our paper we propose a novel framework that for the first time combines the Modern Hopfield Network with a Mixture of Experts (HOPE) to classify the compositions of previously unseen objects. Specifically the Modern Hopfield Network creates a memory that stores label prototypes and identifies relevant labels for a given input image. Subsequently the Mixture of Expert models integrates the image with the appropriate prototype to produce the final composition classification. Our approach achieves SOTA performance on several benchmarks including MIT-States and UT-Zappos. We also examine how each component contributes to improved generalization.

In this paper we propose an algorithm that can be used on top of a wide variety of self-supervised (SSL) approaches to take advantage of hierarchical structures that emerge during training. SSL approaches typically work through some invariance term to ensure consistency between similar samples and a regularization term to prevent global dimensional collapse. Dimensional collapse refers to data representations spanning a lower-dimensional subspace. Recent work has demonstrated that the representation space of these algorithms gradually reflects a semantic hierarchical structure as training progresses. Data samples of the same hierarchical grouping tend to exhibit greater dimensional collapse locally compared to the dataset as a whole due to sharing features in common with each other. Ideally SSL algorithms would take advantage of this hierarchical emergence to have an additional regularization term to account for this local dimensional collapse effect. However the construction of existing SSL algorithms does not account for this property. To address this we propose an adaptive algorithm that performs a weighted decomposition of the denominator of the InfoNCE loss into two terms: local hierarchical and global collapse regularization respectively. This decomposition is based on an adaptive threshold that gradually lowers to reflect the emerging hierarchical structure of the representation space throughout training. It is based on an analysis of the cosine similarity distribution of samples in a batch. We demonstrate that this hierarchical emergence exploitation (HEX) approach can be integrated across a wide variety of SSL algorithms. Empirically we show performance improvements of up to 5.6% relative improvement over baseline SSL approaches on classification accuracy on Imagenet with 100 epochs of training.

Existing camouflaged object detection methods often struggle with detecting small objects and fine object boundaries. To alleviate these issues we propose a novel hourglass vision Transformer with Dual-path Feature Pyramid (HDPNet). Specifically we construct an hourglass Transformer encoder that effectively captures the global semantic cues while extracting detailed feature maps at various scales preserving the spatial details and fine-grained boundaries of the camouflaged object. To ensure the preservation of essential cues of hourglass features we introduce a dual-path feature pyramid decoder (DPFD). This decoder performs coarse-to-fine feature fusion laterally mitigating the dilution of essential feature cues caused by the semantic gaps. In addition to further facilitate the local feature modeling in the encoder to mine the correlation between local features and global semantic cues from the camouflaged region we design a feature interaction enhancement module (FIEM). This module adopts a symmetric structure enables detailed appearance features and global semantic features to complement each other enhancing the model's ability to capture a wide range of fine-grained details. Extensive quantitative and qualitative experiments demonstrate that the proposed model significantly outperforms 25 existing methods across three challenging COD benchmark datasets particularly excelling in the detection of small objects and fine boundaries. The code is available at https://github.com/LittleGrey-hjp/HDPNet.

Forecasting Abnormal Human Behavior (AHB) aims to predict unusual behavior in advance by analyzing early patterns of normal human interactions. Unlike typical action prediction methods this task focuses on observing only normal interactions to predict both short and long term future abnormal behavior. Despite its affirmative impact on society AHB prediction remains under-explored in current research. This is primarily due to the challenges involved in anticipating complex human behaviors and interactions with surrounding agents in real-world situations. Further there exists an underlying uncertainty between the early normal patterns and the future abnormal behavior thereby making the prediction harder. To address these challenges we introduce a novel transformer model that improves early interaction modeling by accounting for uncertainties in both observations and future outcomes. To the best of our knowledge we are the first to explore the task. Therefore we present a new comprehensive dataset referred to as AHB-F which features real-world scenarios with complex human interactions. The AHB-F has a deterministic evaluation protocol that ensures only normal frames to be observed for long and short term future prediction. We extensively evaluate and compare competitive action anticipation methods on our benchmark. Our results show that our method consistently outperforms existing action anticipation approaches both in quantitative and qualitative evaluations.

Adversarial attacks rely on transferability where an adversarial example (AE) crafted on a surrogate classifier tends to mislead a target classifier. Recent ensemble methods demonstrate that AEs are less likely to mislead multiple classifiers in an ensemble. This paper proposes a new ensemble training using a Pairwise Adversarially Robust Loss (PARL) that by construction produces an ensemble of classifiers with diverse decision boundaries. PARL utilizes outputs and gradients of each layer with respect to network parameters in every classifier within the ensemble simultaneously. PARL is demonstrated to achieve higher robustness against black-box transfer attacks than previous ensemble methods as well as adversarial training without adversely affecting clean example accuracy. Extensive experiments using standard Resnet20 WideResnet28-10 classifiers demonstrate the robustness of PARL against state-of-the-art adversarial attacks. While maintaining similar clean accuracy and lesser training time the proposed architecture has a 24.8% increase in robust accuracy (e = 0.07) from the state-of-the art method. Code is available at: https://github.com/shubhishukla10/PARL

Goal-Oriented Vision-and-Language Navigation (VLN) aims to enable agents to navigate to specified locations and identify designated target objects following natural language instruction. This approach has gained popularity due to its close alignment with real-world scenarios. However existing studies have predominantly focused on enhancing navigation performance neglecting the ability to locate objects at the navigation endpoint. This oversight has resulted in a significant discrepancy between the success rates of navigation and object grounding. The challenge is compounded by the complex reasoning required by the instructions and the necessity to synthesize multi-perspective images of objects which overwhelms traditional object grounding methods. We leverage the Multi-Modal Large Language Model (MLLM) to bridge this gap allowing agents to seek assistance from these models when struggling to locate the target object. The agent conducts a multi-stage evaluation to discern the cause of its confusion and promptly extracts and updates the most relevant information for MLLM to assess. Our method is plug-and-play and model-agnostic facilitating integration with numerous existing VLN strategies without the need for retraining. Implementing our approach across four distinct methods has improved performance on the REVERIE and SOON datasets demonstrating the effectiveness and generalizability of our technique.

Skeleton-based video anomaly detection (SVAD) is a crucial task in computer vision. Accurately identifying abnormal patterns or events enables operators to promptly detect suspicious activities thereby enhancing safety. Achieving this demands a comprehensive understanding of human motions both at body and region levels while also accounting for the wide variations of performing a single action. However existing studies fail to simultaneously address these crucial properties. This paper introduces a novel practical and lightweight framework namely Graph-Jigsaw Conditioned Diffusion Model for Skeleton-based Video Anomaly Detection (GiCiSAD) to overcome the challenges associated with SVAD. GiCiSAD consists of three novel modules: the Graph Attention-based Forecasting module to capture the spatio-temporal dependencies inherent in the data the Graph-level Jigsaw Puzzle Maker module to distinguish subtle region-level discrepancies between normal and abnormal motions and the Graph-based Conditional Diffusion model to generate a wide spectrum of human motions. Extensive experiments on four widely used skeleton-based video datasets show that GiCiSAD outperforms existing methods with significantly fewer training parameters establishing it as the new state-of-the-art.

Recent advances in text-to-image (T2I) diffusion models have facilitated creative and photorealistic image synthesis. By varying the random seeds we can generate many images for a fixed text prompt. Technically the seed controls the initial noise and in multi-step diffusion inference the noise used for reparameterization at intermediate timesteps in the reverse diffusion process. However the specific impact of the random seed on the generated images remains relatively unexplored. In this work we conduct a large-scale scientific study into the impact of random seeds during diffusion inference. Remarkably we reveal that the best 'golden' seed achieved an impressive FID of 21.60 compared to the worst 'inferior' seed's FID of 31.97. Additionally a classifier can predict the seed number used to generate an image with over 99.9% accuracy in just a few epochs establishing that seeds are highly distinguishable based on generated images. Encouraged by these findings we examined the influence of seeds on interpretable visual dimensions. We find that certain seeds consistently produce grayscale images prominent sky regions or image borders. Seeds also affect image composition including object location size and depth. Moreover by leveraging these 'golden' seeds we demonstrate improved image generation such as high-fidelity inference and diversified sampling. Our investigation extends to inpainting tasks where we uncover some seeds that tend to insert unwanted text artifacts. Overall our extensive analyses highlight the importance of selecting good seeds and offer practical utility for image generation.

Visual document understanding (VDU) has rapidly advanced with the development of powerful multi-modal language models. However these models typically require extensive document pre-training data to learn intermediate representations and often suffer a significant performance drop in real-world online industrial settings. A primary issue is their heavy reliance on OCR engines to extract local positional information within document pages which limits the models' ability to capture global information and hinders their generalizability flexibility and robustness. In this paper we introduce GlobalDoc a cross-modal transformer-based architecture pre-trained in a self-supervised manner using three novel pretext objective tasks. GlobalDoc improves the learning of richer semantic concepts by unifying language and visual representations resulting in more transferable models. For proper evaluation we also propose two novel document-level downstream VDU tasks Few-Shot Document Image Classification (DIC) and Content-based Document Image Retrieval (DIR) designed to simulate industrial scenarios more closely. Extensive experimentation has been conducted to demonstrate GlobalDoc's effectiveness in practical settings.