WACV 2025 Papers

The research community has shown increasing interest in designing intelligent embodied agents that can assist humans in accomplishing tasks. Although there have been significant advancements in related vision-language benchmarks most prior work has focused on building agents that follow instructions rather than endowing agents the ability to ask questions to actively resolve ambiguities arising naturally in embodied environments. To address this gap we propose an Embodied Learning-By-Asking (ELBA) model that learns when and what questions to ask to dynamically acquire additional information for completing the task. We evaluate ELBA on the TEACh vision-dialog navigation and task completion dataset. Experimental results show that the proposed method achieves improved task performance compared to baseline models without question-answering capabilities. Code is available at https://github.com/PLAN-Lab/ELBA.

Event cameras with high temporal resolution and high dynamic range have limited research on the inter-modality local feature extraction and matching of event-image data. We propose EI-Nexus an unmediated and flexible framework that integrates two modality-specific keypoint extractors and a feature matcher. To achieve keypoint extraction across viewpoint and modality changes we bring Local Feature Distillation (LFD) which transfers the viewpoint consistency from a well-learned image extractor to the event extractor ensuring robust feature correspondence. Furthermore with the help of Context Aggregation (CA) a remarkable enhancement is observed in feature matching. We further establish the first two inter-modality feature matching benchmarks MVSEC-RPE and EC-RPE to assess relative pose estimation on event-image data. Our approach outperforms traditional methods that rely on explicit modal transformation offering more unmediated and adaptable feature extraction and matching achieving better keypoint similarity and state-of-the-art results on the MVSEC-RPE and EC-RPE benchmarks. The source code and benchmarks will be made publicly available at EI-Nexus.

Transformer-based models have made significant progress in edge detection but their high computational cost is prohibitive. Recently vision Mamba have shown excellent ability in efficiently capturing long-range dependencies. Drawing inspiration from this we propose a novel edge detector with Mamba termed EDMB to efficiently generate high-quality multi-granularity edges. In EDMB Mamba is combined with a global-local architecture therefore it can focus on both global information and fine-grained cues. The fine-grained cues play a crucial role in edge detection but are usually ignored by ordinary Mamba. We design a novel decoder to construct learnable Gaussian distributions by fusing global features and fine-grained features. And the multi-grained edges are generated by sampling from the distributions. In order to make multi-granularity edges applicable to single-label data we introduce Evidence Lower Bound loss to supervise the learning of the distributions. On the multi-label dataset BSDS500 our proposed EDMB achieves competitive single-granularity ODS 0.837 and multi-granularity ODS 0.851 without multi-scale test or extra PASCAL-VOC data. Remarkably EDMB can be extended to single-label datasets such as NYUDv2 and BIPED. The source code is available at https://github.com/Li-yachuan/EDMB.

In this work we aim to increase the efficiency and the detection performance of lightweight object detectors with focus on feature fusion and receptive field of the models. For improved feature fusion we introduce the Convolutional Squeeze-and-Excitation (CSE) module which requires only minimal additional computation. For improving the receptive field and feature extraction capabilities in a resource effective manner we introduce the Cross-Stage Partial Context Augmentation Module (CSP-CAM). Furthermore for improving real-time performance we apply two model scaling techniques with minimal impact on the detection performance. We prove the effectiveness of the proposed modules by inserting them into YOLOv7-tiny and YOLOv9-t. We build a new network architecture ECFYOLOv7-tiny which we train on the MS COCO dataset and evaluate the inference speed on NVIDIA Jetson Nano. ECFYOLOv7-tiny achieves 37.8% mAP@[0.5:0.95] on the test set while reaching 9.3 FPS on Jetson Nano outperforming the other state-of-the-art lightweight object detectors on the 416 input image resolution. Code and models are released at: https://github.com/dbacea/ecf-yolov7-tiny.

Diffusion models in image Super-Resolution (SR) treat all image regions uniformly which risks compromising the overall image quality by potentially introducing artifacts during denoising of less-complex regions. To address this we propose "You Only Diffuse Areas" (YODA) a dynamic attention-guided diffusion process for image SR. YODA selectively focuses on spatial regions defined by attention maps derived from the low-resolution images and the current denoising time step. This time-dependent targeting enables a more efficient conversion to high-resolution outputs by focusing on areas that benefit the most from the iterative refinement process i.e. detail-rich objects. We empirically validate YODA by extending leading diffusion-based methods SR3 DiffBIR and SRDiff. Our experiments demonstrate new state-of-the-art performances in face and general SR tasks across PSNR SSIM and LPIPS metrics. As a side effect we find that YODA reduces color shift issues and stabilizes training with small batches.

Long-tailed class-incremental learning (LT-CIL) aims to learn new classes continuously from a long-tailed data stream while simultaneously dealing with challenges such as imbalanced learning of tail classes and catastrophic forgetting. To address these challenges most existing methods employ a two-stage strategy by initializing model training from scratch with further balanced knowledge driven calibration. This strategy faces challenges in deriving discriminative features from cold-started backbones for the long-tailed distribution of data consequently leading to relatively diminished performance. In this paper with the powerful feature extraction capability of pre-trained foundation models we have achieved a one-stage approach that delivers superior performance. Specifically we propose Dynamic Adapter Tuning (DAT) which employs a dynamic adapter cache mechanism to adapt a pre-trained model to learn tasks sequentially. The adapter in the cache is either dynamically selected or created according to task similarity and further compactified with the new task's adapter to mitigate cross-task and cross-class gaps in LT-CIL significantly alleviating catastrophic forgetting and imbalance learning issues respectively. With extensive experimental validation our method consistently achieves state-of-the-art performance under the challenging LT-CIL setting.

The advancement of autonomous driving systems hinges on the ability to achieve low-latency and high-accuracy perception. To address this critical need this paper introduces Dynamic Routering Network (DyRoNet) a low-rank enhanced dynamic routing framework designed for streaming perception in autonomous driving systems. DyRoNet integrates a suite of pre-trained branch networks each meticulously fine-tuned to function under distinct environmental conditions. At its core the framework offers a speed router module developed to assess and route input data to the most suitable branch for processing. This approach not only addresses the inherent limitations of conventional models in adapting to diverse driving conditions but also ensures the balance between performance and efficiency. Extensive experimental evaluations demonstrating the adaptability of DyRoNet to diverse branch selection strategies resulting in significant performance enhancements across different scenarios. This work not only establishes a new benchmark for streaming perception but also provides valuable engineering insights for future work.

Integrating language and images for Composed Image Retrieval (CIR) allows for a flexible representation of search intent making it a focal point in multi-modal research. Traditional CIR methods train models on complex triplet datasets. In contrast Zero-Shot Composed Image Retrieval (ZS-CIR) eliminates the need for constructing datasets and training models for each specific task attracting significant attention from researchers. CIReVL is a training-free method that translates visual content into textual representations employing large language models (LLMs) to perform robust text inference and visual language models (VLMs) to ensure multi-modal alignment during retrieval. Although this approach achieves high performance without training and offers strong interpretability relying solely on single descriptive texts for retrieval has limitations in handling complex search demands. To address this issue we propose a DualCIR framework which utilizes an LLM to generate refined dual-directional descriptions--both positive and negative--alongside generating holistic target descriptions. We score retrieval data separately via these descriptions merge the scores perform ranking and ultimately achieve retrieval. Extensive experiments across three datasets in the natural image and fashion domains show that our approach maintains low inference costs while noticeably enhancing retrieval effectiveness. Compared to existing training-free methods our DualCIR achieves state-of-the-art performance and is even competitive with the training-based methods.

Text-conditional image editing is a practical AIGC task that has recently emerged with great commercial and academic value. For real image editing most diffusion model-based methods use DDIM Inversion as the first stage before editing. However DDIM Inversion often results in reconstruction failure leading to unsatisfactory performance for downstream editing. To address this problem we first analyze why the reconstruction via DDIM Inversion fails. We then propose a new inversion and sampling method named Dual-Schedule Inversion. We also design a classifier to adaptively combine Dual-Schedule Inversion with different editing methods for user-friendly image editing. Our work can achieve superior reconstruction and editing performance with the following advantages: 1) It can reconstruct real images perfectly without fine-tuning and its reversibility is guaranteed mathematically. 2) The edited object/scene conforms to the semantics of the text prompt. 3) The unedited parts of the object/scene retain the original identity.

No-Reference Image Quality Assessment for distorted images has always been a challenging problem due to image content variance and distortion diversity. Previous IQA models mostly encode explicit single-quality features of synthetic images to obtain quality-aware representations for quality score prediction. However performance decreases when facing real-world distortion and restored images from restoration models. The reason is that they do not consider the degradation factors of the low-quality images adequately. To address this issue we first introduce the DRI method to obtain degradation vectors and quality vectors of images which separately model the degradation and quality information of low-quality images. After that we add the restoration network to provide the MOS score predictor with degradation information. Then we design the Representation-based Semantic Loss (RS Loss) to assist in enhancing effective interaction between representations. Extensive experimental results demonstrate that the proposed method performs favorably against existing state-of-the-art models on both synthetic and real-world datasets.

With the popularization of high-end mobile devices Ultra-high-definition (UHD) images have become ubiquitous in our lives. The restoration of UHD images is a highly challenging problem due to the exaggerated pixel count which often leads to memory overflow during processing. Existing methods either downsample UHD images at a high rate before processing or split them into multiple patches for separate processing. However high-rate downsampling leads to significant information loss while patch-based approaches inevitably introduce boundary artifacts. In this paper we propose a novel design paradigm to solve the UHD image restoration problem called D2Net. D2Net enables direct full-resolution inference on UHD images without the need for high-rate downsampling or dividing the images into several patches. Specifically we ingeniously utilize the characteristics of the frequency domain to establish long-range dependencies of features. Taking into account the richer local patterns in UHD images we also design a multi-scale convolutional group to capture local features. Additionally during the decoding stage we dynamically incorporate features from the encoding stage to reduce the flow of irrelevant information. Extensive experiments on three UHD image restoration tasks including low-light image enhancement image dehazing and image deblurring show that our model achieves better quantitative and qualitative results than state-of-the-art methods.

Thanks to their long-range dependencies transformers obtain state-of-the-art performance in diverse research fields such as computer vision and audio processing. In practical scenarios convolutional neural networks (CNNs) are used more than Transformers due to their low complexity. So Transformer-to-CNN knowledge distillation (KD) research where the Transformer is the teacher and the CNN is the student is in demand and receiving attention. In Transformer-to-CNN KD training the capacity gap problem arising from structural differences between the teacher and student networks is the main factor of performance degradation of the student network unlike homogenous architecture KD. However previous KD studies transfer all of a teacher's knowledge to the student without considering structural differences. They cannot overcome problems caused by structural differences and show poor performance in Transformer-to-CNN KD. In this paper we identify general and specific knowledge in feature maps of the teacher and student. General and specific knowledge are the generalized and non-generalized feature representation. We propose a novel KD framework DropKD which extracts general knowledge from the teacher and student while removing specific knowledge and then allows general knowledge of the student network to learn general knowledge of the teacher. Our DropKD empowers the student network to achieve generalization by effectively managing general and specific knowledge. Through extensive experiments on challenging image classification datasets we demonstrate that the proposed method is superior to existing methods.

Given a small number of images of a subject personalized image generation techniques can fine-tune large pre-trained text-to-image diffusion models to generate images of the subject in novel contexts conditioned on text prompts. In doing so a trade-off is made between prompt fidelity subject fidelity and diversity. As the pre-trained model is fine-tuned earlier checkpoints synthesize images with low subject fidelity but high prompt fidelity and diversity. In contrast later checkpoints generate images with low prompt fidelity and diversity but high subject fidelity. This inherent trade-off limits the prompt fidelity subject fidelity and diversity of generated images. In this work we propose DreamBlend to combine the prompt fidelity from earlier checkpoints and the subject fidelity from later checkpoints during inference. We perform a cross attention guided image synthesis from a later checkpoint guided by an image generated by an earlier checkpoint for the same prompt. This enables generation of images with better subject fidelity prompt fidelity and diversity on challenging prompts outperforming state-of-the-art fine-tuning methods.

Articulated 3D reconstruction has valuable applications in various domains yet it remains costly and demands intensive work from domain experts. Recent advancements in template-free learning methods show promising results with monocular videos. Nevertheless these approaches necessitate a comprehensive coverage of all viewpoints of the subject in the input video thus limiting their applicability to casually captured videos from online sources. In this work we study articulated 3D shape reconstruction from a single and casually captured Internet video where the subject's view coverage is incomplete. We propose DreaMo that jointly performs shape reconstruction while solving the challenging low-coverage regions with view-conditioned diffusion prior and several tailored regularizations. In addition we introduce a skeleton generation strategy to create human-interpretable skeletons from the learned neural bones and skinning weights without any predefined skeleton structures. We conduct our study on a self-collected internet video collection characterized by incomplete view coverage. DreaMo shows promising quality in novel-view rendering detailed articulated shape reconstruction and skeleton generation. Extensive qualitative and quantitative studies validate the efficacy of each proposed component and show existing methods are unable to solve correct geometry due to the incomplete view coverage.

This paper introduces the dynamic robust adaptive graph-based tracker (DragonTrack) as a novel end-to-end framework for multi-person tracking (MPT) by integrating a detection transformer model for object detection and feature extraction with a graph convolutional network for re-identification. DragonTrack leverages encoded features from the transformer for precise subject matching and track maintenance while the graphical component processes these features alongside geometric data to predict subsequent positions of tracked people. This methodology aims to enhance tracking accuracy and reliability as evidenced by improvements in key metrics such as higher order tracking accuracy (HOTA) and multiple object tracking accuracy (MOTA). We quantitatively compare DragonTrack with state-of-the-art methods on MOT17 MOT20 and DanceTrack datasets in which DragonTrack outperforms other methods. In challenging scenarios such as DanceTrack DragonTrack achieves an impressive MOTA score of 93.4 significantly higher than the second-best SOTA method ByteTrack which achieves only 89.6. Similarly on MOT17 DragonTrack scores 82.0 in MOTA surpassing the closest competitor with a score of 80.3. On MOT20 DragonTrack attains a HOTA score of 63.2 outperforming the next best method scoring 62.6

Point-based image editing enables accurate and flexible control through content dragging. However the role of text embedding during the editing process has not been thoroughly investigated. A significant aspect that remains unexplored is the interaction between text and image embeddings. During the progressive editing in a diffusion model the text embedding remains constant. As the image embedding increasingly diverges from its initial state the discrepancy between the image and text embeddings presents a significant challenge. In this study we found that the text prompt significantly influences the dragging process particularly in maintaining content integrity and achieving the desired manipulation. Upon these insights we propose DragText which optimizes text embedding in conjunction with the dragging process to pair with the modified image embedding. Simultaneously we regularize the text optimization process to preserve the integrity of the original text prompt. Our approach can be seamlessly integrated with existing diffusion-based drag methods enhancing performance with only a few lines of code.

Dependable visual drone detection is crucial for the secure integration of drones into the airspace. However drone detection accuracy is significantly affected by domain shifts due to environmental changes varied points of view and background shifts. To address these challenges we present the DrIFT dataset specifically developed for visual drone detection under domain shifts. DrIFT includes fourteen distinct domains each characterized by shifts in point of view synthetic-to-real data season and adverse weather. DrIFT uniquely emphasizes background shift by providing background segmentation maps to enable background-wise metrics and evaluation. Our new uncertainty estimation metric MCDO-map features lower postprocessing complexity surpassing traditional methods. We use the MCDO-map in our uncertainty-aware unsupervised domain adaptation method demonstrating superior performance to SOTA unsupervised domain adaptation techniques. The dataset is available at: https://github.com/CARG-uOttawa/DrIFT.git.

Unarguably deep learning models capable of generalizing to unseen domain data while leveraging a few labels are of great practical significance due to low developmental costs. In search of this endeavor we study the challenging problem of semi-supervised domain generalization (SSDG) where the goal is to learn a domain-generalizable model while using only a small fraction of labeled data and a relatively large fraction of unlabeled data. Domain generalization (DG) methods show subpar performance under the SSDG setting whereas semi-supervised learning (SSL) methods demonstrate relatively better performance however they are considerably poor compared to the fully-supervised DG methods. Towards handling this new but challenging problem of SSDG we propose a novel method that can facilitate the generation of accurate pseudo-labels under various domain shifts. This is accomplished by retaining the domain-level specialism in the classifier during training corresponding to each source domain. Specifically we first create domain-level information vectors on the fly which are then utilized to learn a domain-aware mask for modulating the classifier's weights. We provide a mathematical interpretation for the effect of this modulation procedure on both pseudo-labeling and model training. Our method is plug-and-play and can be readily applied to different SSL baselines for SSDG. Extensive experiments on six challenging datasets in two different SSDG settings show that our method provides visible gains over the various strong SSL-based SSDG baselines. Our code is available at github.com/DGWM.

In online applications significant risks exist in peer-to-peer transactions due to malicious behaviors of arbitrary users such as taking advantage of manipulated images or impersonating others using recaptured images. Moreover recent advancements in display screens and imaging devices have made it increasingly challenging to distinguish such spoofing images from the naked eye. However a lack of datasets for object anti-spoofing significantly hinders the practical implementation of object anti-spoofing techniques compared to facial anti-spoofing tasks. To address this data scarcity issue for object anti-spoofing we propose a method that utilizes face anti-spoofing images for training. Our approach leverages low-rank adaptation employing fine-tuning with downstream tasks of large language models to facilitate domain transition between faces and generic objects. We also analyze a power spectrum to select useful patches for spoofing detection and introduce a patch-based learning method to effectively capture spoofing patterns. Lastly we present a novel protocol for assessing domain generalization in the generic object anti-spoofing task. Our model demonstrates state-of-the-art generalization performance compared to existing object anti-spoofing models surpassing even those simply augmented with face datasets.