WACV 2025 论文列表

Despite promising progress in face swapping task realistic swapped images remain elusive often marred by artifacts particularly in scenarios involving high pose variation color differences and occlusion. To address these issues we propose a novel approach that better harnesses diffusion models for face-swapping by making following core contributions. (a) We propose to re-frame the face-swapping task as a self-supervised train-time inpainting problem enhancing the identity transfer while blending with the target image. (b) We introduce a multi-step Denoising Diffusion Implicit Model (DDIM) sampling during training reinforcing identity and perceptual similarities. (c) Third we introduce CLIP feature disentanglement to extract pose expression and lighting information from the target image improving fidelity. (d) Further we introduce a mask shuffling technique during inpainting training which allows us to create a so-called universal model for swapping with an additional feature of head swapping. Ours can swap hair and even accessories beyond traditional face swapping. Unlike prior works reliant on multiple off-the-shelf models ours is a relatively unified approach and so it is resilient to errors in other off-the-shelf models. Extensive experiments on FFHQ and CelebA datasets validate the efficacy and robustness of our approach showcasing high-fidelity realistic face-swapping with minimal inference time. Our code is available at https://github.com/Sanoojan/REFace.

Modern person re-identification (Re-ID) methods have a weak generalization ability and experience a major accuracy drop when capturing environments change. This is because existing multi-camera Re-ID datasets are limited in size and diversity since such data is difficult to obtain. At the same time enormous volumes of unlabeled single-camera records are available. Such data can be easily collected and therefore it is more diverse. Currently single-camera data is used only for self-supervised pre-training of Re-ID methods. However the diversity of single-camera data is suppressed by fine-tuning on limited multi-camera data after pre-training. In this paper we propose ReMix a generalized Re-ID method jointly trained on a mixture of limited labeled multi-camera and large unlabeled single-camera data. Effective training of our method is achieved through a novel data sampling strategy and new loss functions that are adapted for joint use with both types of data. Experiments show that ReMix has a high generalization ability and outperforms state-of-the-art methods in generalizable person Re-ID. To the best of our knowledge this is the first work that explores joint training on a mixture of multi-camera and single-camera data in person Re-ID.

We present Reusable Motion prior (ReMP) an effective motion prior that can accurately track the temporal evolution of motion in various downstream tasks. Inspired by the success of foundation models we argue that a robust spatio-temporal motion prior can encapsulate underlying 3D dynamics applicable to various sensor modalities. We learn the rich motion prior from a sequence of complete parametric models of posed human body shape. Our prior can easily estimate poses in missing frames or noisy measurements despite significant occlusion by employing a temporal attention mechanism. More interestingly our prior can guide the system with incomplete and challenging input measurements to quickly extract critical information to estimate the sequence of poses significantly improving the training efficiency for mesh sequence recovery. ReMP consistently outperforms the baseline method on diverse and practical 3D motion data including depth point clouds LiDAR scans and IMU sensor data. Project page is available in https://hojunjang17.github.io/ReMP.

We introduce a novel Recursive Fusion model dubbed ReFu designed to integrate point clouds and meshes for exemplar-free 3D Class-Incremental Learning where the model learns new 3D classes while retaining knowledge of previously learned ones. Unlike existing methods that either rely on storing historical data to mitigate forgetting or focus on single data modalities ReFu eliminates the need for exemplar storage while utilizing the complementary strengths of both point clouds and meshes. To achieve this we introduce a recursive method which continuously accumulates knowledge by updating the regularized auto-correlation matrix. Furthermore we propose a fusion module featuring a Pointcloud-guided Mesh Attention Layer that learns correlations between the two modalities. This mechanism effectively integrates point cloud and mesh features leading to more robust and stable continual learning. Experiments across various datasets demonstrate that our proposed framework outperforms existing methods in 3D class-incremental learning. Project Page: https://arlo-yang.github.io/ReFu/

Modern Text-to-Image (T2I) Diffusion models have revolutionized image editing by enabling the generation of high-quality photorealistic images. While the de-facto method for performing edits with T2I models is through text instructions this approach is non-trivial due to the complex many-to-many mapping between natural language and images. In this work we address exemplar-based image editing - the task of transferring an edit from an exemplar pair to a content image(s). We propose ReEdit a modular and efficient end-to-end framework that captures edits in both text and image modalities while ensuring the fidelity of the edited image. We validate the effectiveness of ReEdit through extensive comparisons with state-of-the-art baselines and sensitivity analyses of key design choices. Our results demonstrate that ReEdit consistently outperforms contemporary approaches both qualitatively and quantitatively. Additionally ReEdit boasts high practical applicability as it does not require any task-specific optimization and is 4 times faster than the existing state-of-the-art. The code and data for our work is available at https://reedit-diffusion.github.io/.

The remarkable progress in deep learning (DL) showcases outstanding results in various computer vision tasks. However adaptation to real-time variations in data distributions remains an important challenge. Test-Time Training (TTT) was proposed as an effective solution to this issue which increases the generalization ability of trained models by adding an auxiliary task at train time and then using its loss at test time to adapt the model. Inspired by the recent achievements of contrastive representation learning in unsupervised tasks we propose ReC-TTT a test-time training technique that can adapt a DL model to new unseen domains by generating discriminative views of the input data. ReC-TTT uses cross-reconstruction as an auxiliary task between a frozen encoder and two trainable encoders taking advantage of a single shared decoder. This enables at test time to adapt the encoders to extract features that will be correctly reconstructed by the decoder that in this phase is frozen on the source domain. Experimental results show that ReC-TTT achieves better results than other state-of-the-art techniques in most domain shift classification challenges. The code is available at: https://github.com/warpcut/ReC-TTT

Most video restoration networks are slow have high computational load and can't be used for real-time video enhancement. In this work we design an efficient and fast framework to perform real-time video enhancement for practical use-cases like live video calls and video streams. Our proposed method called Recurrent Bottleneck Mixer Network (ReBotNet) employs a dual-branch framework. The first branch learns spatio-temporal features by tokenizing the input frames along the spatial and temporal dimensions using a ConvNext-based encoder and processing these abstract tokens using a bottleneck mixer. To further improve temporal consistency the second branch employs a mixer directly on tokens extracted from individual frames. A common decoder then merges the features form the two branches to predict the enhanced frame. In addition we use a recurrent training approach where the last frame's prediction is leveraged to efficiently enhance the current frame while improving temporal consistency. To evaluate our method we curate two new datasets that emulate real-world video call and streaming scenarios and show extensive results on multiple datasets where ReBotNet outperforms existing approaches with lower computations reduced memory requirements and faster inference time. Code: https://github.com/jeya-mariajose/rebot-net.

Biometric recognition from security camera video is a challenging problem when the individuals change clothes or when they are partly occluded. Others have recently demonstrated that CLIP's visual encoder performs well in this domain but existing methods fail to make use of the model's text encoder or temporal information available in video. In this paper we present VCLIP a method for person identification in videos captured in challenging poses and with changes to a person's clothing. Harnessing the power of pre-trained vision-language models we jointly train a temporal fusion network while fine-tuning the visual encoder. To leverage the cross-modal embedding space we use learned biometric pedestrian attribute features to further enhance our model's person re-identification (Re-ID) ability. We demonstrate significant performance improvements via experiments with the MEVID and CCVID datasets particularly in the more challenging clothes-changing conditions. In support of this and future methods that use textual attributes for Re-ID with multimodal models we release a dataset of annotated pedestrian attributes for the popular MEVID dataset.

Group distributionally robust optimization which aims to improve robust accuracies--worst-group and unbiased accuracies--is a prominent algorithm used to mitigate spurious correlations and address dataset bias. Although existing approaches have reported improvements in robust accuracies these gains often come at the cost of average accuracy due to inherent trade-offs. To control this trade-off flexibly and efficiently we propose a simple class-specific scaling strategy directly applicable to existing debiasing algorithms with no additional training. We further develop an instance-wise adaptive scaling technique to alleviate this trade-off even leading to improvements in both robust and average accuracies. Our approach reveals that a naive ERM baseline matches or even outperforms the recent debiasing methods by simply adopting the class-specific scaling technique. Additionally we introduce a novel unified metric that quantifies the trade-off between the two accuracies as a scalar value allowing for a comprehensive evaluation of existing algorithms. By tackling the inherent trade-off and offering a performance landscape our approach provides valuable insights into robust techniques beyond just robust accuracy. We validate the effectiveness of our framework through experiments across datasets in computer vision and natural language processing domains.

Differentiable volumetric rendering-based methods made significant progress in novel view synthesis. On one hand innovative methods have replaced the Neural Radiance Fields (NeRF) network with locally parameterized structures enabling high-quality renderings in a reasonable time. On the other hand approaches have used differentiable splatting instead of NeRF's ray casting to optimize radiance fields rapidly using Gaussian kernels allowing for fine adaptation to the scene. However differentiable ray casting of irregularly spaced kernels has been scarcely explored while splatting despite enabling fast rendering times is susceptible to clearly visible artifacts. Our work closes this gap by providing a physically consistent formulation of the emitted radiance and density decomposed with Gaussian functions associated with Spherical Gaussians/Harmonics for all-frequency colorimetric representation. We also introduce a method enabling differentiable ray casting of irregularly distributed Gaussians using an algorithm that integrates radiance fields slab by slab and leverages a BVH structure. This allows our approach to finely adapt to the scene while avoiding splatting artifacts. As a result we achieve superior rendering quality compared to the state-of-the-art while maintaining reasonable training times and achieving inference speeds of 25 FPS on the Blender dataset. The associated code is available at: github.com/hugobl1/ray gauss.

Vision transformers (ViTs) have dominated computer vision in recent years. However ViTs are computationally expensive and not well suited for mobile devices; this led to the prevalence of convolutional neural network (CNN) and ViT-based hybrid models for mobile vision applications. Recently Vision GNN (ViG) and CNN hybrid models have also been proposed for mobile vision tasks. However all of these methods remain slower compared to pure CNN-based models. In this work we propose Multi-Level Dilated Convolutions to devise a purely CNN-based mobile backbone. Using Multi-Level Dilated Convolutions allows for a larger theoretical receptive field than standard convolutions. Different levels of dilation also allow for interactions between the short-range and long-range features in an image. Experiments show that our proposed model outperforms state-of-the-art (SOTA) mobile CNN ViT ViG and hybrid architectures in terms of accuracy and/or speed on image classification object detection instance segmentation and semantic segmentation. Our fastest model RapidNet-Ti achieves 76.3% top-1 accuracy on ImageNet-1K with 0.9 ms inference latency on an iPhone 13 mini NPU which is faster and more accurate than MobileNetV2x1.4 (74.7% top-1 with 1.0 ms latency). Our work shows that pure CNN architectures can beat SOTA hybrid and ViT models in terms of accuracy and speed when designed properly.

Estimating six-degree-of-freedom poses is essential but difficult especially under challenging initial conditions as existing methods relying on photometric loss often fail catastrophically. To address this we propose a novel radiance field-based pose estimation framework that combines Monte Carlo initialization and decoupled optimization. First we design an initialization stage based on Monte Carlo sampling which screens the current optimal initial pose by generating multiple pose hypotheses within the known scene space. Subsequently our decoupled optimization strategy independently refines the rotational and translational components significantly improving the motion stability of the camera during the convergence process. Comprehensive evaluations on synthetic and real-world datasets demonstrate that our method outperforms existing approaches in terms of pose regression accuracy and robustness even under challenging initial pose.

RT-DETR is the first real-time end-to-end transformer-based object detector. Its efficiency comes from the framework design and the Hungarian matching. However compared to dense supervision detectors like the YOLO series the Hungarian matching provides much sparser supervision leading to insufficient model training and difficult to achieve optimal results. To address these issues we proposed a hierarchical dense positive supervision method based on RT-DETR named RT-DETRv3. Firstly we introduce a CNN-based auxiliary branch that provides dense supervision that collaborates with the original decoder to enhance the encoder's feature representation. Secondly to address insufficient decoder training we propose a novel learning strategy involving self-attention perturbation. This strategy diversifies label assignment for positive samples across multiple query groups thereby enriching positive supervisions. Additionally we introduce a shared-weight decoder branch for dense positive supervision to ensure more high-quality queries matching each ground truth. Notably all aforementioned modules are training-only. We conduct extensive experiments to demonstrate the effectiveness of our approach on COCO val2017. RT-DETRv3 significantly outperforms existing real-time detectors including the RT-DETR series and the YOLO series. For example RT-DETRv3-R18 achieves 48.1% AP (+1.6%/+1.4%) compared to RT-DETR-R18/RT-DETRv2-R18 while maintaining the same latency. Furthermore RT-DETRv3-R101 can attain an impressive 54.6% AP outperforming YOLOv10-X. The code will be released at https://github.com/clxia12/RT-DETRv3.

Reconstructing an object's shape and appearance in terms of a mesh textured by a spatially-varying bidirectional reflectance distribution function (SVBRDF) from a limited set of images captured under collocated light is an ill-posed problem. Previous state-of-the-art approaches either aim to reconstruct the appearance directly on the geometry or additionally use texture normals as part of the appearance features. However this requires detailed but inefficiently large meshes that would have to be simplified in a post-processing step or suffers from well-known limitations of normal maps such as missing shadows or incorrect silhouettes. Another limiting factor is the fixed and typically low resolution of the texture estimation resulting in loss of important surface details. To overcome these problems we present ROSA an inverse rendering method that directly optimizes mesh geometry with spatially adaptive mesh resolution solely based on the image data. In particular we refine the mesh and locally condition the surface smoothness based on the estimated normal texture and mesh curvature. In addition we enable the reconstruction of fine appearance details in high-resolution textures through a pioneering tile-based method that operates on a single pre-trained decoder network but is not limited by the network output resolution.

Recent advancements in anomaly detection have shifted focus towards Multi-class Unified Anomaly Detection (MUAD) offering more scalable and practical alternatives compared to traditional one-class-one-model approaches. However existing MUAD methods often suffer from inter-class interference and are highly susceptible to domain shifts leading to substantial performance degradation in real-world applications. In this paper we propose a novel robust prompt-driven MUAD framework called ROADS to address these challenges. ROADS employs a hierarchical class-aware prompt integration mechanism that dynamically encodes class-specific information into our anomaly detector to mitigate interference among anomaly classes. Additionally ROADS incorporates a domain adapter to enhance robustness against domain shifts by learning domain-invariant representations. Extensive experiments on MVTec-AD and VISA datasets demonstrate that ROADS surpasses state-of-the-art methods in both anomaly detection and localization with notable improvements in out-of-distribution settings.

We introduce RGB2Point an unposed single-view RGB image to a 3D point cloud generation based on Transformer. RGB2Point takes an input image of an object and generates a dense 3D point cloud. Contrary to prior works based on CNN layers and diffusion-denoising approaches we use pre-trained Transformer layers that are fast and generate high-quality point clouds with consistent quality over available categories. Our generated point clouds demonstrate high quality on a real-world dataset as evidenced by improved Chamfer distance (51.15%) and Earth Mover's distance (36.17%) metrics compared to the current state-of-the-art. Additionally our approach shows a better quality on a synthetic dataset achieving better Chamfer distance (39.26%) Earth Mover's distance (26.95%) and F-score (47.16%). Moreover our method produces 63.1% more consistent high-quality results across various object categories compared to prior works. Furthermore RGB2Point is computationally efficient requiring only 2.3GB of VRAM to reconstruct a 3D point cloud from a single RGB image and our implementation generates the results 15133x faster than a SOTA diffusion-based model.

Mirror detection aims to identify mirror areas in a scene with recent methods either integrating depth information (RGB-D) or making use of temporal information (video). However utilizing both data is still under-explored due to the lack of a high-quality dataset and an effective method for the RGB-D Video Mirror Detection (DVMD) problem. To the best of our knowledge this is the first work to address the DVMD problem. To exploit depth and temporal information in mirror segmentation we first construct a large-scale RGB-D Video Mirror Detection Dataset (DVMD-D) which contains 17977 RGB-D images from 273 diverse videos. We further develop a novel model named DVMDNet which can first locate the mirrors based on triple consistencies: local consistency cross-modality consistency and global consistency and then refine the mirror boundaries through content discontinuity taking the temporal information within videos into account. We conduct a comparative study on the DVMD dataset evaluating 12 state-of-the art models (including single-image mirror detection single-image glass detection RGB-D mirror detection video shadow detection video glass detection and video mirror detection methods). Code is available from https://github.com/UpChen/2025_DVMDNet.

Selecting representative samples plays an indispensable role in many machine learning and computer vision applications under limited resources (e.g. limited communication bandwidth and computational power). Determinantal Point Process (DPP) is a widely used method for selecting the most diverse representative samples that can summarize a dataset. However its adaptability to different tasks remains an open challenge as it is challenging for DPP to perform task-specific tuning. In contrast Rate-Distortion (RD) theory provides a way to measure task-specific diversity. However optimizing RD for a data selection problem remains challenging because the quantity that needs to be optimized is the index set of the selected samples. To tackle these challenges we first draw an inherent relationship between DPP and RD theory. Our theoretical derivation paves the way to take advantage of both RD and DPP for a task-specific data selection. To this end we propose a novel method for task-specific data selection for multi-level classification tasks named RD-DPP. Empirical studies on seven different datasets using five benchmark models demonstrate the effectiveness of the proposed RD-DPP method. Our method also outperforms recent strong competing methods while exhibiting high generalizability to a variety of learning tasks. The source code is available on https://github.com/xiwenc1/RD-DPP

Current deep learning approaches in computer vision primarily focus on RGB data sacrificing information. In contrast RAW images offer richer representation which is crucial for precise recognition particularly in challenging conditions like low-light environments. The resultant demand for comprehensive RAW image datasets contrasts with the labor-intensive process of creating specific datasets for individual sensors. To address this we propose a novel diffusion-based method for generating RAW images guided by RGB images. Our approach integrates an RGB-guidance module for feature extraction from RGB inputs then incorporates these features into the reverse diffusion process with RGB-guided residual blocks across various resolutions. This approach yields high-fidelity RAW images enabling the creation of camera-specific RAW datasets. Our RGB2RAW experiments on four DSLR datasets demonstrate state-of-the-art performance. Moreover RAW-Diffusion demonstrates exceptional data efficiency achieving remarkable performance with as few as 25 training samples or even fewer. We extend our method to create BDD100K-RAW and Cityscapes-RAW datasets revealing its effectiveness for object detection in RAW imagery significantly reducing the amount of required RAW images. The code is available at https:// github.com/SonyResearch/RAW-Diffusion.