WACV 2025 Papers

The ability of Transformers to process longer sequences has led to unprecedented levels of generalization in visual tasks. However the complexity of Transformers is dominated by the quadratic cost associated with the computation of the attention blocks posing a bottleneck that impedes the scaling of sequence length and the realization of more advanced AI solutions. We propose and explore the hypothesis that the self-attention mechanism exhibits regularities that can be exploited to enhance performance and achieve linear-cost attention without significant loss of effectiveness. Specifically we investigate the attention matrix of Visual Transformers to identify and leverage these regularities in order to simplify the computation process. The resulting procedure significantly reduces the computational cost of Transformers by directly reducing attention block complexity. Moreover the designed procedure is 0-shot self-supervised thus it requires no retraining additional data or parameters as all Transformer parameters remain unchanged. Consequently the proposed method can be seamlessly applied to pre-trained Visual Transformers without the need for retraining. Experiments conducted on a series of Vision Transformers pre-trainedon ImageNet-1K dataset demonstrate the effectiveness of our proposed approach.

As Vision-Language Models (VLMs) advance human-centered Assistive Technologies (ATs) for helping People with Visual Impairments (PVIs) are evolving into generalists capable of performing multiple tasks simultaneously. However benchmarking VLMs for ATs remains under-explored. To bridge this gap we first create a novel AT benchmark (@BENCH). Guided by a pre-design user study with PVIs our benchmark includes the five most crucial vision-language tasks: Panoptic Segmentation Depth Estimation Optical Character Recognition (OCR) Image Captioning and Visual Question Answering (VQA). Besides we propose a novel AT model (@MODEL) that addresses all tasks simultaneously and can be expanded to more assistive functions for helping PVIs. Our framework exhibits outstanding performance across tasks by integrating multi-modal information and it offers PVIs a more comprehensive assistance. Extensive experiments prove the effectiveness and generalizability of our framework.

Deformable linear objects are commonly found in our daily lives. Understanding them visually can be challenging even for humans as the same object can become entangled and look completely different. Examples of deformable linear objects include blood vessels and wiring harnesses which are crucial for the proper functioning of systems like the human body and vehicles. Recently some studies have focused on 2D image segmentation of wires. However there are no point cloud datasets available for studying 3D deformable linear objects which are more complex and challenging. To address this gap we introduce two point cloud datasets PointWire and PointVessel generated using our proposed semi-automatic pipeline. We evaluated state-of-the-art methods on these large-scale 3D deformable linear object benchmarks. Additionally we analyzed the generalization capabilities of these methods through transferability experiments on the PointWire and PointVessel datasets.

We introduce a 3D synthesis method for architectural design to allow for the efficient generation of diverse and realistic building designs. In spite of advances in 3D synthesis current off-the-shelf 3D synthesis techniques are inappropriate for architectural design: they are trained primarily on isolated objects have limited diversity blend building facades with background and produce overly complex geometry that is difficult to edit or manipulate a major issue in an iterative design process. We propose an alternative pipeline that integrates auto-generated coarse models with segment-wise texture inpainting and semantics-based editing resulting in diverse style-consistent and shape-precise designs. We show through qualitative and quantitative experiments that our pipeline generates more diverse visually appealing architectures with clean geometries without the need for any extensive training.

Reconstruction of intricate local patterns and large missing regions during 3D shape completion has the contradictory requirements of computation over a wider context and operations for finer detail restoration. To this end we propose a multi-resolution spectral encoding based 3D shape completion approach to work on truncated Signed Distance Field (SDF) based shape representations. Our novelty lies in judiciously integrating multi-resolution 3D convolutional blocks that encode the input shape and a spectral module (SM) that captures the shape-wide context thus addressing the contradictory requirements. SM acts on the features extracted from both partial input scans and shape priors using the multi-resolution convolutional blocks. Our SM contains a 3D convolutional block placed between fast Fourier transform (FFT) and inverse FFT operations which results in the expansion of the receptive field for the appropriate context computation. Our approach has an attention-based encoder-decoder architecture where the encoding of a partial scan is acted upon by shape prior encodings to produce attention maps. These attention maps are leveraged differently in pretraining and in the later training and inference stages of our approach to produce the reconstructed 3D shape. A surface gradient-based loss function is used in addition to the L1 loss both in the pretraining and training stages for emphasizing the differences in minute details. These along with an attention refinement operation often leads to complete reconstruction while restoring finer details. Experiments using standard synthetic and real datasets demonstrate the superiority of our approach over the state-of-the-art.

Supervised 3D part segmentation models are tailored for a fixed set of objects and parts limiting their transferability to open-set real-world scenarios. Recent works have explored vision-language models (VLMs) as a promising alternative using multi-view rendering and textual prompting to identify object parts. However naively applying VLMs in this context introduces several drawbacks such as the need for meticulous prompt engineering and fails to leverage the 3D geometric structure of objects. To address these limitations we propose COPS a COmprehensive model for Parts Segmentation that blends the semantics extracted from visual concepts and 3D geometry to effectively identify object parts. COPS renders a point cloud from multiple viewpoints extracts 2D features projects them back to 3D and uses a novel geometric-aware feature aggregation procedure to ensure spatial and semantic consistency. Finally it clusters points into parts and labels them. We demonstrate that COPS is efficient scalable and achieves zero-shot state-of-the-art performance across five datasets covering synthetic and real-world data texture-less and coloured objects as well as rigid and non-rigid shapes. The code is available at https://3d-cops.github.io.

The semantic reconstruction of a scene relies in part on the curvilinear structure inherent in images. The recovery of curvilinear structure is not only key to the representation of objects via ridges and other object curves but is also critical to the reconstruction from texture-poor images which lack a sufficient number of features. Prior methods advocate for the recovery of curve segments from images and reconstructing these into an organized collection of 3D curve segments often referred to as the 3D curve sketch which serves as the basis for further reconstruction of curves and surfaces. Observing that the process of edge grouping can lead to fictitious curves or missing veridical groupings this paper advocates for a reconstruction of curvilinear structure directly from image edges in the form of a 3D edge sketch. The multiview reconstruction of edges faces significant combinatorial challenges which are effectively addressed in this paper. We demonstrate through experiments that the 3D edge sketch recovers a vast majority of the curvilinear structure and is a reliable substrate from which 3D curves can be constructed.

Preserving boundary continuity in the translation of 360-degree panoramas remains a significant challenge for existing text-driven image-to-image translation methods. These methods often produce visually jarring discontinuities at the translated panorama's boundaries disrupting the immersive experience. To address this issue we propose 360PanT a training-free approach to text-based 360-degree panorama-to-panorama translation with boundary continuity. Our 360PanT achieves seamless translations through two key components: boundary continuity encoding and seamless tiling translation with spatial control. Firstly the boundary continuity encoding embeds critical boundary continuity information of the input 360-degree panorama into the noisy latent representation by constructing an extended input image. Secondly leveraging this embedded noisy latent representation and guided by a target prompt the seamless tiling translation with spatial control enables the generation of a translated image with identical left and right halves while adhering to the extended input's structure and semantic layout. This process ensures a final translated 360-degree panorama with seamless boundary continuity. Experimental results on both real-world and synthesized datasets demonstrate the effectiveness of our 360PanT in translating 360-degree panoramas. Code is available at https://github.com/littlewhitesea/360PanT