CVPR 2026 Papers

Conference on Computer Vision and Pattern Recognition

Decision breakdown

4,042papers

Counted from the official paper list; may differ slightly from the number the conference announced.

  • Oral
    1383.4%
  • Poster
    3,90496.6%

Browse all papers

ShreddingNet: Coarse-to-Fine Restoration for Multi-Source Shredded Manuscripts

Haoyang Cui, Hao Jiang, Yadong Mu

Expand abstract

As an important research task of human cultural heritage, the restoration of artworks and calligraphy is of great significance. Seldom existing works have taken the multi-source (i.e., fragments are not ensured to be from the same piece of artworks) fragment-oriented restoration task into account. We propose ShreddingNet, a coarse-to-fine two-stage pipeline for multi-source manuscript restoration that operates without restrictive conditions. The proposed coarse stage compares the features of each fragment, selecting top-K candidates and clustering fragments by source. This design leverages the key insight that erroneous matches rarely cross source boundaries, enabling high-precision clustering. The proposed fine-grained stage evaluates candidates, yielding matching scores and filters out erroneous matching pairs from the candidate set; producing more precise final matching pairs for global assembly. Experiments conducted on more than 4,000 images from two datasets demonstrate the average reconstruction F1-score achieves 98.37%, which is 5.72% higher than the current state-of-the-art method, confirming the method's effectiveness and robustness. Source code is available at github.com/tqychy/shreddingnet.

Poster
BrepGaussian: CAD reconstruction from Multi-View Images with Gaussian Splatting

Jiaxing Yu, Dongyang Ren, Hangyu Xu, Zhouyuxiao Yang, Yuanqi Li, Jie Guo, Zhengkang Zhou, Yanwen Guo

Expand abstract

The boundary representation (B-Rep) models a 3D solid as its explicit boundaries: trimmed corners, edges, and faces. Recovering B-Rep representation from unstructured data is a challenging and valuable task of computer vision and graphics. Recent advances in deep learning have greatly improved the recovery of 3D shape geometry, but still depend on dense and clean point clouds and struggle to generalize to novel shapes. We propose B-Rep Gaussian Splatting (BrepGaussian), a novel framework that learns 3D parametric representations from 2D images. We employ a Gaussian Splatting renderer with learnable features, followed by a specific fitting strategy. To disentangle geometry reconstruction and feature learning, we introduce a two-stage learning framework that first captures geometry and edges and then refines patch features to achieve clean geometry and coherent instance representations. Extensive experiments demonstrate the superior performance of our approach to state-of-the-art methods.

Poster
Omni-3DEdit: Generalized Versatile 3D Editing in One-Pass

Liyi Chen, Pengfei Wang, Guowen Zhang, Zhiyuan Ma, Lei Zhang

Expand abstract

Most instruction-driven 3D editing methods rely on 2D models to guide the explicit and iterative optimization of 3D representations. This paradigm, however, suffers from two primary drawbacks. First, it lacks a universal design of different 3D editing tasks because the explicit manipulation of 3D geometry necessitates task-dependent rules, e.g., 3D appearance editing demands inherent source 3D geometry, while 3D removal alters source geometry. Second, the iterative optimization process is highly time-consuming, often requiring thousands of invocations of 2D/3D updating. We present Omni-3DEdit, a unified, learning-based model that generalizes various 3D editing tasks implicitly. One key challenge to achieve our goal is the scarcity of paired source-edited multi-view assets for training. To address this issue, we construct a data pipeline, synthesizing a relatively rich number of high-quality paired multi-view editing samples. Subsequently, we adapt the pre-trained generative model SEVA as our backbone by concatenating source view latents along with conditional tokens in sequence space. A dual-stream LoRA module is proposed to disentangle different view cues, largely enhancing our model's representational learning capability. As a learning-based model, our model is free of the time-consuming online optimization, and it can complete various 3D editing tasks in one forward pass, reducing the inference time from tens of minutes to approximately two minutes. Extensive experiments demonstrate the effectiveness and efficiency of Omni-3DEdit.

Poster
CustomTex: High-fidelity Indoor Scene Texturing via Multi-Reference Customization

Weilin Chen, Jiahao Rao, Wenhao Wang, Xinyang Li, Xuan Cheng, Liujuan Cao

Expand abstract

The creation of high-fidelity, customizable 3D indoor scene textures remains a significant challenge. While text-driven methods offer flexibility, they lack the precision for fine-grained, instance-level control, and often produce textures with insufficient quality, artifacts, and baked-in shading. To overcome these limitations, we introduce CustomTex, a novel framework for instance-level, high-fidelity scene texturing driven by reference images. CustomTex takes an untextured 3D scene and a set of reference images specifying the desired appearance for each object instance, and generates a unified, high-resolution texture map. The core of our method is a dual-distillation approach that separates semantic control from pixel-level enhancement. We employ semantic-level distillation, equipped with an instances cross attention, to ensure semantic plausibility and "reference-instance" alignment, and pixel-level distillation to enforce high visual fidelity. Both are unified within a Variational Score Distillation optimization framework. Experiments demonstrate that CustomTex achieves precise instance-level consistency with reference images and produces textures with superior sharpness, reduced artifacts, and minimal baked-in shading compared to state-of-the-art methods. Our work establishes a more direct and user-friendly path to high-quality, customizable 3D scene appearance editing.

Poster
Customized Fusion: A Closed-Loop Dynamic Network for Adaptive Multi-Task-Aware Infrared-Visible Image Fusion

Zengyi Yang, Yu Liu, Juan Cheng, Zhiqin Zhu, Yafei Zhang, Huafeng Li

Expand abstract

Infrared-visible image fusion aims to integrate complementary information for robust visual understanding, but existing fusion methods struggle with simultaneously adapting to multiple downstream tasks. To address this issue, we propose a Closed-Loop Dynamic Network (CLDyN) that can adaptively respond to the semantic requirements of diverse downstream tasks for task-customized image fusion. Specifically, CLDyN introduces a closed-loop optimization mechanism that establishes a semantic transmission chain to achieve explicit feedback from downstream tasks to the fusion network through a Requirement-driven Semantic Compensation (RSC) module. The RSC module leverages a Basis Vector Bank (BVB) and an Architecture-Adaptive Semantic Injection (A2SI) block to customize the network architecture according to task requirements, thereby enabling task-specific semantic compensation and allowing the fusion network to actively adapt to diverse tasks without retraining. To promote semantic compensation, a reward-penalty strategy is introduced to reward or penalize the RSC module based on task performance variations. Experiments on the M3FD, FMB, and VT5000 datasets demonstrate that CLDyN not only maintains high fusion quality but also exhibits strong multi-task adaptability. The code is available at https://github.com/YR0211/CLDyN.

Oral
Wan-Weaver: Interleaved Multi-modal Generation via Decoupled Training

Jinbo Xing, Zeyinzi Jiang, Yuxiang Tuo, Chaojie Mao, Xiaotang Gai, Xi Chen, Jingfeng Zhang, Yulin Pan, Zhen Han, Jie Xiao, Keyu Yan, Chenwei Xie, Chongyang Zhong, Kai Zhu, Tong Shen, Lianghua Huang, Yu Liu, Yujiu Yang

Expand abstract

Recent unified models have made unprecedented progress in both understanding and generation. However, while most of them accept multi-modal inputs, they typically produce only single-modality outputs. This challenge of producing interleaved content is mainly due to training data scarcity and the difficulty of modeling long-range cross-modal context. To address this issue, we decompose interleaved generation into textual planning and visual consistency modeling, and introduce a framework consisting of a planner and a visualizer. The planner produces dense textual descriptions for visual content, while the visualizer synthesizes images accordingly. Under this guidance, we construct large-scale textual-proxy interleaved data (where visual content is represented in text) to train the planner, and curate reference-guided image data to train the visualizer. These designs give rise to Wan-Weaver, which exhibits emergent interleaved generation ability with long-range textual coherence and visual consistency. Meanwhile, the integration of diverse understanding and generation data into planner training enables Wan-Weaver to achieve robust task reasoning and generation proficiency. To assess the model's capability in interleaved generation, we further construct a benchmark that spans a wide range of use cases across multiple dimensions. Extensive experiments demonstrate that, even without access to any real interleaved data, Wan-Weaver achieves superior performance over existing methods.

Oral
Bidirectional Multimodal Prompt Learning with Scale-Aware Training for Few-Shot Multi-Class Anomaly Detection

Yujin Lee, Sewon Kim, Daeun Moon, Seoyoon Jang, Hyunsoo Yoon

Expand abstract

Few-shot multi-class anomaly detection is crucial in real industrial settings, where only a few normal samples are available while numerous object types must be inspected. This setting is challenging as defect patterns vary widely across categories while normal samples remain scarce. Existing vision-language model-based approaches typically depend on class-specific anomaly descriptions or auxiliary modules, limiting both scalability and computational efficiency. In this work, we propose AnoPLe, a lightweight multimodal prompt learning framework that removes reliance on anomaly-type textual descriptions and avoids any external modules. AnoPLe employs bidirectional interactions between textual and visual prompts, allowing class semantics and instance-level cues to refine one another and form class-conditioned representations that capture shared normal patterns across categories. To enhance localization, we design a scale-aware prefix trained on both global and local views, enabling the prompts to capture both global context and fine-grained details. In addition, alignment loss propagates local anomaly evidence to global features, strengthening the consistency between pixel- and image-level predictions. Despite its simplicity, AnoPLe achieves strong performance on MVTec-AD, VisA, and Real-IAD under the few-shot multi-class setting, surpassing prior approaches while remaining efficient and free from expert-crafted anomaly descriptions. Moreover, AnoPLe generalizes well to unseen anomalies and extends effectively to the medical domain.

Poster
URScenes: A Multi-scenario Dataset for Unstructured Road Environments

Runsen Liu, Aizemaitijiang Baoerhan, Zhangyu Wang, Jie Wang, Jinghao Cui, Guizhen Yu, Songyue Yang, WanCheng Sun, Mingjun Tang, Zhanbo Hua, Wenwen Luo

Expand abstract

As autonomous driving technology transitions from small-scale validation to large-scale deployment, its development in unstructured road environments has become a critical and inevitable trend. Autonomous vehicles increasingly rely on high-quality and diverse datasets for perception systems. However, existing public datasets predominantly focus on clear-weather and urban-road scenarios, leaving a significant gap in the coverage of unstructured road environments. To bridge this gap, we construct URScenes, the first multi-scenario, open-source perception dataset for unstructured road environments. The dataset consists of 472 scenes, each lasting 30 seconds, and provides over 28K annotated samples and 119K sweeps. URScenes, for the first time, covers eight typical scenarios, including rainy, snowy, foggy, dusty, high-glare, night, cloudy, and sunny conditions. Additionally, URScenes supports multi-task perception for 3D object detection, multi-object tracking, and 3D occupancy in unstructured road environments. URScenes also provides a unified annotation system and format conversion tools, enabling easy conversion to popular formats such as nuScenes, KITTI, and Waymo datasets. Finally, this study presents comparative experimental results to assess the performance of state-of-the-art algorithms on the URScenes dataset. The data and development toolkit are available at http://www.sav-lab.com.

Poster
Multi-View Hierarchical Alignment Learning for Spatial Transcriptomics

Zhengzhong Zhu, Liangjin Liu, Pei Zhou, Shiquan Min, Jiangping Zhu

Expand abstract

Spatial transcriptomics provides both spatial coordinates and gene expression profiles, enabling the study of tissue organization and cellular heterogeneity. Despite recent progress, current spatial clustering methods still face two major limitations. First, representations learned from spatial and expression views often differ due to view-specific noise and incomplete structural information. Without enforcing sample-level cross-view consistency, embeddings from the two views may not correspond to the same biological identity, reducing discriminative capability. Second, existing approaches lack effective semantic-level supervision. Although node embeddings capture local neighborhood patterns, they do not explicitly reflect high-level semantic structures. Prototype-based modeling can provide such semantic abstraction, yet current methods seldom align prototypes with node representations, leading to weak semantic consistency. To overcome these issues, we propose a Multi-View Hierarchical Alignment Learning for Spatial Transcriptomics (MHAL). At the sample level, MHAL introduce positive sample alignment to enforce consistency between spatial and expression embeddings. At the semantic level, MHAL design prototype level contrastive learning, where prototypes act as semantic anchors and guide the formation of coherent cluster structures. Together, these two alignment mechanisms progressively ensure both local consistency and global semantic discrimination. Extensive experimental results demonstrate that the proposed hierarchical contrastive multi-view clustering method achieves competitive performance in spatial domain identification compared to other state-of-the-art methods.

Poster
Learning Compact 3D Representations from Feed-Forward Novel View Synthesis

Honggyu An, Jaewoo Jung, Mungyeom Kim, Chaehyun Kim, Minkyeong Jeon, Jisang Han, Kazumi Fukuda, Takuya Narihira, Hyunah Ko, Junsu Kim, Sunghwan Hong, Yuki Mitsufuji, Seungryong Kim

Expand abstract

Reconstructing and understanding 3D scenes from unposed sparse views in a feed-forward manner remains as a challenging task in 3D computer vision. Recent approaches use per-pixel 3D Gaussian Splatting for reconstruction, followed by a 2D-to-3D feature lifting stage for scene understanding. However, they generate excessive redundant Gaussians, causing high memory overhead and sub-optimal multi-view feature aggregation, leading to degraded novel view synthesis and scene understanding performance. We propose C3G, a novel feed-forward framework that estimates compact 3D Gaussians only at essential spatial locations, minimizing redundancy while enabling effective feature lifting. We introduce learnable tokens that aggregate multi-view features through self-attention to guide Gaussian generation, ensuring each Gaussian integrates relevant visual features across views. We then exploit the learned attention patterns for Gaussian decoding to efficiently lift features. Extensive experiments on pose-free novel view synthesis, 3D open-vocabulary segmentation, and view-invariant feature aggregation demonstrate our approach's effectiveness. Results show that a compact yet geometrically meaningful representation is sufficient for high-quality scene reconstruction and understanding, achieving superior memory efficiency and feature fidelity compared to existing methods.

Poster
Orthogonal Spatial-Aware Multi-View Anchor Graph Clustering for Incomplete Remote Sensing Data

Yongshan Zhang, Xiaohuan Lin, Lefei Zhang, Zhihua Cai

Expand abstract

Multi-view clustering for remote sensing data has received increasing attention by leveraging diverse data representations to enhance Earth observation. Existing methods are primarily developed under the assumption that each pixel is fully observed across all views. No prior work has investigated the more practical yet challenging scenario where some views suffer from partially missing data. To bridge this gap, this paper presents the first study on clustering incomplete remote sensing data, termed orthogonal spatial-aware multi-view anchor graph clustering (OSMAGC). Specifically, spatial-aware anchors and multi-scale anchor graphs are initially constructed by exploiting the superpixel-based texture characteristics of each view. Based on these, multi-scale anchor graph learning is performed through view weighting and matrix factorization on incomplete data. Structure-aligned consensus feature learning is achieved by aligning the multi-scale graph structures within a shared latent space. To ensure spatial continuity and smoothness, orthogonal spatial-aware regularization is imposed in both horizontal and vertical directions. These three modules are jointly optimized through a well-designed optimization algorithm in a mutually reinforcing manner. Extensive experiments on four benchmark datasets validate the effectiveness and efficiency of our proposed method over the state-of-the-art competitors.

Poster
Reparameterized Tensor Ring Functional Decomposition for Multi-Dimensional Data Recovery

Yangyang Xu, Junbo Ke, You-Wei Wen, Chao Wang

Expand abstract

Tensor Ring (TR) decomposition is a powerful tool for high-order data modeling, but is inherently restricted to discrete forms defined on fixed meshgrids. In this work, we propose a TR functional decomposition for both meshgrid and non-meshgrid data, where factors are parameterized by Implicit Neural Representations (INRs). However, optimizing this continuous framework to capture fine-scale details is intrinsically difficult. Through a frequency-domain analysis, we demonstrate that the spectral structure of TR factors determines the frequency composition of the reconstructed tensor and limits the high-frequency modeling capacity. To mitigate this, we propose a reparameterized TR functional decomposition, in which each TR factor is a structured combination of a learnable latent tensor and a fixed basis. This reparameterization is theoretically shown to improve the training dynamics of TR factor learning. We further derive a principled initialization scheme for the fixed basis and prove the Lipschitz continuity of our proposed model. Extensive experiments on image inpainting, denoising, super-resolution, and point cloud recovery demonstrate that our method achieves consistently superior performance over existing approaches. Code is available at https://github.com/YangyangXu2002/RepTRFD.

Poster
Garments2Look: A Multi-Reference Dataset for High-Fidelity Outfit-Level Virtual Try-On with Clothing and Accessories

Junyao Hu, Zhongwei Cheng, Waikeung Wong, Xingxing Zou

Expand abstract

Virtual try-on (VTON) has advanced single-garment visualization, yet real-world fashion centers on full outfits with multiple garments, accessories, fine-grained categories, layering, and diverse styling, remaining beyond current VTON systems. Existing datasets are category-limited and lack outfit diversity. We introduce Garments2Look, the first large-scale multimodal dataset for outfit-level VTON, comprising 80K many-garments-to-one-look pairs across 40 major categories and 300+ fine-grained subcategories. Each pair includes an outfit with 3-12 reference garment images (Average 4.48), a model image wearing the outfit, and detailed item and try-on textual annotations. To balance authenticity and diversity, we propose a synthesis pipeline. It involves heuristically constructing outfit lists before generating try-on results, with the entire process subjected to strict automated filtering and human validation to ensure data quality. To probe task difficulty, we adapt SOTA VTON methods and general-purpose image editing models to establish baselines. Results show current methods struggle to try on complete outfits seamlessly and to infer correct layering and styling, leading to misalignment and artifacts.

Poster
WeatherCity: Urban Scene Reconstruction with Controllable Multi-Weather Transformation

Wenhua Wu, Huai Guan, Zhe Liu, Hesheng Wang

Expand abstract

Editable high-fidelity 4D scenes are crucial for autonomous driving, as they can be applied to end-to-end training and closed-loop simulation. However, existing reconstruction methods are primarily limited to replicating observed scenes and lack the capability for diverse weather simulation. While image-level weather editing methods tend to introduce scene artifacts and offer poor controllability over the weather effects. To address these limitations, we propose WeatherCity, a novel framework for 4D urban scene reconstruction and weather editing. Specifically, we leverage a text-guided image editing model to achieve flexible editing of image weather backgrounds. To tackle the challenge of multi-weather modeling, we introduce a novel weather Gaussian representation based on shared scene features and dedicated weather-specific decoders. This representation is further enhanced with a content consistency optimization, ensuring coherent modeling across different weather conditions. Additionally, we design a physics-driven model that simulates dynamic weather effects through particles and motion patterns. Extensive experiments on multiple datasets and various scenes demonstrate that WeatherCity achieves flexible controllability, high fidelity, and temporal consistency in 4D reconstruction and weather editing. Our framework not only enables fine-grained control over weather conditions (e.g., light rain and heavy snow) but also supports object-level manipulation within the scene. Codes are released at https://github.com/IRMVLab/WeatherCity.

Poster
Learning Anchor in Dual Orthogonal Space for Fast Multi-view Clustering

Yalan Qin, Hanzhou Wu

Expand abstract

Large-scale multi-view clustering aims to explore the complementary and consistent information among different views in efficient manner. Despite the impressive performance gained by the existing methods, they just perform anchor learning in a single space with the orthogonal or some other constraints from the multi-view data, leading to undesired anchors. The anchors can simultaneously occur in more spaces and the complementary information among these spaces is able to be adopted for learning anchors. Meanwhile, the space with basis being the anchored cluster center is neglected to learn anchors by most existing works. In this work, we propose learning anchor in Dual Orthogonal Space for Fast Multi-view Clustering (DOSFMVC). DOSFMVC conducts anchor learning in dual orthogonal space, aiming at utilizing the complementary information among two spaces in producing anchors with high quality. DOSMFVC introduces the consensus anchored cluster center as basis of the extra space and clustering indicator of anchors based on this basis in anchor learning. The anchor learning and partition are integrated into a unified model, where the final cluster assignment can be adopted for clustering results. Extensive experiments confirm the superiority of our method compared with some state-of-the-art methods on several benchmark datasets.

Poster
V^2-SAM: Marrying SAM2 with Multi-Prompt Experts for Cross-View Object Correspondence

Jiancheng Pan, Runze Wang, Tianwen Qian, Mohammad Mahdi, Yanwei Fu, Xiangyang Xue, Xiaomeng Huang, Luc Van Gool, Danda Pani Paudel, Yuqian Fu

Expand abstract

Cross-view object correspondence, exemplified by the representative task of ego-exo object correspondence, aims to establish consistent associations of the same object across different viewpoints (e.g., ego-centric and exo-centric). This task poses significant challenges due to drastic viewpoint and appearance variations, making existing segmentation models, such as SAM2, non-trivial to apply directly. To address this, we present V^ 2 -SAM, a unified cross-view object correspondence framework that adapts SAM2 from single-view segmentation to cross-view correspondence through two complementary prompt generators. Specifically, the Cross-View Anchor Prompt Generator (V^ 2 -Anchor), built upon DINOv3 features, establishes geometry-aware correspondences and, for the first time, unlocks coordinate-based prompting for SAM2 in cross-view scenarios, while the Cross-View Visual Prompt Generator (V^ 2 -Visual) enhances appearance-guided cues via a novel visual prompt matcher that aligns ego-exo representations from both feature and structural perspectives. To effectively exploit the strengths of both prompts, we further adopt a multi-expert design and introduce a Post-hoc Cyclic Consistency Selector (PCCS) that adaptively selects the most reliable expert based on cyclic consistency. Extensive experiments validate the effectiveness of V^ 2 -SAM, achieving new state-of-the-art performance on Ego-Exo4D (Ego-Exo object correspondence), DAVIS-2017 (video object tracking), and HANDAL-X (robotic-ready cross-view correspondence).

Poster
MV-RoMa: From Pairwise Matching into Multi-View Track Reconstruction

Jongmin Lee, Seungyeop Kang, Sungjoo Yoo

Expand abstract

Establishing consistent correspondences across images is essential for 3D vision tasks such as structure-from-motion (SfM), yet most existing matchers operate in a pairwise manner, often producing fragmented and geometrically inconsistent tracks when their predictions are chained across views. We propose MV-RoMa, a multi-view dense matching model that jointly estimates dense correspondences from a source image to multiple co-visible targets. Specifically, we design an efficient model architecture which avoids high computational cost of full cross-attention for multi-view feature interaction: (i) multi-view encoder that leverages pair-wise matching results as a geometric prior, and (ii) multi-view matching refiner that refines correspondences using pixel-wise attention. Additionally, we propose a post-processing strategy that integrates our model's consistent multi-view correspondences as high-quality tracks for SfM. Across diverse and challenging benchmarks, MV-RoMa produces more reliable correspondences and substantially denser, more accurate 3D reconstructions than existing sparse and dense matching methods.

Poster
TaskIT: Memory-Efficient Fine-Tuning of Multi-LoRA LLMs via Cross-Task Importance Transfer

Cheng Fang, Zimu Zhou, Ke Ma, Bin Guo

Expand abstract

On-device AI systems increasingly adopt a single foundation model equipped with task-specific Low-Rank Adaptation (LoRA) modules, forming a multi-LoRA LLM that supports multiple tasks.We study how to adapt such a model to a new task on memory-constrainted devices.Although LoRA reduces trainable parameters, fine-tuning a full set of modules remains memory-intensive.To improve efficiency, we apply sparse updating, training a subset of LoRA modules within the memory budget.However, existing sparse updating methods assume all candidate parameters are instantiated and cannot estimate the importance of modules that do not yet exist, while prior memory models designed for sequential networks fail to capture the blockwise parallel structure of Transformers.We propose TaskIT, a framework for memory-efficient fine-tuning via cross-task importance transfer. TaskIT predicts pre-insertion module importance by transferring from previously tuned tasks and employs a block-based memory predictor that captures parallel and sequential dependencies of Transformer blocks. A dynamic programming scheduler then selects module locations, numbers, and ranks to maximize accuracy within the memory budget.Experiments on uni-modal and cross-modal benchmarks show that TaskIT achieves superior accuracy-memory tradeoffs compared with Zero-FT, non-LoRA, and LoRA-based fine-tuning methods.

Poster
Dual-Prototype-Guided Multi-task Learning for Unsupervised Anomaly Detection and Classification

Qianhao Luo, Jiajia Mi, Mingtao Yan, JingSheng Liu, ShuYang Pang, Weiling Li

Expand abstract

Unsupervised Anomaly Detection (UAD) and anomaly classification are frequently used in industrial and medical scenarios. Specifically, UAD identifies anomalous regions at the fine-grained pixel-level, while anomaly classification distinguishes anomaly types at the anomaly region level. However, existing approaches typically treat these tasks independently and sequentially, overlooking the benefits of jointly training them to suppress Local Visual Ambiguity (LVA) caused by the similarities of different types of anomalies in local visual patterns. Moreover, a multi-task learning framework cannot be directly applied to jointly train the two tasks, since UAD and anomaly classification exhibit feature preference incompatibility. To address these limitations, we propose the Prototype-Guided Semi-Supervised Feature Disentanglement (PG-SFD) framework, which makes a paradigm shift from implicit feature sharing to explicit feature disentanglement and explicitly constructs normal and category prototypes to eliminate implicit normal-abnormal semantic coupling via a Dual-Prototype Disentanglement Module (DPRM). Moreover, for cross-task feature differential injection and gradient conflict mitigation, the Differential Gated Interaction (DGI) and Geometry-Regularized Optimization (GRO) are proposed to form a cohesive framework with DPRM. PG-SFD demonstrates high effectiveness in both UAD tasks and weakly supervised classification tasks. Meanwhile, it exhibits stable performance across multiple types of datasets, including industrial and medical datasets, indicating its strong generalizability.

Poster
Multi-Modal Representation Learning via Semi-Supervised Rate Reduction for Generalized Category Discovery

Wei He, Xianghan Meng, Zhiyuan Huang, Xianbiao Qi, Rong Xiao, Chun-Guang Li

Expand abstract

Generalized Category Discovery (GCD) aims to identify both known and unknown categories, with only partial labels given for the known categories, posing a challenging open-set recognition problem. State-of-the-art approaches for GCD are usually built on multi-modality representation learning, which pays heavily attention upon inter-modality alignment rather than intra-modality alignment. In this paper, we propose a novel and effective multi-modal representation learning approach for GCD via Semi-Supervised Rate Reduction, called SSR^2-GCD, to learn cross-modality representations with desired underlying structure properties via properly harnessing intra-modality alignment. Moreover, to boost knowledge transfer, we integrate prompt candidates by leveraging the inter-modal alignment offered by Vision Language Models. We conduct extensive experiments on generic and fine-grained benchmark datasets, demonstrating superior performance of the proposed approach and verifying the importance of harnessing an proper intra-modality alignment.

Poster