ICML 2026 论文列表

International Conference on Machine Learning

录用类型

6,341篇论文

按官方录用论文列表统计,可能与会议公布的数字略有出入。

  • Spotlight
    5368.5%
  • Poster
    5,80591.5%

浏览全部论文

Distilling Linearized Behavior into Non-linear Fine-Tuning for Effective Task Arithmetic

Thomas Sommariva, Francesca Morandi, Simone Calderara, Angelo Porrello

展开摘要

Task vector composition has emerged as a promising paradigm for editing pre-trained models, enabling model merging through addition and unlearning through subtraction. Fine-tuning in the tangent space of a pre-trained model (linear fine-tuning) has proven effective, as it produces task vectors that are naturally disentangled and resistant to interference. However, linearized models suffer from limited expressivity during training and incur higher computational costs at inference time, which restrict their practical applicability. In this work, we bridge the gap between linear and standard non-linear fine-tuning. We show that linearity with respect to weight perturbations, a property defined in parameter space, can be enforced through constraints in activation space during training. Concretely, we distill hidden representations from a curvature-regularized linearized teacher into a non-linear student trained via conventional fine-tuning. We find that the resulting model inherits key properties of linearized models for task arithmetic, enabling effective composition of task vectors and achieving strong performance across vision and language benchmarks without incurring any inference-time overhead.

Poster
Rethinking Human Intent-to-CAD: Parametric CAD Model Generation via Cooperative Multi-Task Alignment and Spatial-Aware Reinforcement Learning

Qingwang Zhang, Jiahao Li, Xiangdong Zhou

展开摘要

Parametric Computer-Aided-Design (CAD) modeling from human intent remains challenging, particularly during the conceptual design stage, where design goals are expressed through incomplete and unstructured modalities (e.g., hand-drawn sketches and textual descriptions). In this work, we rethink the human intent-to-CAD pipeline and propose a unified method that directly maps multi-level human intents to executable codes, without assuming the prior existence of target CAD models. To support our study, we construct HiCAD, the first large-scale dataset aligning hand-drawn sketches, textual descriptions, and parametric CAD codes. Based on this, we introduce HiCAD, a two-stage framework comprising Cooperative Multi-Task Alignment to bridge the representational gap between heterogeneous inputs, and Spatial-Aware Reinforcement Learning to enforce geometric and topological consistency. Extensive experiments demonstrate that our method significantly outperforms existing baselines across multiple tasks, validating its effectiveness and robustness in transforming heterogeneous human intents into high-fidelity parametric CAD models. Our project page: https://zqwlearning.github.io/HiCAD.

Poster
Task-Aware Structured Memory for Dynamic Multi-modal In-Context Learning

Zhirui Chen, Ziwei Chen, Ling Shao

展开摘要

Multi-modal large language models (MLLMs) depend on in-context learning (ICL) for rapid task adaptation, but their scalability is severely limited by finite context windows and the growing cost of key–value (KV) caches in long multi-modal sequences. Existing memory compression approaches typically rely on rigid token removal or sample-dependent importance estimation, which introduces bias, disrupts semantic structure—particularly for visual representations—and yields static memories that cannot adapt to new queries. We introduce TASM (Task-Aware Structured Memory), a training-free framework that addresses these limitations through task-aware, structure-preserving, and dynamically accessible memory construction. TASM employs Task-Vector Guided Compression to replace sample-specific signals with a task-level direction that captures shared relevance across demonstrations. To preserve the underlying information manifold, it further applies Semantics-Aware Token Merging, formulating compression as a Bipartite Graph Matching problem that merges tokens without destructive pruning. Finally, TASM organizes compressed representations into a multi-resolution hierarchy consisting of a compact Core Memory and a Latent Bank, enabling Query-Adaptive Dynamic Activation and Dynamic Retrieval at inference time. Empirical evaluations show that TASM sustains strong multi-modal ICL performance under high compression ratios, demonstrating an effective balance between efficiency, adaptability, and semantic fidelity.

Poster
LEMUR: Learned Multi-Vector Retrieval

Elias Jääsaari, Ville Hyvönen, Teemu Roos

展开摘要

Multi-vector representations generated by late interaction models, such as ColBERT, enable superior retrieval quality compared to single-vector representations in information retrieval applications. In multi-vector retrieval systems, both queries and documents are encoded using one embedding per token, and similarity between queries and documents is measured by the MaxSim similarity measure. However, the improved quality of multi-vector retrieval comes at the expense of significantly increased search latency. In this work, we introduce LEMUR, a simple yet efficient framework for multi-vector similarity search. LEMUR consists of two consecutive problem reductions: First, we formulate multi-vector similarity search as a supervised learning problem that can be solved using a one-hidden-layer neural network. Second, we reduce inference under this model to single-vector similarity search in its latent space, enabling the use of existing single-vector search indexes to accelerate retrieval. LEMUR is an order of magnitude faster than prior multi-vector similarity search methods. Our code is available at https://github.com/ejaasaari/lemur

Poster
Shared Lexical Task Representations Explain Behavioral Variability In LLMs

Zhuonan Yang, Xiaochen Li, Francisco Piedrahita Velez, Eric Todd, David Bau, Michael Littman, Stephen Bach, Ellie Pavlick

展开摘要

One of the most common complaints about large language models (LLMs) is their prompt sensitivity---that is, the fact that their ability to perform a task or provide a correct answer to a question can depend unpredictably on the way the question is posed. We investigate this variation by comparing two very different but commonly-used styles of prompting: *instruction-based* prompts, which describe the task in natural language, and *example-based* prompts, which provide in-context few-shot demonstration pairs to illustrate the task. We find that, despite large variation in performance as a function of the prompt, the model engages some common underlying mechanisms across different prompts of a task. Specifically, we identify task-specific attention heads whose outputs literally describe the task --- which we dub *lexical task heads*---and show that these heads are shared across prompting styles and trigger subsequent answer production. We further find that behavioral variation between prompts can be explained by the degree to which these heads are activated, and that failures are at least sometimes due to competing task representations that dilute the signal of the target task. Our results together present an increasingly clear picture of how LLMs' internal representations can explain behavior that otherwise seems idiosyncratic to users and developers.

Poster
OPTION: Optimal Transport–Guided Flow Matching for Incomplete and Unaligned Multi-View Clustering

Siyuan Zhou, Zhibin Gu

展开摘要

Multi-view clustering effectively exploits rich information from multiple views, yet real-world applications are frequently challenged by missing views and cross-view sample misalignment, hindering cross-view modeling and resulting in inferior clustering performance. To address these challenges, this paper presents a novel method, **OP**timal **T**ransport–Gu**I**ded fl**O**w Matchi**N**g for incomplete and unaligned multi-view clustering (**OPTION**). Specifically, OPTION employs conditional flow matching to learn deterministic transport paths for missing-view imputation, enabling stable manifold-preserving recovery and more discriminative representations. To support alignment-free fusion, we introduce a Gromov-Wasserstein-inspired structural regularization that aligns intra-view geometric structures in the latent space without solving hard correspondences. Furthermore, an optional contrastive regularization is incorporated to enhance cross-view consistency specifically for aligned settings. Extensive experiments demonstrate that OPTION outperforms state-of-the-art methods across ideal, incomplete, and unaligned scenarios evaluated separately. Code: https://github.com/TimoZhou1024/OPTION.

Poster
TACTIC: Task-Aware Sparse Coordination Graphs for Multi-Task Multi-agent Reinforcement Learning

Kexing Peng, Pengyi Li, tinghuai ma, Jianye Hao

展开摘要

Value factorization eases non-stationarity in MARL, but its static coordination assumptions hinder generalization on long-horizon tasks with shifting dependencies. Prior VQ-VAE methods abstract trajectories yet miss time-varying inter-agent dependencies. We present TACTIC, a CTDE framework with three components: (i) VQ-VAE-based trajectory abstraction that learns discrete task-semantic classes; (ii) semantic-conditioned sparse coordination graphs that adapt dependencies by pruning edges according to variance-based pairwise payoff sensitivity; and (iii) a pretrained, frozen trajectory-class predictor that conditions local policies while decoupling task recognition from control. On SMAC and SUMO, TACTIC shows strong overall competitiveness and adaptive coordination under sparse rewards and dynamic task structures.

Poster
Incomplete Multi-View Clustering via Neighborhood-Conditioned Diffusion

Qian Guo, Gaohui Zuo, Bingbing Jiang, Guangrui Fan, Zhihua Cui, Xinyan Liang, Jianjian Ding

展开摘要

Incomplete multi-view clustering (IMVC) aims to uncover shared clustering structures from heterogeneous views with partial observations. Recently, existing generative IMVC methods have made significant progress in this field; however, they still remain limited in two aspects. On the one hand, they rely on weak cross-view signals, resulting in unstable latent recovery when facing missing data. On the other hand, they overlook stable cross-view neighborhood structures, leading to weak structural constraint. To address these limitations, we propose neighborhood-conditioned diffusion for incomplete multi-view clustering (IMVC-NCD), which achieves robust latent completion. Our method learns compact view-specific latent representations and constructs a unified conditioning vector by aggregating stable local neighborhood structures from available views while encoding missingness states, providing reliable guidance for diffusion-based denoising. With neighborhood-level conditioning, IMVC-NCD produces semantically aligned and view-consistent latent representations that are well suited for clustering, even under high missing-view ratios. Extensive experiments on four benchmark datasets demonstrate the effectiveness and robustness of our method compared with state-of-the-art IMVC approaches. Our code is available at https://github.com/zgh1115/IMVC-NCD.

Poster
Is Task-Specific Training Necessary for Anomaly Detection?

Xingwu Zhang, Guanxuan Li, Paul Henderson, Gerardo Aragon-Camarasa, ZIJUN LONG

展开摘要

Current state-of-the-art multi-class unsupervised anomaly detection (MUAD) methods rely on training encoder--decoder models to reconstruct anomaly-free features. However, we argue that such task-specific training is costly under distribution shifts, and that reconstruction-based residual scoring further faces a fidelity--stability dilemma. Existing training-free alternatives, in turn, remain prone to cross-category and cross-region mismatches in MUAD. Motivated by these limitations, we propose Retrieval-based Anomaly Detection (RAD), a task-specific training-free framework that stores anomaly-free features in a memory and detects anomalies through multi-level retrieval, matching test patches against the memory. Experiments demonstrate that RAD achieves state-of-the-art performance across four established benchmarks (MVTec-AD, VisA, Real-IAD, 3D-ADAM) under both standard and few-shot settings. On MVTec-AD, RAD reaches 96.7% Pixel AUROC with just a single anomaly-free image compared to 98.5% of RAD's full-data performance. Collectively, these findings overturn the assumption that MUAD requires task-specific training, showing that state-of-the-art anomaly detection is feasible with training-free memory-based retrieval. Our code is available at https://github.com/longkukuhi/RAD.

Poster
Masked Multi-path Contrast with Confidence-Gated Semantic Imputation for Incomplete Multi-view Clustering

Fan Yang, Haikun Xu

展开摘要

Incomplete multi-view clustering (IMVC) becomes particularly challenging under heavy missingness and view-availability imbalance. In this regime, scarce co-observed pairs make cross-view correspondences unreliable. Imputation-first pipelines may trigger cascading reconstruction errors, while purely consistency-based alignment often degrades sharply and gives limited control over semantic convergence across views. We propose MAGIC (Masked multi-pAth contrast with confIdence-Gated semantIc imputation), a unified framework that learns calibrated cluster semantics before conservative completion. MAGIC builds multiple correlated representation and prediction paths from lightly augmented latent codes, and couples them with a masked multi-path contrastive consensus objective and prediction-consistency regularization. The resulting posteriors are aggregated into view-wise soft assignments to reduce overconfidence and alleviate dominance by more frequently observed views. Based on these calibrated semantics, MAGIC performs similarity-guided semantic transfer in label space with confidence-aware gating, and completes missing representations through a geometry-preserving prototype fallback. Experiments on four benchmarks across different missing ratios show consistent gains over prior IMVC methods, and the ablations support the roles of masked multi-path consensus learning and confidence-gated semantic imputation.

Poster
Cross-Tactile Sensor Representation Learning

Yan Zhang, Zheng Wang, Pengpeng Zeng, Xing Xu, Jingkuan Song, Heng Tao Shen

展开摘要

Visuo-tactile sensors have been widely adopted in robotic manipulation. However, inherent heterogeneity in sensor designs hinders the learning of unified tactile representations in cross-sensor scenarios. Existing methods that focus on reconstruction or task-specific supervision often fail to capture the common information between different tactile sensors, particularly in the presence of substantial sensor variations, resulting in limited generalization to unseen sensors. To address this, we propose Cross-Tactile Sensor Representation Learning (CTSRL), a unified framework for sensor-agnostic tactile representation learning. CTSRL introduces a Cross-Sensor Modulator (CSM) to eliminate sensor-specific biases and adopts a two-stage learning paradigm: (1) leveraging aligned synthetic data for cross-sensor self-supervised learning to extract shared latent representations across sensor domains; and (2) integrating real-world multimodal tactile data to bridge the sim-to-real semantic gap through cross-modal alignment, thereby enriching representations with fine-grained semantic attributes. Experimental results show that our method demonstrates strong multi-sensor generalization, significantly improving sensor-agnostic representation learning.

Poster