ICML 2026 论文列表

Continual learning requires models to mitigate catastrophic forgetting of prior knowledge while learning a sequence of tasks. Although existing methods based on orthogonal projection prevent interference by constraining parameter updates, they tend to limit plasticity as the task sequence progresses. The reliance on the linear approximation further causes the projected gradients to deviate from the nonlinear manifold. To address these issues, we propose Metabolic Optimization for Continual Learning (MoCL), a rehearsal-free framework that strikes a balance between stability and plasticity. To capture the geometric manifold of prior knowledge, MoCL introduces a factorized subspace approximation that avoids expensive explicit matrix inversion. Given the heavy-tailed distribution of the Fisher Information Matrix, we employ a metabolic gating based on Tsallis entropy to suppress updates that conflict with historical knowledge. Theoretical and empirical analyses show that MoCL suppresses interference while supporting shared low-loss behavior across sequential tasks. Extensive experimental results across multiple benchmarks demonstrate that MoCL outperforms state-of-the-art methods in both classification performance and efficiency.

Just as LEGO pieces can be assembled into an unlimited variety of structures, heterogeneous federated learning (HFL) can be viewed as the assembly of diverse model components. Inspired by this analogy, we reformulate HFL as a LEGO-like assembly game. The central challenge in HFL lies in learning across heterogeneous model architectures, which hinders direct parameter sharing. To address this challenge, we propose to decompose models into a set of modular components—analogous to LEGO pieces and collaboratively learn these components across clients under predefined composition rules. Based on this perspective, we develop a novel federated learning framework, termed LEGO-FL, which enables flexible model construction while preserving collaborative learning. Extensive experiments validate the effectiveness of LEGO-FL under different heterogeneous settings and system scales.

Manifold clustering has demonstrated strong capability in capturing complex data structures and has been widely studied in cluster analysis. However, many existing methods mainly focus on combining K-means with manifold learning, while overlooking the consistency between data structures and clustering labels, and often suffer from high computational cost when handling large scale data. To address these issues, we propose a manifold balanced clustering method based on anchor induced distance(LMBC), grounded in the relationship between K-means clustering and manifold learning. Specifically, the LMBC uses label information to guide the construction of the manifold structure, thereby ensuring consistency between data structures and clustering labels. To enable large scale clustering, we introduce an anchor induced distance representation that models manifold structure in a compact anchor space, significantly reducing computational complexity while preserving essential structural information. Furthermore, to naturally maintain class balance during clustering, we maximize the Schatten-p norm of the label representation and provide theoretical analysis to support its effectiveness. Experimental results on several benchmark datasets demonstrate the effectiveness and scalability of the proposed method.

Multi-vector representations generated by late interaction models, such as ColBERT, enable superior retrieval quality compared to single-vector representations in information retrieval applications. In multi-vector retrieval systems, both queries and documents are encoded using one embedding per token, and similarity between queries and documents is measured by the MaxSim similarity measure. However, the improved quality of multi-vector retrieval comes at the expense of significantly increased search latency. In this work, we introduce LEMUR, a simple yet efficient framework for multi-vector similarity search. LEMUR consists of two consecutive problem reductions: First, we formulate multi-vector similarity search as a supervised learning problem that can be solved using a one-hidden-layer neural network. Second, we reduce inference under this model to single-vector similarity search in its latent space, enabling the use of existing single-vector search indexes to accelerate retrieval. LEMUR is an order of magnitude faster than prior multi-vector similarity search methods. Our code is available at https://github.com/ejaasaari/lemur

When designing interventions to promote desired actions, two-stage agent heterogeneity -- encompassing both engagement with the intervention and completion of the desired action -- creates significant challenges in identifying optimal intervention policies. While this two-dimensional heterogeneity creates distinct agent response types with varying marginal policy returns, existing literature typically falls short in full identification of all agent types, leading to inefficient intervention allocations. To address the challenge of learning optimal policies that account for two-stage outcomes, we propose a minimax approach within a counterfactual principal strata framework. A value function, accommodating varying policy returns across six potentially non-identifiable principal strata, is designed and partially identified to minimize the worst-case value loss relative to three benchmark policies: never-treat, always-treat, and oracle. We introduce three estimators for optimal policy learning: Principal Outcome Regression (P-OR), Principal Inverse Propensity Scoring (P-IPS), and Principal Doubly Robust (P-DR), providing theoretical guarantees for their unbiasedness, robustness, and regret upper bounds. Extensive numerical experiments demonstrate the effectiveness and superiority of the proposed approach.

Multi-view clustering effectively exploits rich information from multiple views, yet real-world applications are frequently challenged by missing views and cross-view sample misalignment, hindering cross-view modeling and resulting in inferior clustering performance. To address these challenges, this paper presents a novel method, **OP**timal **T**ransport–Gu**I**ded fl**O**w Matchi**N**g for incomplete and unaligned multi-view clustering (**OPTION**). Specifically, OPTION employs conditional flow matching to learn deterministic transport paths for missing-view imputation, enabling stable manifold-preserving recovery and more discriminative representations. To support alignment-free fusion, we introduce a Gromov-Wasserstein-inspired structural regularization that aligns intra-view geometric structures in the latent space without solving hard correspondences. Furthermore, an optional contrastive regularization is incorporated to enhance cross-view consistency specifically for aligned settings. Extensive experiments demonstrate that OPTION outperforms state-of-the-art methods across ideal, incomplete, and unaligned scenarios evaluated separately. Code: https://github.com/TimoZhou1024/OPTION.

The attention mechanism with softmax normalisation is a foundational component of Transformer-based large language models. However, with very long contexts, attention scores are known to diminish, raising fundamental questions about token distinguishability and how it can be preserved. In this work, we provide a formal characterisation of token distinguishability in attention as a function of context length and embedding dimension. We introduce Aitchison distance to quantify relative differences among attention probabilities, and show that, with Gaussian queries and keys, even in the long-context regime, token distinguishability converges to a finite, non-zero limit rather than vanishing. Leveraging the linear relationship between inverse-temperature scaling and Aitchison distance, we derive a theoretical lower bound of $\Omega(\sqrt{\log L})$ on the logit scaling required to produce a sharp attention distribution. Finally, we demonstrate that Aitchison distance provides a principled and practical alternative to entropy for monitoring training and inference, as it captures the full compositional structure, including the smaller components of the attention probabilities.

Speculative decoding can significantly accelerate LLM inference, especially given that its cloud-edge collaborative deployment offers cloud workload offloading, offline robustness, and privacy enhancement. However, existing collaborative inference frameworks with speculative decoding are constrained by (i) sequential token generation and communication with low resource utilization, and (ii) inflexible cloud non-autoregressive verification (NAV) triggering that induces premature verification or costly rollbacks. In this paper, we propose PipeSD, an efficient cloud-edge collaborative pipeline inference framework with speculative decoding. PipeSD overlaps token generation and communication by a token-batch pipeline scheduling mechanism optimized by dynamic programming, and improves verification flexibility through a dual-threshold NAV triggering mechanism with a lightweight Bayesian optimization autotuner. We implement PipeSD using llama-cpp-python, PyTorch, and FastAPI, and evaluate it on a real-world cloud-edge testbed with two draft-target model pairs across four scenarios. Results show that PipeSD consistently outperforms state-of-the-art baselines, achieving $1.16\times$-$2.16\times$ speedup and reducing energy consumption by $14.3$%-$25.3$%. Our code is available at https://github.com/Ghanyunhe/PipeSD.

The Tubal Nuclear Norm (TNN), derived from the tensor Singular Value Decomposition (t-SVD), is a widely used low-rank modeling tool that promotes sparsity of frequency-domain singular values. However, as a direct extension of the matrix nuclear norm, TNN applies a uniform element-wise penalty to transformed singular values, without explicitly distinguishing sparsity across frequency components from low-rankness within each component. This can be restrictive for real-world tensor data that exhibit multi-level spectral structures, where spectral energy is concentrated in a subset of frequency components while active components remain low-rank. To overcome this limitation, we propose the tensor $\ell_p$-Schatten-$q$ quasi-norm ($p,q\in(0,1]$), which enables explicit control of dual spectral sparsity by jointly regularizing inter-frequency sparsity and intra-frequency low-rankness. This formulation includes TNN as a special case and subsumes several existing tensor regularizers by coupling global frequency sparsity with local spectral low-rankness, yielding a more flexible modeling principle. We establish minimax error bounds under the proposed dual spectral sparsity model, develop a reweighted optimization algorithm for the resulting nonconvex problem, and demonstrate its effectiveness and robustness on noisy and Poisson tensor completion as well as image clustering tasks.

Autoregressive learning of time-stepping operators provides an effective approach to data-driven partial differential equation (PDE) simulation, yet for conservation laws, they face a fundamental challenge: learned updates may violate global conservation over long rollouts. For the important subclass of mass-conservation-type equations, the problem is compounded by inherent physical bounds (e.g., nonnegativity or concentrations in [0,1]) whose violation further destabilizes predictions. We introduce FluxNet, which learns cumulative transport amounts representing the total conserved quantity redistributed between each cell and a configurable neighborhood over the full surrogate interval. A conservative update guarantees exact discrete conservation by construction; modular capacity-constrained transport heads (L, U, and D) enforce lower bounds, upper bounds, or near-zero dual-bound violations through architectural design. Unlike flux-rate surrogates that require temporal integration and thus inherit CFL constraints, FluxNet involves no such integration; configurable transport neighborhoods enable large-timestep prediction at full spatial resolution. Ghost cells extend the framework to non-periodic boundaries. Experiments on four benchmarks (1D convection--diffusion, 2D shallow water, 1D traffic flow, 2D Cahn--Hilliard) demonstrate exact conservation, structural bound preservation, architecture modularity, and superior stability over flux-rate surrogates at large temporal strides. The code is publicly available at: https://github.com/Lan-zs/FluxNet.

How can we generate samples from a conditional distribution that we never fully observe? This question arises across a broad range of applications in both modern machine learning and classical statistics, including image post-processing in computer vision, approximate posterior sampling in simulation-based inference, and conditional distribution modeling in complex data settings. In such settings, compared with unconditional sampling, additional feature information can be leveraged to enable more adaptive and efficient sampling. Building on this, we introduce Conditional Generator using MMD (CGMMD), a novel framework for conditional sampling. Unlike many contemporary approaches, our method frames the training objective as a simple, adversary-free direct minimization problem. A key feature of CGMMD is its ability to produce conditional samples in a single forward pass of the generator, enabling practical one-shot sampling with low test-time complexity. We establish rigorous theoretical bounds on the loss incurred when sampling from the CGMMD sampler, and prove convergence of the estimated distribution to the true conditional distribution. In the process, we also develop a uniform concentration result for nearest-neighbor based functionals, which may be of independent interest. Finally, we show that CGMMD performs competitively on synthetic tasks involving complex conditional densities, as well as on practical applications such as image denoising and image super-resolution.

Federated continual learning (FCL) of large language models on edge devices is constrained by a communication--stability--plasticity trilemma. We reveal a less-is-more phenomenon: beyond a moderate layer upload ratio, stability loss offsets saturated plasticity gains, so overall continual performance no longer improves. Moreover, layer-wise conflict is heavy-tailed and concentrates in a few layers; denser uplink increasingly includes these layers, which disproportionately drives forgetting and motivates selective sparse communication. Therefore, we introduce RieSelect, which treats stability as staying within a Fisher-metric safe basin around historical solutions. Under this safe-basin constraint, we derive a layer-wise conflict score and a closed-form certified safe step size for finite local updates, and formulate selective uplink as a knapsack-based utility--risk selection, balancing plasticity gains against stability risks. Extensive experiments show that, under a per-round uplink budget, RieSelect achieves the best performance across task orders. Beyond this matched-budget setting, under standard communication protocols, RieSelect improves average accuracy by 18.99–28.14 points while reducing total uplink by 53–115$\times$.

We present a unified framework for quantifying the similarity between representations through the lens of \textit{usable} information, offering a rigorous theoretical and empirical synthesis across three key dimensions. First, addressing functional similarity, we establish a formal link between stitching performance and conditional mutual information. We further reveal that stitching is inherently asymmetric, demonstrating that robust functional comparison necessitates a bidirectional analysis rather than a unidirectional mapping. Second, concerning representational similarity, we find that reconstruction-based metrics and standard tools (e.g., CKA, RSA) act as estimators of usable information under specific constraints. Crucially, we show that similarity is relative to the capacity of the predictive family: representations that appear distinct to a rigid observer may be identical to a more expressive one. Third, we demonstrate that representational similarity is sufficient but not necessary for functional similarity. We unify these concepts through a task-granularity hierarchy: similarity on a complex task guarantees similarity on any coarser derivative, establishing representational similarity as the limit of maximum granularity: input reconstruction.

Large language models (LLMs) increasingly act as autonomous agents, using tools to execute code, read and write files, and access networks, creating novel security risks. To mitigate these risks, agents are commonly deployed and evaluated in isolated "sandbox" environments, often implemented using Docker/OCI containers. We introduce SandboxEscapeBench, an open benchmark that safely measures an LLM's capacity to break out of these sandboxes. The benchmark is implemented as an Inspect AI Capture the Flag (CTF) evaluation utilising a nested sandbox architecture with the outer layer containing the flag and no known vulnerabilities. Following a threat model of a motivated adversarial agent with shell access inside a container, SandboxEscapeBench covers a spectrum of sandbox-escape mechanisms spanning misconfiguration, privilege allocation mistakes, kernel flaws, and runtime/orchestration weaknesses. We find that, when vulnerabilities are added, LLMs are able to identify and exploit them, showing that use of evaluation like SandboxEscapeBench is needed to ensure sandboxing continues to provide the encapsulation needed for highly-capable models.

We introduce Unsupervised Partner Design (UPD), a population-free multi-agent reinforcement learning method for robust ad-hoc teamwork. UPD generates training partners on-the-fly and selects them adaptively based on a learnability criterion, removing the need for pre-trained partner populations or manual parameter tuning. We show that this simple mechanism enables effective partner diversity and can be extended to joint partner-environment selection when a procedural level generator is available. Across Level-Based Foraging, Overcooked-AI, and the Overcooked Generalisation Challenge, UPD consistently achieves strong performance compared to both population-based and population-free baselines. In a human-AI user study, agents trained with UPD achieve higher returns and are rated as more adaptive, more human-like, and less frustrating than all evaluated baseline methods.

Likelihood-based training is the dominant paradigm in survival prediction. Under independent censoring, we can factorize the likelihood and optimize only the terms related to event modeling, effectively treating the censoring mechanism as incidental. This is justified when censoring is *non-informative*, i.e., when the censoring process shares no parameters with the event-time model. However, this may not hold in practice, and ignoring censoring contributions may discard useful signals for learning representations that can help to effectively estimate event distributions. Motivated by this, we argue that explicitly modeling censoring can improve representation learning and time-to-event estimation, particularly when event and censoring processes are coupled. We introduce a latent decomposition view in which observed covariates are mapped to latent components corresponding to event-specific, censoring-specific, confounding, and irrelevant information. We then learn decomposed representations for the first three categories to guide a better estimation of the event distribution. We instantiate our method on 4 popular deep-learning survival models and evaluate on 10 datasets (2 semi-synthetic and 8 real-world), showing consistent gains over strong baselines and multiple SOTA methods.

Causal machine learning aims to answer "what if" questions using machine learning algorithms, making it a promising tool for high-stakes decision-making. Yet, empirical evaluation practices remain limited. Existing benchmarks often rely on a handful of hand-crafted or semi-synthetic datasets, leading to brittle, non-generalizable conclusions. To bridge this gap, we introduce CausalProfiler, a synthetic benchmark generator for causal machine learning methods. Based on a set of explicit design choices about the class of causal models, queries, and data considered, CausalProfiler randomly samples causal models, data, queries, and ground truths constituting the synthetic causal benchmarks. In this way, causal machine learning can be rigorously and transparently evaluated under a variety of conditions. This work offers the first random generator of synthetic causal benchmarks with coverage guarantees and transparent assumptions operating on the three levels of causal reasoning: observation, intervention, and counterfactual. We demonstrate its utility by evaluating several state-of-the-art methods under diverse conditions and assumptions, both in and out of the identification regime, illustrating the types of analyses and insights CausalProfiler enables.

Incomplete multi-view clustering (IMVC) aims to uncover shared clustering structures from heterogeneous views with partial observations. Recently, existing generative IMVC methods have made significant progress in this field; however, they still remain limited in two aspects. On the one hand, they rely on weak cross-view signals, resulting in unstable latent recovery when facing missing data. On the other hand, they overlook stable cross-view neighborhood structures, leading to weak structural constraint. To address these limitations, we propose neighborhood-conditioned diffusion for incomplete multi-view clustering (IMVC-NCD), which achieves robust latent completion. Our method learns compact view-specific latent representations and constructs a unified conditioning vector by aggregating stable local neighborhood structures from available views while encoding missingness states, providing reliable guidance for diffusion-based denoising. With neighborhood-level conditioning, IMVC-NCD produces semantically aligned and view-consistent latent representations that are well suited for clustering, even under high missing-view ratios. Extensive experiments on four benchmark datasets demonstrate the effectiveness and robustness of our method compared with state-of-the-art IMVC approaches. Our code is available at https://github.com/zgh1115/IMVC-NCD.

We study infinite-horizon discounted Markov decision processes (DMDPs) under a generative model. Motivated by the Algorithms with Advice framework (Mitzenmacher and Vassilvitskii, 2022), we propose a novel framework to investigate how black-box predictions of the transition matrix can enhance sample efficiency in solving DMDPs and improve sample complexity bounds. We focus on DMDPs with $N$ state–action pairs and discount factor $\gamma$. We first provide an impossibility result showing that, in the presence of predictions with unknown accuracy, no sampling policy can compute an $\epsilon$-optimal policy with a sample complexity better than $\tilde{O}((1-\gamma)^{-3} N \epsilon^{-2})$, which matches the state-of-the-art minimax sample complexity bound without prediction. In complement, we design an algorithm based on minimax optimization techniques that leverages predictions of the transition matrix without requiring knowledge of the prediction error. Our algorithm achieves a sample complexity bound that depends on the prediction error and is uniformly better than $\tilde{O}((1-\gamma)^{-4} N \epsilon^{-2})$, the previous best result derived from convex optimization methods. In some cases, our bound even improves upon the state-of-the-art $\tilde{O}((1-\gamma)^{-3} N \epsilon^{-2})$, despite not having access to the prediction quality.