ICML 2026 Papers

Diffusion models have achieved remarkable success in graph generation, but enforcing hard constraints on generated graphs remains challenging, limiting their deployment in constraint-critical applications. Existing approaches either fail to guarantee strict constraint satisfaction or are limited to narrow constraint types, lacking the flexibility to handle diverse constraint specifications. To address this challenge, we exploit the discrete structure of graphs, which allows hard constraints to be formulated as symbolic reasoning problems. Building on this insight, we propose NSPSG, a framework that integrates unconstrained diffusion models with discrete projection operators. NSPSG employ an SMT (Satisfiability Modulo Theories)-based projector to ensure that the generated graphs strictly satisfy constraints while remaining within the training data distribution. To further accelerate generation, we employ a supervised auto-regressive neural projector to approximate the symbolic reasoning process. Across heterogeneous constraints and various graph generation datasets, NSPSG achieves 99%-100% validity rates, demonstrating state-of-the-art performance. Notably, for a complex non-linear constraint, it improves data validity by up to 43% and reaches 99% validity while maintaining comparable generation efficiency.

Continual learning (CL) aims to acquire new knowledge from a non-stationary data stream while retaining performance on previously learned tasks. Memory-based replay methods mitigate catastrophic forgetting by storing and revisiting past samples, but their effectiveness is fundamentally constrained by limited memory capacity, as each stored example represents only a single data instance. In this work, we propose data reassembly for CL, a new paradigm that significantly increases memory efficiency by reassembling composite replay samples from existing training data. Instead of storing raw training examples, we partition the current task training data into elementary patches and dynamically reassemble them into coherent replay instances through an energy-based optimization framework. The proposed objective jointly enforces semantic compatibility with target labels and global consistency among assembled patches. To make this optimization tractable, we derive an efficient variational inference algorithm that constructs a compact yet diverse set of reassembled samples for replay. Extensive theoretical analysis and experiments across multiple CL benchmarks demonstrate that data reassembly consistently outperforms existing memory-based approaches, achieving stronger retention of past knowledge while maintaining competitive computational efficiency.

Reinforcement learning often produces high-frequency oscillatory control signals that undermine the safety and stability required for physical deployment. Explicit action chunking addresses this by predicting fixed-horizon trajectories but scales the policy output dimension proportionally with the horizon length, leading to optimization difficulties and incompatibility with standard step-wise interaction. To overcome these challenges, this paper proposes Dual-Window Smoothing (DWS), an implicit action chunking framework for smooth continuous control. Unlike explicit methods, DWS enforces temporal coherence without expanding the action space. It uses a dual-window design: an execution window that ensures physical smoothness through deterministic modulation, and a value window that aligns temporal-difference targets over the horizon to correct critic bias caused by open-loop execution. DWS also includes a lightweight actor-side temporal regularizer based on first-order action differences to promote global continuity. This design effectively bridges the gap between temporal abstraction and reactive step-wise control. Experiments on benchmarks including the DeepMind Control Suite and industrial energy management tasks show that DWS outperforms state-of-the-art (SOTA) baselines. In complex vision-based autonomous driving tasks, DWS achieves smoother control, safer behavior with reduced jitter, and attains a 100% success rate.

This work demonstrates how increasing the number of neurons in a network without increasing its total number of non-zero parameters improves performance. We show that this gain corresponds with a decrease in interference between multiple features that would otherwise share the same neurons. On symbolic Boolean tasks, splitting each neuron into sparser sub-neurons with knowledge of the clauses systematically reduces polysemanticity metrics and yields higher task accuracy. Notably, even random splits of neuron weights approximate these gains, indicating that reduced collisions, not precise assignment, are a primary driver. Consistent with the superposition hypothesis, the benefits of this framework grow with increasing interference: when polysemantic load is high, accuracy improvements are the largest. Transferring these insights to more realistic models, including classifiers over CLIP embeddings, convolutional neural networks, and deeper multilayer networks, we find that widening networks while maintaining a constant non-zero parameter count consistently increases accuracy. These results identify an interpretability-grounded mechanism to leverage width against superposition, improving performance without increasing the number of non-zero parameters. Such a direction is well matched to modern accelerators, where memory movement of non-zero parameters, rather than raw compute, is often a dominant bottleneck.

Graph learning has been increasingly deployed in critical and sensitive domains, raising pressing demands for trustworthiness-robustness, fairness, and beyond. However, these properties are often undermined by various perturbations, which induce distributional uncertainty and compromise the trustworthiness of graph learning. To address this, we propose DICT, a novel framework that models distributional uncertainty to achieve trustworthy graph learning. Specifically, DICT formulates a unified optimization objective that captures perturbation-induced distributional shifts in graph topology, node features, and labels, and minimizes the worst-case risk over the uncertainty set. However, directly optimizing this objective in its primal form leads to an infinite-dimensional problem. To make this problem tractable, we integrate strong duality and local Lipschitz continuity of the loss, reformulating the objective as a finite-dimensional min-max problem. We focus on robustness and fairness as primary instantiations of DICT because they are not only critical in real-world applications, but also provide transferable modeling principles for broader trustworthiness objectives. By formulating fairness in the form of an uncertainty set, DICT pioneers unified robustness and fairness within a single optimization framework. Extensive experiments across diverse benchmarks and backbones demonstrate that DICT consistently improves both robustness and fairness, validating the effectiveness and adaptability of the DICT framework.

Federated causal discovery for time series is becoming increasingly important in many application domains. In practice, intervention policies on each client often change over time, causing the local underlying causal mechanisms to drift rather than remain fixed. Moreover, different sampling frequencies across clients yield incompatible time scales in the observed data, making the resulting local causal graphs naturally heterogeneous and difficult to aggregate consistently. Accordingly, we propose Fed-CAW, a Credibility-Aware Weighting Federated causal discovery framework for time series. Specifically, we define edge-level credibility scores that quantify per-edge reliability by summarizing (i) within-client temporal stability across windows and (ii) cross-client temporal consistency after mapping heterogeneous sampling frequencies onto a unified time scale. We then aggregate privatized edge statistics under differential privacy, treating credibility scores as weights to recover a global causal graph while preserving personalized local structures without sharing raw data. Theoretically, we demonstrate the rationale for the unified time scale mapping and establish rigorous differential privacy guarantees. Experimental results on synthetic and real-world datasets demonstrate the effectiveness of our proposed method.

Evaluating generative AI models is increasingly resource-intensive due to slow inference, expensive raters, and a rapidly growing landscape of models and benchmarks. We propose ProEval, a proactive evaluation framework that leverages transfer learning to efficiently estimate performance and identify failure cases. ProEval employs pre-trained Gaussian Processes (GPs) as surrogates for the performance score function, mapping model inputs to metrics such as the severity of errors or safety violations. By framing performance estimation as Bayesian quadrature (BQ) and failure discovery as superlevel set sampling, we develop uncertainty-aware decision strategies that actively select or synthesize highly informative inputs for testing. Theoretically, we prove that our pre-trained GP-based BQ estimator is unbiased and bounded. Empirically, extensive experiments on reasoning, safety alignment, and classification benchmarks demonstrate that ProEval is significantly more efficient than competitive baselines. It requires 8–65x fewer samples to achieve estimates within $\pm1\%$ of the ground truth, while simultaneously revealing more diverse failure cases under a stricter evaluation budget. Our open-sourced code and data can be found at https://github.com/google-deepmind/proeval.

Incomplete multi-view multi-label classification addresses scenarios where views and labels are partially missing. While existing methods treat labels solely as supervision signals, they overlook the semantic structure inherent in partial annotations. We propose Label-Guided Representation Learning (LGRL) that systematically exploits label semantics as structural priors throughout learning. Our framework constructs a semantic-informed mixture prior via learnable category prototypes to guide representation extraction, and introduces category-specific conditional posteriors where prototypes act as Bayesian experts for multi-view fusion. We further derive a principled label-driven information bottleneck objective balancing reconstruction sufficiency with cross-view consistency, enabling category-conditional reasoning. Extensive experimental results demonstrate the effectiveness of LGRL across benchmark datasets as well as real-world applications in sports analytics and medical imaging.

Reverse Chain-of-Thought Generation (RCG) synthesizes reasoning traces from query-answer pairs, but it risks producing post-hoc rationalizations: when models can see the answer during generation, the answer serves as a cognitive anchor that shapes the entire explanation. We formalize this phenomenon through a three-level measurement hierarchy: lexical, entropic, and probabilistic anchoring, which capture surface artifacts, entropy dynamics, and latent answer dependence, respectively. We analyze semantic suppression, the intuitive mitigation strategy that instructs models to ignore the answer, and find that it is counterproductive: while it reduces lexical overlap, it paradoxically increases entropic and probabilistic anchoring. We attribute this failure to active monitoring of the forbidden answer, which inadvertently deepens dependence on it. To break this cycle, we propose Structural Skeleton-guided Reasoning (SSR), whose core contribution is to replace answer suppression with structural decoupling: SSR first generates an answer-invariant functional skeleton and then uses it as a content-neutral target for full trace generation. Experiments across open-ended reasoning benchmarks show that SSR consistently mitigates anchoring, and that Distilled SSR (SSR-D), a distillation variant that internalizes skeleton-guided reasoning from teacher-generated traces, achieves up to 10\% improvement over suppression baselines while mitigating out-of-distribution (OOD) degradation.

Multi-modal neuroimaging-based psychiatric diagnosis must integrate cross-modal agreement with modality-specific complementarity, yet in real multi-site cohorts these signals are frequently entangled with site- and cohort-dependent correlations, yielding shortcut-driven predictions and limited interpretability. We propose Structured Multi-modal Graph Disentanglement (SMGD), which explicitly factorizes multi-modal graph representations into four components with distinct roles: shared diagnostic evidence, complementary diagnostic evidence, incidental cross-modal agreement, and modality-specific non-robust correlations, with the former two forming the diagnostic core and the latter two suppressed as shortcuts. SMGD is realized as geometry-driven structure learning: under a mild distributional assumption, we develop mini-batch estimable surrogate regularizers that shape subspace organization and cross-modal relations, enforcing semantic consistency through relational geometry rather than centroid coincidence while suppressing confounded dependencies. Experiments on large multi-site datasets show improved in-domain diagnosis and more reliable cross-dataset generalization in the presence of a Modality Gap, without relying on expert-crafted diagnostic biomarkers.

Neural simulators promise efficient surrogates for physics simulation, but scaling them is bottlenecked by the prohibitive cost of generating high-fidelity training data. Pre-training on abundant off-the-shelf geometries offers a natural alternative, yet faces a fundamental gap: supervision on static geometry alone ignores dynamics and can lead to negative transfer on physics tasks. We present GeoPT, a unified pre-trained model for general physics simulation based on lifted geometric pre-training. The core idea is to augment geometry with synthetic dynamics, enabling dynamics-aware self-supervision without physics labels. Pre-trained on over one million samples, GeoPT consistently improves industrial-fidelity benchmarks spanning fluid mechanics for cars, aircraft, and ships, and solid mechanics in crash simulation, reducing labeled data requirements by 20-60% and accelerating convergence by 2$\times$. These results show that lifting with synthetic dynamics bridges the geometry-physics gap, unlocking a scalable path for neural simulation and potentially beyond. Code is available at https://github.com/Physics-Scaling/GeoPT.

Understanding cellular phenotypes and how they respond to perturbations is critical for disease biology and therapeutic design. Single-cell RNA sequencing enables characterization at cellular resolution, yet the combinatorial space of conditions makes exhaustive experimental mapping infeasible. We introduce single-cell Concept Bottleneck Generative Models (scCBGM), a framework for interpretable and precise counterfactual editing of individual cells. scCBGM adapts concept bottleneck architectures for single-cell data through decoder skip connections and a cross-covariance penalty that promotes disentanglement without dimensional constraints. We extend the framework to flow matching models, enabling concept-guided editing in both encoding-decoding and generation regimes. To enable rigorous evaluation, we develop a synthetic benchmark with ground-truth counterfactuals. Across multiple real datasets, scCBGM demonstrates superior performance in combinatorial generalization and counterfactual prediction, supported by cell-level validation on synthetic data and population-level benchmarks on real datasets.

Tool-use agents that address day-to-day digital tasks such as ordering groceries must not only operate applications, but also interact with the user, e.g., to ask clarification questions, prompt for confirmation, and inform the user when the instruction is infeasible. However, current benchmarks for evaluating agent-user interactions do not capture the diversity of such interactions. Further, they operate in small environments with few, often non-state-changing, APIs. To address this gap, we introduce AppWorld-UL, a ``user-in-the-loop'' benchmark of 516 challenging tasks requiring diverse agent-user interactions. Building upon the AppWorld framework with 9 popular simulated apps like Amazon and Spotify, we systematically modify original tasks to introduce ambiguities and constraints that necessitate various types of agent-user interaction. User behavior is simulated by an LLM prompted to respond with carefully designed knowledge boundaries, offering more reliable simulation than the unconstrained or overly rigid alternatives used in prior work. Our evaluation reveals that a state-of-the-art LLM, Claude Opus 4.7, achieves only 48.6\% success on AppWorld-UL, and only 35.7\% on the harder, compositional subset. On the stricter, scenario-level metric, compositional task performance drops to only 21.3\%. Our analysis reveals that correct user-interaction is crucial for success. This demonstrates the benchmark's difficulty and its potential to advance research on user-in-the-loop tool-use agents.

Recent advances in large-scale diffusion models have intensified concerns about their potential misuse, particularly in generating realistic yet harmful or socially disruptive content. This challenge has spurred growing interest in effective machine unlearning, the process of selectively removing specific knowledge or concepts from a model without compromising its overall generative capabilities. Among various approaches, Low-Rank Adaptation (LoRA) has emerged as an effective and efficient method for fine-tuning models toward targeted unlearning. However, LoRA-based methods often exhibit limited adaptability to concept semantics and struggle to balance removing closely related concepts with maintaining generalization across broader meanings. Moreover, these methods face scalability challenges when multiple concepts must be erased simultaneously. To address these limitations, we introduce UnHype, a framework that incorporates hypernetworks into single- and multi-concept LoRA training. The proposed architecture can be directly plugged into Stable Diffusion as well as modern flow-based text-to-image models, where it demonstrates stable training behavior and effective concept control. During inference, the hypernetwork dynamically generates adaptive LoRA weights based on the CLIP embedding, enabling more context-aware, scalable unlearning. We evaluate UnHype across several challenging tasks, including object erasure, celebrity erasure, and explicit content removal, demonstrating its effectiveness and versatility.

We study bilinear bandits with partially observable features on both the user and item sides. In each of $T$ rounds, the learner selects a user--item pair and observes only the reward for the chosen pair. The reward model is bilinear in the user and item features with an unknown parameter matrix. Existing work commonly reduces this problem to a linear bandit via Kronecker-product features, increasing dimensionality and losing bilinear structure. We propose \texttt{BiRoLF}, an algorithm robust to latent features, which directly leverages the bilinear structure without such linearization. It augments observed feature spaces on both sides with orthogonal complement bases and employs doubly robust (DR) estimation to impute rewards for unselected pairs, constructing matrix-valued pseudo-rewards. We estimate the effective parameter using a bilinear DR-Lasso estimator, which promotes sparsity in components orthogonal to observed features. \texttt{BiRoLF} achieves a $\tilde{O}(\sqrt{(d_x + d_{h_x})(d_y + d_{h_y}) T})$ regret bound, where $d_x$ and $d_y$ are observable feature dimensions, and $d_{h_x}$ and $d_{h_y}$ denote effective orthogonal-complement dimensions. By exploiting the induced block-diagonal Gram structure, \texttt{BiRoLF} performs an exact blockwise update that preserves the full DR-Lasso objective. Numerical experiments show strong regret performance and computational gains.

Large Language Models with Chain-of-Thought reasoning capabilities represent valuable intellectual property, yet existing black-box watermarking methods often trade robustness for reasoning fidelity by perturbing final answers or relying on fragile trigger patterns. We propose BiCoT, a watermarking framework that embeds ownership signals into the internal geometry of reasoning traces by aligning high-saliency structural anchors with a private signature subspace while regularizing ordinary control tokens to preserve semantic capacity. This design couples the watermark with reasoning-relevant representations, making removal difficult without disrupting the features that support coherent reasoning. To enable verification under model theft and representation drift, we introduce Robust Subspace Registration(RSR), a Top-$k$ logprob-based black-box verifier that uses sentinel tokens to calibrate systematic shifts in the output distribution. Experiments show that BiCoT preserves reasoning fidelity across diverse complex reasoning tasks while achieving robust detection under fine-tuning, quantization, model-level perturbations, and adaptive output-level attacks across in-domain and out-of-distribution settings.

Stability is a property of fundamental importance in real-world systems. Although it has been widely studied and well understood in *control theory* (CT) for deterministic systems, it is largely overlooked in stochastic systems such as *Markov decision processes* (MDPs). In this paper, we aim to translate the steady-state control problem, well established in CT, where the goal is to synthesize a controller with prescribed asymptotic stability properties, into the MDP framework. To this end, we propose the novel *fast-mixing steady-state* (FMSS) problem. Given an ergodic MDP and a target steady-state distribution, the objective is to synthesize a Markovian policy that induces this distribution with the fastest possible convergence rate. Addressing this problem requires controlling the spectral properties of the induced *Markov chain* (MC) transition matrix, which generally leads to non-convex programs. Thus, we derive a tractable surrogate objective that leads to a convex program, whose properties we study in terms of approximation quality, feasibility, and computational complexity. We then move to the learning setting and propose an "offline" sample-based algorithm for FMSS (FMSS-SV), designed for tabular MDPs, in which the environment’s transition model is estimated from data. We quantify the impact of transition model estimation errors on both the objective value and the learned policy, and provide a finite-sample complexity analysis.

Mixture-of-Experts (MoE) scales model capacity while preserving computational efficiency through sparse expert activation. However, training high-quality MoEs from scratch is prohibitively expensive. An alternative is to convert pretrained dense models into sparse MoEs. Existing dense-to-MoE methods fall into two categories: \textbf{dynamic structural pruning} that converts dense models into MoEs with moderate sparsity to balance performance and efficiency, and \textbf{downcycling} approaches that use pretrained dense models to initialize highly sparse MoEs. However, existing methods break the intrinsic activation patterns within dense models, leading to suboptimal expert construction. In this work, we argue that the Gated Linear Unit (GLU) provides a natural blueprint for dense-to-MoE conversion. We show that the fine-grained neuron-wise activation patterns of GLU reveal a coarse-grained structure, uncovering an inherent MoE architecture composed of consistently activated universal neurons and dynamically activated specialized neurons. Leveraging this discovery, we introduce ExpertWeaver, a training-free framework that partitions neurons according to their activation patterns and constructs shared experts and specialized routed experts with layer-adaptive configurations. Experiments demonstrate that ExpertWeaver outperforms existing methods, both as a training-free dynamic structural pruning technique and as a downcycling strategy for MoE initialization.

Credal predictors are epistemic-uncertainty-aware models that produce a convex set of probabilistic predictions. They provide a principled framework for quantifying predictive epistemic uncertainty (EU) and have been shown to improve model robustness across a range of settings. However, most state-of-the-art (SOTA) methods primarily define EU as disagreement induced by random training initializations, which mainly reflects sensitivity to optimization randomness rather than uncertainty from more substantive sources. In response, we formulate EU as disagreement between models trained under different degrees of relaxation of the i.i.d. assumption between the training and test distributions. Building on this idea, we propose *CreDRO*, which learns an ensemble of plausible models via distributionally robust optimization. As a result, CreDRO captures EU arising not only from training randomness but also from informative disagreement due to potential train–test distribution shifts. Empirically, CreDRO consistently outperforms SOTA credal approaches on downstream tasks, including out-of-distribution detection on extensive benchmarks and selective classification in medical settings.