ICML 2026 论文列表

Spatial-temporal time series forecasting is challenging due to complex lead-lag dependencies, which are often ignored or inadequately modeled by existing methods. Thus, we propose LagLLM, the first LLM-empowered framework that explicitly models lead–lag dependencies by unifying data-driven dynamics modeling and knowledge-driven semantic reasoning. Specifically, LagLLM constructs a lead-lag graph by integrating learnable embeddings, spatial proximity, and prompt-guided reasoning from a frozen LLM, which can capture lead-lag dependencies informed by the underlying data structure and semantic knowledge. In addition, LagLLM introduces structural token sorting based on the graph, which can make a fine-tuned LLM explicitly perceive directional and delayed interactions. Experiments on eight real-world datasets show that LagLLM achieves the state-of-the-art performance with improved accuracy, robustness, and interpretability. The code is available at https://github.com/w2obin/LagLLM-2026.

Structured pruning reduces LLM inference cost by removing low-importance architectural components. This can be viewed as learning a multiplicative gate for each component under an $\ell_0$ sparsity constraint. Due to the discreteness of the $\ell_0$ norm, prior work typically adopts stochastic hard-concrete relaxations to enable differentiable optimization; however, this stochasticity can introduce a train--test mismatch when sampled masks are discretized for deployment and restricting masks to a bounded, near-binary range. To address this, we propose Deterministic Differentiable Pruning (DDP), a mask-only optimization method that eliminates stochasticity by directly optimizing a deterministic soft surrogate of the discrete $\ell_0$ objective. Compared with prior approaches, DDP offers greater expressiveness, reduced train--test mismatch, and faster convergence. We apply our method to several dense and MoE models, including Qwen3-32B and Qwen3-30B-A3B, achieving a performance loss as small as 1\% on downstream tasks while outperforming previous methods at 20\% sparsity. We further demonstrate end-to-end inference speedups in realistic deployment settings with vLLM.

Diffusion language models (DLMs) are an attractive alternative to autoregressive models because they promise sublinear-time, parallel generation, yet practical gains remain elusive as high-quality samples still demand hundreds of refinement steps. In continuous domains, consistency training along the probability-flow ODE is a popular recipe to accelerate diffusion. For discrete diffusion, no analogous sample-space ODE exists, making direct adaptation ill-defined. We argue that the right discrete substitute is the exact posterior bridge—the closed-form conditional law linking any two noise levels—which is available for broad corruptions including masked and uniform diffusion. Building on this observation, we introduce Multi-Path Discrete Consistency (MPDC), a new principle that trains a denoiser to be path-invariant in expectation across these stochastic bridges, and instantiate it as the Consistent Diffusion Language Model (CDLM), a single-stage training framework that does not require an already trained teacher model. Our CDLM objective recovers masked diffusion, continuous consistency models, and progressive or discrete distillation as analytic limits or empirical approximations of one common view. Empirically, CDLM establishes a new state of the art on both conditional and unconditional text-generation, consistently outperforming strong base discrete diffusion models and often even multi-stage distilled baselines across sampling budgets, with the largest gains in the few-step regime. Together, these results position CDLM as a principled and scalable foundation for the next generation of fast, high-fidelity discrete generative modeling.

A wide range of optimization problems can often be written in terms of generalized convex functions (GCFs). When this structure is present, it can convert certain nested bilevel objectives into single-level problems amenable to standard first-order optimization methods. We provide a new differentiable layer with a convex parameter space and show (Theorems 5.1 and 5.2) that it and its gradient are universal approximators for GCFs and their gradients. We demonstrate how this parameterization can be leveraged in practice by (i) learning optimal transport maps with general cost functions and (ii) learning optimal auctions of multiple goods. In both these cases, we show how our layer can be used to convert the existing bilevel or min-max formulations into single-level problems that can be solved efficiently with first-order methods.

A probability distribution $\mathbb{P}$ is a measure on a $\sigma$-algebra, assigning mass to sets rather than points, which poses a challenge for the training of neural networks that often struggle to reconstruct the global topology of continuous manifolds from sparse samples. We mitigate this issue by transforming $\mathbb{P}$ into a continuous function via spectral methods, providing theoretical guarantees for the convergence of the learned distribution to the true distribution. Specifically, we introduce a network, PriSpecNet, with a single-function evaluation (1-NFE) Pseudoinverse L\'evy Inversion (PiLI) solver that regards generation as a fast analytical problem, eliminating the need for iterative numerical integration while maintaining full compatibility with the stochastic interpolants. We test our PriSpecNet in two applications: for time series, it unifies generation and forecasting, outperforming state-of-the-art (SOTA) baselines with Context-FID reductions of 50.0\%, 41.5\%, 80.6\%, and 63.1\% on Sines, Solar, ETTh, and Stock benchmarks, respectively, also decreasing forecasting MSE by 29.8\% on Solar and 23.8\% on Stock. For ImageNet $256 \times 256$, 1-NFE PiLI achieves a competitive FID of **1.66** using only **26** Gflops, representing a **170$\times$** reduction in total Gflops compared to the 4,436 Gflops required by the 25-NFE DPM-Solver++.

Large Language Models (LLMs) have demonstrated sophisticated comprehension of sequential code, yet their capacity for reasoning about concurrent programs remains largely unquantified. We introduce DRPBench, a benchmark designed to evaluate the concurrent code comprehension of LLMs by measuring their data race prediction performance. To address the challenge of runtime non-determinism for evaluation on concurrent programs, we frame the evaluation as a fine-grained static prediction task using 1,003 programs from the SV-COMP suite, featuring 557 manually annotated data races with precise variable- and line-level granularity. Our evaluation of 15 state-of-the-art LLMs—spanning standard, reasoning, and agentic variants—reveals that DRPBench effectively differentiates concurrent code comprehension capabilities of LLMs. While the top-performing model (Gemini 3 with test-time reasoning) achieves an F1 score of 74.89%, most models struggle significantly (scoring less than 60%), with Llama 3 70B achieving only 8.80%. Beyond benchmarking, we characterize two primary failure modes: (1) shared-variable distraction, where multiple variable appearances degrade comprehension accuracy, and (2) synchronization-logic myopia, the inability to interpret non-standard synchronization implementations. Our findings provide a diagnostic roadmap for enhancing concurrent code comprehension of LLMs in future development.

Accelerating scientific discovery requires the identification of which experiments would yield the best outcomes before committing resources to costly physical validation. While existing benchmarks evaluate LLMs on scientific knowledge and reasoning, their ability to predict experimental outcomes---a task where AI could significantly exceed human capabilities---remains largely underexplored. We introduce SciPredict, a benchmark comprising 405 tasks derived from recent empirical studies in 33 specialized sub-fields of physics, biology, and chemistry. SciPredict addresses two critical questions: (a) *can LLMs predict the outcome of scientific experiments with sufficient accuracy?* and (b) *can such predictions be reliably used in the scientific research process?* Evaluations reveal fundamental limitations on both fronts. Model accuracies are 14-26\% and human expert performance is $\approx$20\%. Although some frontier models exceed human performance model accuracy is still far below what would enable reliable experimental guidance. Even within the limited performance, models fail to distinguish reliable predictions from unreliable ones, achieving only $\approx$20\% accuracy regardless of their confidence or whether they judge outcomes as predictable without physical experimentation. Human experts, in contrast, demonstrate strong calibration: their accuracy increases from $\approx$5\% to $\approx$80\% as they deem outcomes more predictable without conducting the experiment. SciPredict establishes a rigorous framework demonstrating that superhuman performance in experimental science requires not just better predictions, but better awareness of prediction reliability. For reproducibility all our data and code are provided at https://anonymous.4open.science/r/SciPredict-AI01.

Multimodal large language models (MLLMs) are trained on massive multimodal data, making data unlearning increasingly important as data owners may request the removal of specific content. In practice, these requests often arrive sequentially over time, giving rise to the challenging problem of *MLLM Lifelong Unlearning*. However, most existing benchmarks are limited in scale and scope, failing to capture the complexities of MLLM lifelong unlearning. To fill this gap, we introduce the MLUBench, a large-scale and comprehensive benchmark featuring 127 entities across 9 classes under lifelong unlearning requests. We perform extensive experiments using MLUBench and reveal that existing unlearning methods suffer from severe, cumulative degradation. More critically, we further identify the unique challenge of this problem: unlike in unimodal models, MLLM lifelong unlearning is constrained by the need to preserve multimodal alignment. Continually unlearning from one modality could degrade the entire model. To alleviate this challenge, we propose LUMoE, an effective and efficient method. Experiments demonstrate that LUMoE significantly mitigates the degradation problem faced by baselines. The source code and the MLUBench dataset are open-sourced in this [URL](https://github.com/lihe-maxsize/Lifelong_Unlearning_main).

The anchor-based multi-view clustering methods have received extensive attention due to their efficiency and scalability in large-scale data scenarios. Existing anchor-based methods still face challenges in learning compact and semantically discriminative anchors. Current mainstream approaches typically rely on random sampling strategies or orthogonal constraints for anchor selection and learning. However, they often optimize anchor learning and cluster assignment in a relatively separate manner, leaving the clustering semantics in the sample space insufficiently exploited for calibrating the anchor space. As a result, the learned anchors may suffer from redundant coverage and ambiguous cluster boundaries. Unlike existing anchor-based multi-view clustering methods, this paper proposes a Dual-Calibration Multi-view Clustering via Compact Anchor Learning (DCMC), which effectively improves anchor quality through a dual-space alignment mechanism. Specifically, DCMC initializes view-specific anchors to capture the underlying data distribution, and then enforces bidirectional consistency between the anchor space and the clustering space to jointly optimize both the sample-to-anchor assignments and the cluster assignments. The alternating optimization process effectively enhances cross-view semantic consistency while preserving the discriminative characteristics of each view. Experimental results demonstrate that DCMC outperforms state-of-the-art methods across multiple benchmark tests, confirming its effectiveness and reliability.

Estimating density ratios between pairs of intractable data distributions is a core problem in probabilistic modeling, enabling principled comparisons of sample likelihoods under different data-generating processes across conditions. While exact-likelihood models such as normalizing flows offer a promising approach to density ratio estimation, naive evaluations are computationally expensive and prone to discretization errors because they require simulating each distribution's likelihood independently. In this work, we leverage condition-aware flow matching to derive a single dynamical formulation for tracking density ratios along generative trajectories. We demonstrate competitive performance on simulated benchmarks for closed-form ratio estimation, and show that our method supports versatile tasks in single-cell genomics data analysis, where likelihood-based comparisons of cellular states across experimental conditions enable treatment effect estimation and batch correction evaluation.

Large language models are increasingly deployed in streaming scenarios, rendering conventional post-hoc safeguards ineffective as they fail to interdict unsafe content in real-time. While streaming safeguards based on token-level supervised training could address this, they necessitate expensive annotations and suffer from severe overfitting. In this work, we challenge the paradigm that streaming safety must rely on token-level supervised training. Instead, it is an inherent capability of well-trained post-hoc safeguards, as they already encode token-level risk signals in hidden representations. Hence, we introduce NExT-Guard, a framework that achieves streaming safeguards by monitoring interpretable latent features from Sparse Autoencoders (SAEs). It uses pretrained SAEs from publicly available base LLMs, enabling flexible, low-cost deployment without token-level supervision. Experimental results show that NExT-Guard outperforms both post-hoc and streaming safeguards based on supervised training, with superior robustness across models, SAE variants, and risk scenarios. These results make NExT-Guard a universal and scalable paradigm for real-time safety, accelerating the practical deployment of streaming safeguards.

Recent advances in large language models (LLMs) have substantially expanded the scope of Table Question Answering (TableQA). However, existing benchmarks primarily treat TableQA as a passive, single-turn natural language understanding task, lacking the capacity to evaluate autonomous reasoning and tool-call trajectories in realistic, multi-turn scenarios. To bridge this gap, we introduce TableAgent-Bench, a large-scale bilingual benchmark that reformulates TableQA as proactive, agentic interactions over structurally complex, multi-table environments. With a topology-aware construction strategy, TableAgent-Bench captures dynamic intent evolution through 1,310 multi-turn dialogues grounded in 2,275 industrial tables. Furthermore, we propose the Table-centric Agent Evaluation Framework (TAEF) to assess agent interactions with complex table structures. Specifically, TAEF integrates a specialized agent toolset and 4 metric categories to systematically diagnose intermediate failure modes, assessing performance across table localization, tool-invocation rationality, and trajectory-level pass rate. Extensive experiments with 25 state-of-the-art LLM agents reveal a substantial capability gap, with even the strongest model Gemini-3-Pro-Preview achieving only 53.4% information coverage. We expect TableAgent-Bench to serve as a rigorous testbed for developing and evaluating agents capable of robust table-centric reasoning.

Generative models frequently suffer miscalibration, wherein statistics of the sampling distribution—such as the fraction of generations in a given class—deviate from desired values. We frame calibration as a constrained optimization problem and seek the closest model in Kullback-Leibler divergence satisfying a calibration constraint. To address the intractability of imposing these constraints exactly, we introduce two surrogate objectives for fine-tuning: (1) the relax loss, which replaces the constraint with a miscalibration penalty, and (2) the reward loss, which converts calibration into a reward fine-tuning problem. We demonstrate that these approaches substantially reduce calibration error across hundreds of simultaneous constraints and models with up to nine billion parameters, spanning applications in protein design, image generation, and language modeling. Code is available at https://github.com/smithhenryd/cgm.

Research on the expressive power of transformers shows that transformers are equivariant to permutations and can approximate all permutation-equivariant continuous functions on a compact domain. However, these results assume real parameters and exact operations, whereas real-world implementations on computers can only use a finite set of numbers and inexact machine operations with round-off errors. In this work, we investigate the representability of floating-point transformers that use floating-point parameters and floating-point operations. Unlike existing results under exact arithmetic, we first show that floating-point transformers can represent non-permutation-equivariant functions even without positional encoding. Furthermore, we prove that floating-point transformers can represent all permutation-equivariant functions when the sequence length is bounded, but they cannot when the sequence length is large. We also identify the minimal equivariance property in floating-point transformers, and show that all non-trivial additive positional encoding can harm the representability of floating-point transformers.

Expert-annotated time series datasets often suffer from low agreement, especially in medical applications where decisions rely on subjective criteria and inconsistent thresholds. Such variability degrades annotation quality and thus limits the reliability of supervised classification models. To address this, we propose to rely on a pairwise comparison-based approach, which provides a more robust alternative to individual annotation, since relative judgments are typically easier and yield higher consistency. The problem is thus transformed into a ranking problem and we introduce an ideal point model adapted to time series data using elastic similarity measures such as Dynamic Time Warping (DTW) and Time Warp Edit Distance (TWED). We prove Lipschitz continuity of these distances and demonstrate several convergence guarantees for this model. To facilitate gradient-based optimization, we also introduce a differentiable version of the TWED. Finally, we show through multiple experiments that our approach produces accurate and robust rankings under noisy annotation conditions.

The minimal norm weight perturbations of DNNs required to achieve a specified change in output are derived and the factors determining its size are discussed. These single-layer exact formulae are contrasted with more generic multi-layer Lipschitz constant based robustness guarantees; both are observed to be of the same order which indicates similar efficacy in their guarantees. These results are applied to precision-modification-activated backdoor attacks, establishing provable compression thresholds below which such attacks cannot succeed, and show empirically that low-rank compression can reliably activate latent backdoors while preserving full-precision accuracy. These expressions reveal how back-propagated margins govern layer-wise sensitivity and provide certifiable guarantees on the smallest parameter updates consistent with a desired output shift.

Text-conditioned image generation has made rapid progress, yet rendering images with long-form text remains challenging due to the limitations of existing datasets, which predominantly focus on short and simple text. We introduce TextAtlas5M, a large-scale dataset designed to evaluate long-text rendering, where “long text” encompasses not only textual length but also layout complexity and semantic richness. TextAtlas5M contains 5 million generated and collected images across diverse data types, enabling comprehensive evaluation of large-scale generative models. We further curate 4,000 human-improved test cases (TextAtlasEval) spanning four domains, forming one of the most extensive benchmarks for text rendering. Evaluations show that TextAtlas5M poses substantial challenges even for state-of-the-art proprietary models (e.g., GPT-4o), with significantly larger gaps observed for open-source models. Training on TextAtlas5M consistently improves text rendering for both diffusion-based and autoregressive models, demonstrating its effectiveness for advancing text-rich image generation.

Vertex-level clustering for directed graphs (digraphs) remains challenging as edge directionality breaks the key assumptions underlying popular spectral methods, which also incur the overhead of eigen-decomposition. This paper proposes *Parametrized Power Iteration Clustering* (ParPIC), a random-walk-based clustering method for weakly connected digraphs. This builds over the Power-Iteration Clustering paradigm, which uses the rows of the iterated diffusion operator as a data embedding. ParPIC has three important features: the use of parametrized reversible random walk operators, the automatic tuning of the diffusion time, and the efficient truncation of the final embedding, which produces low-dimensional data representations and reduces complexity. Empirical results on synthetic and real-world graphs demonstrate that ParPIC achieves competitive clustering accuracy with improved scalability relative to spectral and teleportation-based methods.

Conventional time-series discriminative forecasting relies on point-wise regression, which inherently induces over-smoothing and fails to capture stochastic volatility in complex systems. While first-order generative flow matching methods mitigate this issue, they ignore system inertia, resulting in phase-space ambiguities and high sensitivity to noise. We introduce KineFlow, a generative time-series forecasting framework that augments flow matching with a phase-space Neural Acceleration Field, treating exogenous inputs as driving forces that produce gradual momentum shifts rather than abrupt state perturbations. This second-order formulation serves as a structural filter via double integration, suppressing high-frequency noise and producing robust, physically consistent predictions. Extensive experiments on six real-world benchmarks demonstrate that KineFlow achieves an average 15% MSE improvement over discriminative baselines and an 8% gain in CRPS compared to state-of-the-art generative methods.