ICML 2026 Papers

Does natural language text have an intrinsic curvature? Language is increasingly modeled in curved geometries—hyperbolic spaces for hierarchy, mixed-curvature manifolds for compositional structure—yet a basic scientific question remains unresolved: what does curvature mean for text itself, in a way that is native to language rather than an artifact of the embedding space we choose? We argue that text does indeed have curvature, and show how to detect it, define it, and use it. To this end, we propose Texture, a text-native, word-level discrete curvature signal, and make three contributions. (a) Existence: We provide empirical and theoretical certificates that semantic inference in natural corpora is non-flat. (b) Definition: We define Texture as a signed two-axis curvature of the word-in-context belief field—the differential of reconciliation between prefix and suffix—measuring, via a debiased Schrödinger transport divergence, whether adding context from one side contracts the semantic effect of context from the other side (focus, positive) or expands it into competing continuations (fan-out, negative). (c) Utility: Texture is actionable: it serves as a general-purpose measurement and control primitive enabling geometry without geometric training; we instantiate it on two representative tasks, improving long-context inference through curvature-guided compression and retrieval-augmented generation through curvature-guided routing. Together, our results establish a text native curvature paradigm, making Texture practically useful.

This paper proposes Omni Dense Captioning, a novel task designed to generate continuous, fine-grained, and structured audio-visual narratives with explicit timestamps. To ensure dense semantic coverage, we introduce a six-dimensional structural schema to create "script-like" captions, enabling readers to vividly imagine the video content scene by scene, akin to a cinematographic screenplay. To facilitate research, we construct OmniDCBench, a high-quality, human-annotated benchmark, and propose SodaM, a unified metric that evaluates time-aware detailed descriptions while mitigating scene boundary ambiguity. Furthermore, we construct a training dataset, TimeChatCap-42K, and present TimeChat-Captioner-7B, a strong baseline trained via SFT and GRPO with task-specific rewards. Extensive experiments demonstrate that TimeChat-Captioner achieves state-of-the-art performance, surpassing Gemini-2.5-Pro, while its generated dense descriptions significantly boost downstream capabilities in audio-visual reasoning (DailyOmni and WorldSense) and temporal grounding (Charades-STA). All datasets, models, and code are publicly available at https://github.com/yaolinli/TimeChat-Captioner.

Multi-objective combinatorial optimization seeks Pareto-optimal solutions over exponentially large discrete spaces, yet existing methods sacrifice generality, scalability, or theoretical guarantees. We reformulate it as an online learning problem over a decomposed decision space, solving position-wise bandit subproblems via adaptive expert-guided sequential construction. This formulation admits regret bounds of $O(d\sqrt{T \log T})$ depending on subproblem dimensionality \(d\) rather than combinatorial space size. On standard benchmarks, our method achieves 80--98\% of specialized solvers performance while achieving two to three orders of magnitude improvement in sample and computational efficiency over Bayesian optimization methods. On real-world hardware-software co-design for AI accelerators with expensive simulations, we outperform competing methods under fixed evaluation budgets. The advantage grows with problem scale and objective count, establishing bandit optimization over decomposed decision spaces as a principled alternative to surrogate modeling or offline training for multi-objective optimization.

Expressivity has been a major focus in the design of Graph Neural Networks (GNNs), yet a significant gap persists between theoretical universal expressivity and practical performance. While many expressive GNNs are efficient and achieve strong results, they often focus on specific graph properties and lack theoretical expressivity for general graph tasks. Conversely, theoretically universal-expressive models often suffer from high computational costs or poor generalization, limiting their real-world applicability. To bridge this gap, we introduce Equivariant Noise GNNs (ENGNNs), a framework that utilizes random noise features to enhance the expressivity of GNNs. Crucially, unlike prior methods that naively use noise, we enforce equivariance to nodewise noise transformations, such as orthogonal transformations. We prove that this property reduces the model's theoretical sample complexity, thereby improving generalization. Our framework simultaneously reaches theoretical universal expressivity, maintains the linear scalability of standard Message-Passing Neural Networks in practice, and achieves performance comparable to computationally expensive, high-expressivity models. Extensive experiments confirm strong performance across node, link, subgraph, and graph-level prediction tasks, demonstrating that the equivariant use of noise provides a powerful and practical pathway for building expressive GNNs. Our code is available at \url{https://github.com/MuLabPKU/EquivNoiseGNN}.

Large language models are increasingly used to predict human preferences in both scientific and business endeavors, yet current approaches rely exclusively on analyzing model outputs without considering the underlying mechanisms. Using election forecasting as a test case, we introduce *mechanistic forecasting*, a method that demonstrates that probing internal model representations offers a fundamentally different---and sometimes more effective--- approach to preference prediction. Examining over 24 million configurations across 7 models, 6 national elections, multiple persona attributes, and prompt variations, we systematically analyze how demographic and ideological information activates latent party-encoding components within the respective models. We find that leveraging this internal knowledge via mechanistic forecasting, opposed to solely relying on surface-level predictions, can improve prediction accuracy. The effects vary across demographic versus opinion-based attributes, political parties, national contexts, and models. Our findings demonstrate that the latent representational structure of LLMs contains systematic, exploitable information about human preferences, establishing a new paradigm for using language models in social science prediction tasks.

While LLMs have seen substantial improvement in reasoning capabilities, they also sometimes overthink, generating unnecessary reasoning steps, particularly under uncertainty, given ill-posed or ambiguous queries. We introduce statistically principled early stopping methods that monitor uncertainty signals during generation to mitigate this issue. Our first approach is nonparametric and provides finite-sample guarantees on the probability of halting too early on well-posed queries. Our second approach is parametric: it models inter-arrival times of uncertainty keywords as a renewal process and applies sequential testing for stopping. We conduct empirical evaluations on reasoning tasks across several domains and models. Our results indicate that uncertainty-aware early stopping can improve both efficiency and reliability in LLM reasoning. The performance varies across domains, and we observe especially significant gains for math reasoning.

Recent advances in vision-language-action (VLA) models for robotics have highlighted the importance of reliable uncertainty quantification in sequential tasks. However, assessing and improving calibration in such settings remains mostly unexplored, especially when only partial trajectories are observed. In this work, we formulate *sequential calibration* for episodic tasks, where task-success confidence is produced along an episode, while success is determined at the end of it. We introduce a sequential extension of the Brier score and show that, for binary outcomes, its risk minimizer coincides with the VLA policy’s value function. This connection bridges uncertainty calibration and reinforcement learning, enabling the use of temporal-difference (TD) value estimation as a principled calibration mechanism over time. We empirically show that TD calibration improves performance relative to the state-of-the-art on simulated and real-robot data. Interestingly, we show that when calibrated using TD, the VLA's single-step action probabilities can yield competitive uncertainty estimates, in contrast to recent findings that employed different calibration techniques.

Machine learning is transforming molecular sciences by accelerating property prediction, simulation, and the discovery of new molecules and materials. Acquiring labeled data in these domains is often costly and time-consuming, whereas large collections of unlabeled molecular data are readily available. Standard semi-supervised learning methods often rely on label-preserving augmentations, which are challenging to design in the molecular domain, where minor changes can drastically alter properties. In this work, we show that semi-supervised methods that rely on an ensemble consensus can boost predictive accuracy across a diverse range of molecular datasets, task types, and graph neural network architectures. We find that training with an ensemble consensus objective increases robustness in models and exhibits an effect similar to knowledge distillation; an individual member of an ensemble trained this way outperforms a full ensemble trained in a traditional supervised fashion in almost all cases. In addition, this type of semi-supervised training reduces calibration error.

The success of Hyper-Connections (HC) in neural networks (NN) has also highlighted issues related to training instability and restricted scalability. The Manifold-Constrained Hyper-Connections (mHC) mitigate these challenges by projecting the residual connection space onto a Birkhoff polytope, however, it faces two issues: 1) its iterative Sinkhorn-Knopp (SK) algorithm does not always yield exactly doubly stochastic residual matrices; 2) mHC incurs a prohibitive $\mathcal{O}(n^3C)$ parameter complexity with $n$ as the width of the residual stream and $C$ as the feature dimension. The recently proposed mHC-lite reparametrizes the residual matrix via the Birkhoff-von-Neumann theorem to guarantee double stochasticity, but also faces a factorial explosion in its parameter complexity, $\mathcal{O} \left( nC \cdot n! \right)$. To address both challenges, we propose **KromHC**, which uses the $\underline{\text{Kro}}$necker products of smaller doubly stochastic matrices to parametrize the residual matrix in $\underline{\text{mHC}}$. By enforcing manifold constraints across the factor residual matrices along each mode of the tensorized residual stream, KromHC guarantees exact double stochasticity of the residual matrices while reducing parameter complexity to only $\mathcal{O}(n^2C)$. Experiments show that KromHC matches or even outperforms other state-of-the-art (SOTA) mHC variants, while requiring significantly fewer trainable parameters. The code is at https://github.com/wz1119/KromHC.

Benchmarks are widely used to evaluate and compare the performance of artificial intelligence systems. However, some approaches to computing benchmark metrics produce invalid uncertainty estimates or make unrecognized assumptions about the evaluation setting. We leverage statistical modeling to make two contributions to the practice of AI benchmarking. First, we formally distinguish measurements of benchmark accuracy from generalized accuracy (performance on all potential test items similar to those included in the benchmark). Then, in a simulated setting and with large-scale evaluation of 22 API-access frontier large language models on 3 popular benchmarks, we show how analysis via generalized linear mixed model can produce correct estimates of generalized accuracy while more efficiently quantifying uncertainty compared to existing regression-free approaches. We also show how this approach can equip evaluators with important context on evaluation results, including variance decomposition and item difficulty estimates that illuminate important aspects of LLM performance and benchmark construction.

Stereo vision is widely utilized in automotive imagery and 3D reconstruction, creating a demand for compressing stereo images. Existing methods for stereo image compression often employ VAE-like architectures based on distortion optimization, leading to subpar perceptual quality at low bitrates. While generative compression excels in high perceptual fidelity at low bitrates, it struggles to maintain consistency across viewpoints, making decoded images less useful for critical downstream tasks. To address this, we introduce SDiD, a distributed stereo image compression architecture based on shared pre-trained diffusion priors. We employ a diffusion prior alignment module to efficiently obtain the main-view-prior from the foundation diffusion, and utilize a prior transformation structure to enable the auxiliary view to achieve reliable and fast perceptual enhancement while maintaining consistency. Through extensive experiments, we demonstrate that SDiD outperforms existing methods in perceptual quality across multiple datasets. Even at extremely low bitrates, SDiD can accurately recover depth information between decoded images. On the InStereo2K dataset, SDiD requires only one-third of the bits compared to the state-of-the-art baseline (0.02 bpp vs. 0.06 bpp) to reconstruct image pairs with similar depth information.

3D CT analysis spans a continuum from low-level perception to high-level clinical understanding. Existing 3D-oriented analysis methods adopt either isolated task-specific modeling or task-agnostic end-to-end paradigms to produce one-hop outputs, impeding the systematic accumulation of perceptual evidence for downstream reasoning. In parallel, recent multimodal large language models (MLLMs) exhibit improved visual perception and can integrate visual and textual information effectively, yet their predominantly 2D-oriented designs fundamentally limit their ability to perceive and analysis volumetric medical data. To bridge this gap, we propose 3DMedAgent, an unified agent that enables 2D MLLMs to perform general 3D CT analysis without 3D-specific fine-tuning. 3DMedAgent coordinates heterogeneous visual and textual tools through a flexible MLLM agent, progressively decomposing complex 3D analysis into tractable subtasks that transition from global to regional views, from 3D volumes to informative 2D slices, and from visual evidence to structured textual representations. Central to this design, 3DMedAgent maintains a long-term structured memory that aggregates intermediate tool outputs and supports query-adaptive, evidence-driven multi-step reasoning. We further introduce the DeepChestVQA benchmark for evaluating unified perception-to-understanding capabilities in 3D thoracic imaging. Experiments across over 40 tasks demonstrate that 3DMedAgent consistently outperforms general, medical, and 3D-specific MLLMs, highlighting a scalable path toward general-purpose 3D clinical assistants. Code and data are available \href{https://github.com/jinlab-imvr/3DMedAgent}{here}.

Endpoint Detection and Response (EDR) systems are crucial for identifying malicious activities on endpoint devices, yet existing methods struggle to efficiently model ultra-long log sequences and to provide interpretable reasoning for security analysts. We propose WatchLog, a novel framework that represents raw logs as video-structured data, enabling scalable and expressive video-language modeling of endpoint behaviors. Each event is encoded as a key–value-guided image, and the resulting images are temporally organized into a video sequence. To capture long-range dependencies, WatchLog employs a temporal cross-attention adapter that enables pixel-wise interaction across time. The adapter acts as an auxiliary temporal reasoning pathway, aligning spatial representations with relevant temporal contexts while preserving the original behavioral semantics. We adopt a two-stage pre-training strategy followed by supervised fine-tuning to generate behavior explanations grounded in event-level semantics and detection outcomes. Experiments on our newly constructed EDR8M-20R dataset and a public benchmark demonstrate that WatchLog consistently outperforms state-of-the-art methods in detection accuracy and recall, while offering more interpretable reasoning traces and significantly improved inference efficiency. Extensive ablation studies further support the robustness and interpretability of the proposed method.

Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as a prevailing paradigm for enhancing reasoning in Multimodal Large Language Models (MLLMs). However, relying solely on outcome supervision risks reward hacking, where models learn spurious reasoning patterns to satisfy final answer checks. While recent rubric-based approaches offer fine-grained supervision signals, they suffer from high computational costs of instance-level generation and inefficient training dynamics caused by treating all rubrics as equally learnable. In this paper, we propose **Stratified Rubric-based Curriculum Learning (RuCL)**, a novel framework that reformulates curriculum learning by shifting the focus from data selection to reward design. RuCL generates generalized rubrics for broad applicability and stratifies them based on model competence, dynamically adjusting their weights to guide the model from foundational perception to advanced logical reasoning. Extensive experiments on various visual reasoning benchmarks show that RuCL yields a remarkable **+7.83%** average improvement over the Qwen2.5-VL-7B model, achieving a state-of-the-art accuracy of **60.06%**.

Surgical video datasets are essential for scene understanding, enabling procedural modeling and intra-operative support. However, these datasets are often heavily imbalanced, with rare actions and tools under-represented, which limits the robustness of downstream models. We address this challenge with *SurgiFlowVid*, a sparse and controllable video diffusion framework for generating surgical videos of under-represented classes. Our approach introduces a dual-prediction diffusion module that jointly denoises RGB frames and optical flow, providing temporal inductive biases to improve motion modeling from limited samples. In addition, a sparse visual encoder conditions the generation process on lightweight signals (e.g., sparse segmentation masks or RGB frames), enabling controllability without dense annotations. We validate our approach on three surgical datasets across tasks including action recognition, tool presence detection, and laparoscope motion prediction. Synthetic data generated by our method yields consistent gains of $10$–$20$% over competitive baselines, establishing *SurgiFlowVid* as a promising strategy to mitigate data imbalance and advance surgical video understanding methods.

Implicit neural representations (INRs) encode data as continuous coordinate-based functions parameterized by neural networks, shifting downstream tasks such as image recognition to operate on functional rather than discrete representations. Despite their increasing adoption, the adversarial robustness of INR-based classification pipelines remain largely underexplored. In this work, we present the first systematic study of adversarial robustness in INR-based classifiers. A key challenge is that generating an INR requires $\text{\emph{training}}$ a neural network for each input sample, resulting in an optimization-in-the-loop forward pass that renders standard gradient-based attacks computationally prohibitive. To address this, we design surrogate models that amortizes the INR-generation process, serving as a practical proxy for attacking INR-based classifiers. We also develop speed-up techniques that substantially reduce the training cost of the surrogate. We show that in contrast to recent work, INR-based classifiers are vulnerable: under adversarial input perturbations, classification accuracy collapses to near zero. Moreover, existing countermeasures designed to operate on discrete representations offer limited protection.

Saliency maps are widely used to interpret image classification models and build trust in their predictions; however, their reliability remains a central concern, as randomized networks can produce saliency maps that closely resemble those of trained models. We identify a previously underappreciated architectural contributor to this phenomenon: a *center-focused saliency bias* induced by common convolutional design choices. Through controlled ablations, we show that architectural components such as zero padding and receptive field growth induce a center-focused saliency prior that is already present in randomly initialized CNNs and under randomized inputs. In contrast, this behavior is largely absent in non-convolutional architectures such as Vision Transformers (ViTs) and multilayer perceptrons (MLPs). To investigate the interaction between architectural priors and learning, we introduce a corner-shift benchmark and a Center-Shift Index that quantify how saliency redistributes under object relocation. We show that training can partially shift saliency toward object regions, while randomized models remain dominated by architectural center bias, providing one mechanism by which trained-random similarity can be inflated and clarifying how architectural priors can confound standard saliency evaluations.

Neural collapse (NC) describes the structured geometry that emerges in the features and weights of trained classifiers. Recent theory suggests NC can be suboptimal in deep architectures, attributing this to an explicit low-rank bias from L2 regularization. We study the deep unconstrained feature model (UFM)—equivalent to a deep linear network with orthogonal inputs—trained without regularization, to isolate how gradient descent and depth alone shape NC. We show that depth induces an implicit low-rank bias: low-rank matrices propagate norm more efficiently through successive multiplications, promoting low-rank alternatives to NC. These alternatives, we argue, correspond to softmax codes: max-margin solutions previously found in width-bottlenecked networks. Analyzing training dynamics under spectral initialization, we identify an early-time repulsion among singular values that drives low-rank emergence, and characterize how depth shrinks NC's basin of attraction. Finally, we show that some effects act in the opposite direction: for randomly initialized networks, increasing width biases training toward higher-rank solutions. Our results provide the first asymptotic and dynamic characterization of implicit bias in deep UFMs trained with unregularized multiclass cross-entropy.

Accurate prediction of nuclear magnetic resonance (NMR) chemical shifts is fundamental to spectral analysis and molecular structure elucidation, yet existing machine learning methods rely on limited, labor-intensive atom-assigned datasets. We propose a semi-supervised framework that learns NMR chemical shifts from millions of literature-extracted spectra without explicit atom-level assignments, integrating a small amount of labeled data with large-scale unassigned spectra. We formulate chemical shift prediction from literature spectra as a permutation-invariant set supervision problem, and show that under commonly satisfied conditions on the loss function, optimal bipartite matching reduces to a sorting-based loss, enabling stable large-scale semi-supervised training beyond traditional curated datasets. Our models achieve substantially improved accuracy and robustness over state-of-the-art methods and exhibit stronger generalization on significantly larger and more diverse molecular datasets. Moreover, by incorporating solvent information at scale, our approach captures systematic solvent effects across common NMR solvents for the first time. Overall, our results demonstrate that large-scale unlabeled spectra mined from the literature can serve as a practical and effective data source for training NMR shift models, suggesting a broader role of literature-derived, weakly structured data in data-centric AI for science.