AI conference papers

AI conference paper library

Search papers from 30+ top conferences, including NeurIPS, ICLR, ICML, CVPR, ACL and AAAI. Filter by keyword, then open any paper for its abstract, authors and links.

ICML 2026

MacroGuide: Topological Guidance for Macrocycle Generation

Alicja Maksymiuk, Alexandre Duplessis, Michael M. Bronstein, Alexander Tong, Fernanda Duarte, Ismail Ilkan Ceylan

Macrocycles are ring-shaped molecules that offer a promising alternative to small-molecule drugs due to their enhanced selectivity and binding affinity against difficult targets. Despite their chemical value, they remain underexplored in generative modeling, likely owing to their scarcity in public datasets and the challenges of enforcing topological constraints in standard deep generative models. We introduce MacroGuide: Topological Guidance for Macrocycle Generation, a diffusion guidance mechanism that uses Persistent Homology to steer the sampling of pretrained molecular generative models toward the generation of macrocycles, in both unconditional and conditional (protein pocket) settings. At each denoising step, MacroGuide constructs a Vietoris-Rips complex from atomic positions and promotes ring formation by optimizing persistent homology features. Empirically, applying MacroGuide to pretrained diffusion models increases macrocycle generation rates from 1% to 99%, while matching or exceeding state-of-the-art performance on key quality metrics such as chemical validity, diversity, and PoseBusters checks. The implementation can be found at https://github.com/ala1705/MacroGuide.

ICML 2026

FS-I2P: A Hierarchical Focus–Sweep Registration Network with Dynamically Allocated Depth

Zhixin Cheng, Yujia Chen, Xujing Tao, Bohao Liao, Xiaotian Yin, Baoqun Yin, Tianzhu Zhang

Image-to-point cloud registration is often challenged by viewpoint changes, cross-modal discrepancies, and repetitive textures, which induce scale ambiguity and consequently lead to erroneous correspondences. Recent detection-free methods alleviate this issue by leveraging multi-scale features and transformer-based interactions. However, they still suffer from attention drift across layers and intra-scale inconsistencies, hindering precise registration. Inspired by complex scene observation, we propose a ``Focus--Sweep'' paradigm and develop a Hierarchical Mamba Interaction Module within an SSM-based framework to enhance multi-level cross-modal feature association. In addition, we introduce a Dynamic Layer Allocation Strategy that adaptively determines the iteration depth to better exploit geometric constraints and improve matching robustness. Extensive experiments and ablations on two benchmarks, RGB-D Scenes V2 and 7-Scenes, demonstrate that our approach achieves state-of-the-art performance.

ICML 2026

Quartet II: Accurate LLM Pre-Training in NVFP4 by Improved Unbiased Gradient Estimation

Andrei Panferov, Erik Schultheis, Soroush Tabesh, Dan Alistarh

The NVFP4 lower-precision format, supported in hardware by NVIDIA Blackwell GPUs, promises to allow, for the first time, end-to-end fully-quantized pre-training of massive models such as LLMs. Yet, existing quantized training methods still sacrifice some of the representation capacity of this format in favor of more accurate unbiased quantized gradient estimation by stochastic rounding (SR), losing noticeable accuracy relative to standard FP16 and FP8 training. In this paper, improve the state of the art for quantized training in NVFP4 via a novel unbiased quantization routine for micro-scaled formats, called MS-EDEN, that has more than 2x lower quantization error than SR. We integrate it into a novel fully-NVFP4 quantization scheme for linear layers, called Quartet II. We show analytically that Quartet II achieves consistently better gradient estimation across all major matrix multiplications, both on the forward and on the backward passes. In addition, our proposal synergizes well with recent training improvements aimed specifically at NVFP4. We further validate Quartet II on end-to-end LLM training with up to 1.9B parameters on 38B tokens. We provide kernels for execution on NVIDIA Blackwell GPUs with up to 4.2x speedup over BF16.

ICML 2026

Hallucination Detection from Structural Reasoning Model

Jianbo Sun, Pengkun Yang

Hallucinations pose a key challenge for large language models, and chain-of-thought prompting exposes intermediate reasoning but usually treats traces as linear sequences, making crossstep dependencies and unsupported intermediate claims difficult to identify. We propose a structural reasoning model to describe interactions among local reasoning steps. To detect hallucinations, we extract a directed acyclic reasoning graph over conditions and intermediate claims, verify each claim against its parent nodes, and aggregate the step signals with a simple mass-flow rule. Under a probabilistic erasure-gate abstraction, we interpret this aggregation as measuring information loss along the reasoning graph. Experiments on GSM8K, MATH, HumanEval, and HotpotQA show that the proposed method is most advantageous on longer, dependency-rich reasoning traces such as math and code generation, while remaining competitive on shorter factual QA; these results provide a structured perspective on chain-of-thought evaluation. All code and data are available at https://github.com/soncheinbok/FlowScore.

ICML 2026

Spatiotemporal Imputation with Graph-Informed Flow Matching

Zepeng Zhang, Aref Einizade, Jhony H. Giraldo, Olga Fink

Missing data is a common challenge in spatiotemporal systems, arising in applications such as air quality monitoring and urban traffic management. Traditional machine learning approaches, like recurrent and graph neural networks, rely on iterative propagation, which tends to accumulate errors over time and space. Recent diffusion-based methods mitigate error propagation but require iterative sampling and often depend on problem-agnostic Gaussian priors, limiting both efficiency and effectiveness. To address these limitations, we propose **GiFlow**, a *Graph-Informed Flow Matching* framework for spatiotemporal imputation. GiFlow replaces the typical Gaussian prior with a graph-informed prior constructed via spatiotemporal filtering of observable signals, which better aligns the source distribution to the target and thereby simplifies the generation trajectory. The flow field is parameterized by a hybrid vector field model that integrates spatial attention, temporal attention, and spatiotemporal propagation, enabling joint modeling of spatial and temporal dependencies. Extensive experiments on both synthetic and real-world datasets demonstrate that the proposed GiFlow outperforms the state-of-the-art approaches in spatiotemporal imputation. The code is available at https://github.com/zepengzhang/GiFlow.

ICML 2026

At the Edge of Understanding: Sparse Autoencoders Trace The Limits of Transformer Generalization

Praneet Suresh, Jack Stanley, Sonia Joseph, Luca Scimeca, Danilo Bzdok

Pre-trained transformers have demonstrated remarkable generalization abilities, at times extending beyond the scope of their training data. Yet, real-world deployments often face unexpected or adversarial data that diverges from training data distributions. Without explicit mechanisms for handling such shifts, model reliability and safety degrade, urging more disciplined study of out-of-distribution (OOD) settings for transformers. By systematic experiments, we present a mechanistic framework for delineating the precise contours of transformer model robustness. We find that OOD inputs, including subtle typos and jailbreak prompts, drive language models to operate on an increased number of fallacious concepts in their internals. We leverage this device to quantify and understand the degree of distributional shift in prompts, enabling a mechanistically grounded fine-tuning strategy to robustify LLMs. Expanding the very notion of OOD from input data to a model’s private computational processes—a new transformer diagnostic at inference time—is a critical step toward making AI systems safe for deployment across science, business, and government.

ICML 2026

S2GS: Streaming Semantic Gaussian Splatting for Online Scene Understanding and Reconstruction

Renhe Zhang, Yuyang Tan, Jingyu Gong, Zhizhong Zhang, Lizhuang Ma, Yuan Xie, Xin Tan

Existing offline feed-forward methods for joint scene understanding and reconstruction on long image streams often perform global computation over an ever-growing set of past observations, causing runtime and GPU memory to increase rapidly with sequence length. We propose Streaming Semantic Gaussian Splatting (S2GS), a strictly causal and incremental framework that builds a 3D Gaussian semantic field from image streams without accessing future frames or reprocessing historical observations. S2GS continuously updates scene geometry, appearance, and instance-level semantics through a geometry--semantic decoupled dual-backbone design. The geometry branch performs causal modeling for incremental Gaussian updates, while the semantic branch leverages a 2D foundation vision model and a query-driven decoder to predict masks and identity embeddings. Query-level contrastive alignment and lightweight online association with an instance memory further stabilize temporal identities. Experiments show that S2GS matches or outperforms strong offline baselines, while substantially improving long-horizon scalability: it processes over 1,000 frames with much slower runtime and GPU memory growth, whereas offline global-processing baselines typically run out of memory at around 80 frames under the same setting. Project Page: https://stdcoutzrh.github.io/projects/s2gs/.

ICML 2026

VeriSimpl: Robust Optimization Modeling from Natural Language using Simplification-based Verification

Sumaya Abdul Rahman, Seckhen Ariel Andrade Cuellar, Ghani Raissov, Mohammad Raza

Natural language interfaces can greatly benefit the accessibility and usability of optimization modeling, and recent advances in large language models (LLMs) show promise in automatically translating textual problem descriptions into executable solver formulations. However, a key challenge for existing approaches is to ensure that the inferred formulation correctly implements the intended task, even if it may execute without errors. We introduce VeriSimpl, a solver–LLM framework for robust natural-language-to-optimization formalization. Our approach is based on the idea of simplification-based verification, where the optimization solver is leveraged to generate simplified diagnostic queries about a candidate formulation to allow the LLM to tractably reason about the correctness of the formulation with respect to the task description. We present such simplification strategies along different dimensions with respect to problem constraints and decision variables, which allow the LLM to reason locally under fixed global contexts. Evaluations on a range of optimization benchmarks show how our approach provides consistent improvements in accuracy over existing methods, while also providing a novel high-precision self-verification signal.

ICML 2026

ActiveUltraFeedback: Efficient Preference Data Generation using Active Learning

Davit Melikidze, Marian Schneider, Jessica Lam, Martin Wertich, Ido Hakimi, Barna Pásztor, Andreas Krause

Reinforcement Learning from Human Feedback (RLHF) has become the standard for aligning Large Language Models (LLMs), yet its efficacy is bottlenecked by the high cost of acquiring preference data, especially in low-resource and expert domains. To address this, we introduce ActiveUltraFeedback, a modular active learning pipeline that leverages uncertainty estimates to dynamically identify the most informative responses for annotation. Our pipeline facilitates the systematic evaluation of standard response selection methods alongside Double Reverse Thompson Sampling (DRTS) and DeltaUCB, two novel methods prioritizing response pairs with large predicted quality gaps, leveraging recent results showing that such pairs provide good signals for fine-tuning. Our experiments demonstrate that ActiveUltraFeedback yields high-quality datasets that lead to significant improvements in downstream performance, notably achieving comparable or superior results with as little as one-sixth of the annotated data relative to static baselines. Our pipeline is available at https://github.com/lasgroup/ActiveUltraFeedback and our preference datasets at https://huggingface.co/ActiveUltraFeedback.

ICML 2026

InfVSR: Toward Consistency-Driven Streaming Generative Video Super-Resolution

Ziqing Zhang, Kai Liu, Zheng Chen, Xi Li, Yucong Chen, Bingnan Duan, Linghe Kong, Yulun Zhang

Real-world videos often extend over thousands of frames. Existing generative video super-resolution (VSR) approaches, however, face two persistent challenges when processing long sequences: (1) inefficiency due to the heavy cost of multi-step denoising for full-length sequences; and (2) poor consistency hindered by temporal decomposition that causes artifacts and discontinuities. To break these limits, we propose InfVSR, which reformulates VSR as an autoregressive-one-step-diffusion paradigm, and enables streaming inference with video diffusion priors. First, we adapt the pretrained DiT into a causal structure, maintaining both local and global coherence via rolling KV-cache and joint visual guidance. Second, we distill the diffusion process into a single step efficiently, with patch-wise pixel supervision and cross-chunk distribution matching. To fill the gap in long-form video evaluation, we build a new benchmark tailored for extended sequences and further introduce semantic-level metrics to comprehensively assess temporal consistency. Our method pushes the frontier of long-form VSR, achieves state-of-the-art quality with enhanced semantic consistency, and delivers up to 58x speed-up over existing methods such as MGLD-VSR. Our code and models are available at https://github.com/Kai-Liu001/InfVSR .

ICML 2026

Beyond Point Predictions: Manifold Expansion and Dual Alignment for Robust Time Series Distillation

Junyao Hong, Zesheng Lai, Xinyi Xiao, Suyang Zhou, Aodong Shen, Youyong Kong

Knowledge Distillation (KD) promises to bridge the gap between the high computational costs of Transformer-based models and the expressiveness limitations of linear models in long-term time series forecasting. Many existing time series distillation methods inherit the computer vision paradigm, constraining student models by minimizing point-wise prediction matching (output-level distillation) errors. However, blindly mimicking teacher predictions, which can be uncertain, may induce negative transfer. To address this, we propose Dynamic Structural Distillation (DSD), a robust framework that goes beyond the prediction-matching paradigm. First, we design LMP-Net, leveraging manifold expansion to project features into a high-dimensional latent space, alleviating the expressiveness bottleneck while preserving lightweight inference. Second, to address token–point architectural mismatch, we propose Dual Manifold Alignment, employing Similarity-Preserving Knowledge Distillation (SPKD) and Optimal Transport (OT) to align features at the topological and geometric levels, respectively. Finally, we introduce Regime-Aware Adaptive Distillation (RAAD) to reduce the risk of teacher misguidance via a dataset-level regime prior and a confidence-based adaptive gating mechanism. Extensive experiments on five benchmarks show that DSD is compatible with diverse teacher architectures, improves lightweight students across data regimes, and achieves a favorable accuracy–efficiency trade-off. Code is available at https://github.com/jyh0526/DSD.

ICML 2026

PLASH: Provably Linear-Time Attention with Selective Higher-Order Feature Sketching

Yuwen Huang, Xiang Pan

Standard softmax attention scales quadratically with sequence length, which makes long-context training and inference expensive. We introduce PLASH, an attention block whose cost grows linearly in the number of keys. PLASH compresses the original keys and values into $M$ learned prototypes, where $M\in\mathbb{Z}_{>0}$ is much smaller than the number of keys. The compressed prototypes are then enriched with randomized polynomial features that recover inter-token information lost to compression. The output is computed by exact scaled dot-product softmax attention from each query to the enriched prototypes, so PLASH preserves the standard attention interface.The construction applies to self- and cross-attention. We prove sketch-error bounds for the enrichment step, a per-input certificate that upper-bounds the deviation from standard softmax attention on each forward pass, and a runtime bound linear in the number of queries and keys. Experiments on long-context language modeling (Qwen3-4B on PG-19) and time-series forecasting (ETT, ECL, Weather) show competitive accuracy and favorable scaling against efficient-attention baselines.

ICML 2026

LC-QAT: Data-Efficient 2-Bit QAT for LLMs via Linear-Constrained Vector Quantization

Haoyu Wang, Xingyu Yu, Haiyan Zhao, Fengxiang Wang, XU HAN

Quantization-aware training (QAT) is essential for extremely low-bit large language models (LLMs). Current QAT methods are mainly based on scalar quantization (SQ), which enables efficient optimization but suffers from severe performance degradation at 2-bit precision. On the other hand, vector quantization (VQ) provides substantially higher representational capacity, but its discrete codebook lookup prevents end-to-end training. We propose LC-QAT, a 2-bit weight-only VQ-QAT framework that represents quantized weights via a learned affine mapping over discrete vectors, which yields a high-quality PTQ initialization and enables fully differentiable end-to-end optimization without explicit codebook lookup in the training forward pass. This strong post-training initialization makes LC-QAT highly data-efficient. Experiments across diverse LLMs demonstrate that LC-QAT consistently outperforms state-of-the-art QAT methods while using only 0.1%–10% of the training data. Our results establish LC-QAT as a practical and scalable solution for extreme low-bit model deployment.

ICML 2026

Optimal Regret for Policy Optimization in Contextual Bandits

Orin Levy, Yishay Mansour

We present the first high-probability optimal regret bound for a policy optimization technique applied to the problem of stochastic contextual multi-armed bandit (CMAB) with general offline function approximation. Our algorithm is both efficient and achieves an optimal regret bound of $\widetilde{O}(\sqrt{ K|\mathcal{A}|\log|\mathcal{F}|})$, where $K$ is the number of rounds, $\mathcal{A}$ is the set of arms, and $\mathcal{F}$ is the function class used to approximate the losses. Our results bridge the gap between theory and practice, demonstrating that the widely used policy optimization methods for the contextual bandit problem can achieve a rigorously-proved optimal regret bound. We support our theoretical results with an empirical evaluation of our algorithm.

ICML 2026

Transformed Latent Variable Multi-Output Gaussian Processes

Xiaoyu Jiang, Xinxing Shi, Sokratia Georgaka, Magnus Rattray, Mauricio A Álvarez

Multi-Output Gaussian Processes (MOGPs) provide a principled probabilistic framework for modelling correlated outputs but face scalability bottlenecks when applied to datasets with high-dimensional output spaces. To maintain tractability, existing methods typically resort to restrictive assumptions, such as employing low-rank or sum-of-separable kernels, which can limit expressiveness. We propose the Transformed Latent Variable MOGP (T-LVMOGP), a novel framework that scales MOGPs to a massive number of outputs while preserving the capacity to capture meaningful inter-output dependencies. T-LVMOGP constructs a flexible multi-output deep kernel by mapping inputs and output-specific latent variables into an embedding space using a Lipschitz-regularised neural network. Combined with stochastic variational inference, our model effectively scales to high-dimensional output settings. Across diverse benchmarks, including climate modelling with over $10,000$ outputs and zero-inflated spatial transcriptomics data, T-LVMOGP outperforms baselines in both predictive accuracy and computational efficiency.

ICML 2026

Correcting Overparameterization Effects in Fair Empirical Risk Minimization

Xiaoyi MAI, Jean-Michel Loubes

Bias mitigation is particularly challenging for overparameterized machine learning (ML) models. Overfitting of training points not only amplifies data bias induced by spurious correlations, but also causes the failure of usual bias mitigation methods. To provide actionable insights to address this challenge, we propose a precise analysis of fair empirical risk minimization (ERM) in the overparameterized regime. Importantly, we show that even though conventional fair ERM fails on overparameterized models, this approach can be corrected by modifying the equality fairness constraint to allow for bias overcompensation. Moreover, our analysis presents an empirical criterion for strong equalized odds: balanced group-conditional means of representer coefficients, indicating equal average contribution from each sensitive group. Motivated by this result, we provide an estimable search interval that localizes the required overcompensation level for balanced coefficients. Despite the asymptotic nature of our findings, they closely capture the statistical behavior of moderately large ML models.

ICML 2026

MixReasoning: Switching Modes to Think

Haiquan Lu, Gongfan Fang, Xinyin Ma, Qi Li, Xinchao Wang

Reasoning models enhance performance by tackling problems in a step-by-step manner, decomposing them into sub-problems and exploring long chains of thought before producing an answer. However, applying extended reasoning to every step introduces substantial redundancy, as sub-problems vary widely in difficulty and complexity: a small number of pivotal steps are genuinely challenging and decisive for the final answer, while many others only involve straightforward revisions or simple computations. Therefore, a natural idea is to endow reasoning models with the ability to adaptively respond to this variation, rather than treating all steps with the same level of elaboration. To this end, we propose MixReasoning, a framework that dynamically adjusts the depth of reasoning within a single response. MixReasoning enables fine-grained mode switching by training a lightweight concise LoRA adapter and control its strength to trigger switches based on reasoning difficulty estimated from sliding-window token confidence, yielding human-like transitions between fast and slow reasoning. The resulting chain of thought then becomes a mixture of detailed reasoning on difficult steps and concise inference on simpler ones. Experiments on AIME24, MATH-500, GPQA, and GSM8K demonstrate that MixReasoning shortens reasoning length by 13\%--49\% across benchmarks of varying difficulty, delivering consistent efficiency gains while maintaining performance.

ICML 2026

Efficient Distributed MLLM Training with Cornstarch

Insu Jang, Runyu Lu, Nikhil Bansal, Ang Chen, Mosharaf Chowdhury

Multimodal large language models (MLLMs) extend the capabilities of large language models (LLMs) by combining heterogeneous model architectures to handle diverse modalities like images and audio. However, this inherent heterogeneity in MLLM model structure and data types makes makeshift extensions to existing LLM training frameworks unsuitable for efficient MLLM training, especially in distributed training. In this paper, we present Cornstarch, an efficient distributed MLLM training framework that contemplates MLLM's unique characteristics in both model and data parallelization. Cornstarch introduces frozen-aware pipeline parallelism and workload-balanced context parallelism to improve MLLM training throughput. Our extensive evaluation shows that Cornstarch outperforms state-of-the-art solutions by $2.26\times$ on average in terms of MLLM training throughput. Cornstarch is an open-source project and available on [Github](https://github.com/cornstarch-org/Cornstarch).

ICML 2026

Predicting What Matters: Robust Generalist Robot Policy Learning via Future Semantic Mask

Yunfan Lou, Xiaowei Chi, Xiaojie Zhang, Zezhong Qian, Chengxuan Li, Rongyu Zhang, Yaoxu Lyu, Guoyu Song, Chuyao Fu, Haoxuan Xu, Pengwei Wang, Shanghang Zhang

World models derived from large-scale video generative pre-training have emerged as a promising paradigm for generalist robot policy learning. However, standard approaches often focus on high-fidelity RGB video prediction, this can result in overfitting to irrelevant factors, such as dynamic backgrounds and illumination changes. These distractions reduce the model's ability to generalize, ultimately leading to unreliable and fragile control policies. To address this, we introduce the Mask World Model (MWM), which leverages video diffusion architectures to predict the evolution of semantic masks instead of pixels. This shift imposes a geometric information bottleneck, forcing the model to capture essential physical dynamics and contact relations while filtering out visual noise. We seamlessly integrate this mask dynamics backbone with a diffusion-based policy head to enable robust end-to-end control. Extensive evaluations demonstrate the superiority of MWM on the LIBERO and RLBench simulation benchmarks, significantly outperforming the state-of-the-art RGB-based world models. Furthermore, real-world experiments and robustness evaluation (via random token pruning) reveal that MWM exhibits superior generalization capabilities and robust resilience to texture information loss.

ICML 2026

Decision Tree Learning on Product Spaces

Arshia Soltani Moakhar, Faraz Ghahremani, Kiarash Banihashem, MohammadTaghi Hajiaghayi

Decision tree learning has long been a central topic in theoretical computer science, driven by its practical importance. A fundamental and widely used method for decision tree construction is the top-down greedy heuristic, which recursively splits on the most influential variable. Despite its empirical success, theoretical analysis of this heuristic has been limited. A recent breakthrough by Blanc et al. (ITCS, 2020) provided the first rigorous theoretical guarantees for the greedy approach, but only under the uniform distribution. We extend this analysis to the more general and practically relevant setting of arbitrary product distributions. Our main result shows that for any function $f$ computable by an optimal decision tree of size $s$, maximum depth $D_{\text{opt}}$, and average depth $\Delta_{\text{opt}}$, the greedy heuristic constructs an $\epsilon$-approximating tree whose size grows at most with $\exp(\Delta_{\text{opt}} D_{\text{opt}} \log(e/\epsilon))$. In the special case where the optimal tree is a full binary tree, this bound improves upon the bound of Blanc et al. and holds under a strictly broader class of distributions. Moreover, we present an algorithm based on the top-down greedy heuristic that is entirely **parameter-free**---it requires no prior knowledge of the optimal tree's size or depth---offering a practical advantage over Blanc et al.'s method.

Keep browsing

More papers from top AI conferences

30+ top conferences, by conference, year, and topic.