NeurIPS 2026 Papers

Phase transitions occur when a physical system undergoes a dramatic transformation as it crosses a transition. We study whether regression methods trained only in one phase can predict the physical properties that characterize the unobserved phase. Our test sets consist of the analytically tractable Heisenberg model and angle-resolved photoemission spectroscopy measurements of a venerable high transition temperature superconductor. We find that polynomial regression and kernel methods recover the unobserved phase structure, while standard neural networks fail. To bound the error, we adopt the generalized eigenvalue problem (GEVP) into an upper bound on the worst-case transfer coefficient, defined as the ratio of mean squared error (MSE) loss on the extrapolation region to the MSE loss on the training region. Empirically, we find that our GEVP bound stays approximately constant with respect to the empirical transfer coefficient across training-set sizes and distance between the training and extrapolation region for both datasets, providing a principled framework for studying extrapolation feasibility.

One key application of autoencoders is to learn how to compress token sequences. Since the difficulty of compression depends on factors such as sequence length, vocabulary size, structural regularity, and input noise, it would be valuable to understand how these factors affect the performance of different architectures. Surprisingly, however, there is no benchmark for comparing autoencoder architectures that allows control over these aspects of difficulty. We introduce the Bottleneck Benchmark (BB), an open experiment framework in which encoder and decoder communicate only through a fixed-size latent and that allows to configure the previously mentioned difficulty aspects for any customizable data generator. As a case study, we evaluate BiLSTM, Transformer, and Bidirectional SSM encoders on copy, sort, and set tasks, finding that performance differences arise from inductive biases under noise and length rather than raw capacity.

Multinomial logit models (MNLs) are ubiquitous in machine learning, particularly in recommendation systems. While these models are typically evaluated using global metrics over a full catalog of items (such as $\ell_2$-error), practical applications often involve making predictions on small, curated subsets or ``slates'' of items. A model that minimizes global error may nevertheless yield very poor predictions on specific sub-universes. Yet, evaluating performance across all possible subsets remains computationally challenging. In this paper, we present a set of new efficient algorithms to evaluate the small-slate performance of MNL models. Using these tools, we conduct an empirical study comparing standard non-adaptive and adaptive learning algorithms. We demonstrate that while methods that compute the Maximum Likelihood Estimate (MLE) suffer from poor worst-case performance on small slates, specialized adaptive algorithms that dynamically sample difficult subsets can mitigate this issue. Conversely, we find that non-adaptive algorithms often achieve competitive average-case performance; we provide a theoretical model explaining this phenomenon. We conclude with recommendations for practitioners regarding when to utilize adaptive sampling based on the specific robustness requirements of their application.

Modern end-to-end autonomous driving systems often suffer from fragile out-of-distribution (OOD) generalization and limited decision transparency, largely due to their reliance on raw sensory observations and susceptibility to spurious visual correlations. Although memory-augmented approaches have been explored to improve generalization, existing methods typically store raw observations, symbolic rules, or entangled latent features, making retrieval sensitive to surface-level similarity rather than decision-relevant driving structure. To address this limitation, we propose DriveMem, a plug-and-play framework that moves beyond raw observations by distilling and storing invariant driving memories for generalizable driving systems. DriveMem integrates two core components: Invariant Feature Abstraction that first distills perturbation-stable, decision-relevant representations from visual tokens, and Prototype Memory Bank that then stores and retrieves prototypical driving experiences grounded in invariant representations. Extensive open-loop and closed-loop experiments demonstrate that DriveMem improves OOD generalization and safety-related planning metrics across diverse unseen scenarios. Notably, under severe sensor disruptions, DriveMem achieves up to a 39.5\% relative reduction in Collision Rate compared with a strong domain-generalization baseline. These results suggest that building prototype memory in an invariant representation space is a promising approach for improving robustness, while offering prototype-grounded traceability for model behavior.

Generative molecular design is shaped by simple proxy benchmarks for drug-like properties and models pretrained on large pharmaceutical datasets. This combination drives strong benchmark metrics but limits transferability to domains structurally distinct from drug discovery. To overcome this limitation and drive discovery toward real, scientifically grounded targets, we introduce the Nanotechnology Molecular Optimization (NMO) Benchmark, which bridges machine learning (ML) and quantum materials science. NMO acts simultaneously as a rigorous testbed for the ML community and a discovery engine for nanotechnology research. The suite replaces proxy oracles with quantum simulations and introduces strict protocols that prioritize scientific utility over leaderboard-oriented overfitting. The physics-based NMO tasks impose hard structural constraints and rugged fitness landscapes, posing fundamentally new requirements on generative models. Notably, advanced molecular optimization methods underperform much simpler approaches on the NMO tasks. We develop a new baseline method identifying the critical components to solve the NMO tasks, including a novel representation for modeling structural constraints and a domain-agnostic pretraining strategy to eliminate pharmaceutical dataset bias. Our results surpass state-of-the-art physical properties and reveal previously unknown structural motifs, offering new insights for the nanotechnology community and demonstrating that ML can drive genuine scientific discovery.

The rapid advancement of large language models (LLMs) has expanded their potential in the legal domain. However, existing legal benchmarks remain largely English-centric and oriented toward common law, leaving a critical gap in evaluating LLMs for civil law systems that govern most jurisdictions worldwide. To address this gap, we introduce Vietnamese Legal Benchmark VietLegal, a cognitively grounded benchmark designed for the hierarchical and codified structure of Vietnamese law. Although instantiated in Vietnamese legislation, VietLegal provides a replicable evaluation framework for civil law systems characterized by complex statutory hierarchies and frequent amendments. Inspired by Bloom’s taxonomy, VietLegal assesses multiple levels of legal understanding through tasks that mirror real-world legal assistant use cases, including legal question answering, multi-step reasoning, and scenario-based problem solving. The benchmark contains 10,450 expert-annotated samples, each cross-validated against authoritative legal sources to ensure fidelity to practical legal workflows. By offering the first standardized legal benchmark for Vietnamese, VietLegal enables systematic assessment of LLMs in civil law contexts and supports the development of more reliable and interpretable AI-assisted legal systems.

Recent research-level physics-reasoning benchmarks for LLMs reveal a failure mode that lies not in knowledge or execution, but in modeling the world being reasoned about. When faced with unseen problem setups, models often silently alter given quantities, fabricate laws, or inject unstated assumptions to force the problem into familiar patterns, and subsequent reasoning then proceeds within this corrupted world. We define this premise-level divergence as modeling drift: the gap between a model's internal axiomatic world model (AWM) and the true physical world specified in the problem. Although RLVR and PRM have yielded substantial gains in mathematics and code, their reward shapes do not directly supervise world modeling in axiomatic domains such as physics. We propose Reinforcement Learning with Axiomatic World Modeling (RLAWM), which optimizes the policy by penalizing divergence between its inferred axiomatic world model and the true physics world. RLAWM probes physics reasoning at two hierarchical levels (modeling and solving), each scored by a physically grounded reward. The modeling reward targets abstract knowledge formulation, evaluating whether the model captures the true axiomatic world before attempting calculation. The solving reward targets instantiated knowledge reasoning, checking whether the model's concrete derivation remains consistent with the axiomatic world. This two-level structure separates what world the model believes it is reasoning in from how it reasons within that world. The framework operates exclusively during training, with no extra cost during inference. RLAWM improves performance by up to $+10.8$ points on PhysReason and $+5.2$ on PHYSICS over the baseline. RLAWM also exhibits strong zero-shot generalization, yielding a $2.45{\times}$ gain over the strongest baseline on unseen physics problems.

Physics-Informed Neural Networks (PINNs) struggle to capture high-frequency components of PDE solutions, a failure mode known as spectral bias. Most existing fixes adjust the loss, the sampling, or the input encoding, but the network itself remains agnostic to the equation it is supposed to solve. We take a different route and embed physics priors into the architecture. The network's internal spectrum is shaped to match the spectrum of the target PDE through three components that are trained end-to-end: a bounded coordinate warping that contracts regions where the solution varies sharply, a spectral attention layer driven by the PDE residual that emphasizes physically active Fourier modes, and an inter-harmonic gate that lets low-frequency channels modulate high-frequency ones. We test the design on four forward PDEs (Helmholtz, Wave, Klein--Gordon, Burgers), the Burgers inverse problem, and seismic reconstruction, including Gulf of Mexico field data. Across these tasks, the architecture clearly outperforms seven PINN baselines on accuracy and produces noticeably fewer artifacts. Code will be made publicly available upon acceptance.

Matrix-vector products (MVPs) are a key primitive in numerical linear algebra and machine learning. However, the naive quadratic running time for exact computation is prohibitive for large matrices, motivating approximate methods. In this paper, we study approximate MVPs under a natural ``lightness'' assumption, which bounds the total $\ell_1$-mass of a $n \times n$ input matrix $A$ by $\gamma n$. For general matrices, we give an algorithm for approximating $Ax$ up to additive $\epsilon \|x\|_2$ error in $O(\gamma n^{1.5}/\epsilon)$ query time and polynomial preprocessing. We extend this algorithm to kernel matrices by establishing a black-box reduction to Kernel Density Estimation data structures and analyzing a noisy entry sampling scheme. For kernel matrices, our algorithm does not require any preprocessing and improves the best-known running times for Gaussian kernel matrices of [Indyk, Kapralov, Sheth, Wagner; ICLR `25] by polynomial factors in $n$ and $1/\epsilon$, and also provides the first algorithms leveraging the lightness assumption for other kernel functions as well.

Complex Instruction Following (CIF) is a foundational capability for deploying Large Language Models (LLMs) in production-grade applications. It demands stable, precise outputs under increasing horizon length, deeply nested rules, and multi-turn interactions. Yet existing benchmarks are predominantly atomic and short-horizon, failing to capture the logical depth and distractor pressure of real-world settings. We introduce **AppCIF**, the first application-level CIF benchmark. AppCIF stratifies difficulty along two orthogonal axes: *Horizon Length* ($H$), the span over which distractors accumulate and critical signals must be tracked; and *Logical Depth* ($L$), the nesting depth of rules to be unfolded. Together, $\langle H, L\rangle$ defines four hierarchical complexity levels, scaling from clean few-step execution (L1) to extreme pressure deduction (L4). AppCIF comprises 466 curated samples across five application scenarios. It adopts a "**Prompt-as-Environment**" paradigm, pairing Complex System Prompts (averaging ~14k words) with executable Code Simulation Engines (averaging ~3.5k lines of Python) to sustain rigorous state-machine deduction. The benchmark is constructed via a **Labor-Divided Human-AI pipeline** with scenario-specific collaboration strategies, and evaluated on 12 representative LLMs. Results reveal sharp performance stratification: Hard Satisfaction Rate ranges from 1.1% to 45.9%, exposing severe bottlenecks in long-horizon, distractor-laden instruction execution.

Depth-of-field control is a fundamental tool in photography, yet post-capture bokeh editing from a single image remains challenging. A practical editor should handle images captured under arbitrary focus and aperture settings. Existing methods typically assume an all-in-focus input, or first recover an all-in-focus image before rendering new bokeh. Such pipelines can discard useful blur cues from the source image and propagate reconstruction artifacts into the final edit. We introduce **AnyBokeh**, a physics-guided framework for **any-to-any bokeh editing**. Instead of treating source blur merely as a degradation to be removed, AnyBokeh estimates the **source blur state** with a signed circle-of-confusion map and a disparity map. By modeling the linear relation between signed circle of confusion and disparity difference, AnyBokeh estimates a source-specific **optical fingerprint** and transfers the source optical characteristics to the desired focus and aperture setting. A generative editor conditioned on both source and target circle-of-confusion maps then performs relative blur synthesis, enabling spatially adaptive deblurring, preservation, and defocus rendering. To support physically supervised learning, we further construct a high-fidelity synthetic dataset with accurate depth, focus distance, and full EXIF metadata. Experiments on real-world benchmarks show that AnyBokeh achieves faithful and controllable editing across any-to-any bokeh editing, all-in-focus-to-bokeh rendering, and defocus deblurring, while avoiding all-in-focus reconstruction and test-time bokeh-level calibration commonly required by existing approaches. The code and dataset will be made publicly available.

Online forecasters sometimes know a regime boundary before any post-boundary labels arrive. Standard adaptive conformal methods still react through later coverage errors, so they can pay a detection-delay cost even when the boundary is public. We study conformal prediction with an announced partition of the stream and separate two post-break audits. The reset empirical quantile in segmented split-CP attains the minimax per-time probability-gap rate $\Theta(M^{-1/2})$ after $M$ post-reset scores, with a matching Le Cam lower bound. A pure bounded constant-step announced-break ACI recursion attains an $O(M^{-1})$ time-averaged empirical-frequency gap by telescoping, but this is a realized-frequency statement, not a guarantee for the next interval. The practical AB-ACI implementation used in experiments adds burn-in and projection, so its rate claim includes explicit correction terms and is not unconditional. Synthetic experiments recover both slopes and show short-window gains when the break is large. Two real-data applications are null under leakage-free protocols. ERCOT RTC+B gives 0.863 post-reform coverage for AB-ACI versus 0.880 for ACI, and FOMC yield breaks give 0.892 versus 0.900. The message is diagnostic: use the reset empirical quantile for per-prediction reliability, use AB-ACI only for long-run frequency audits, and expect gains only when the realized score shift is large enough to make detection delay costly.

Protein binder design has largely optimized for affinity alone, leaving conformational selectivity unaddressed: for allosteric targets such as kinases, nuclear receptors, and GPCRs, a binder that engages both active and inactive states provides no functional specificity regardless of how tightly it binds. We introduce **AlloGen**, a modular framework that decouples backbone generation from a learned state-selectivity scorer $Q_\theta$, an SE(3)-invariant interface graph transformer trained via a two-phase curriculum that first grounds interface geometry before imposing conformational discrimination. Because $Q_\theta$ is fully differentiable and generator-agnostic, it integrates with any backbone generator as a passive reranker or an active gradient-based guide without retraining. Trained on 65 targets spanning 15 protein families, $Q_\theta$ generalizes to held-out out-of-distribution targets where energy-based baselines fail entirely, and all 15 evaluated generator--guidance combinations achieve positive conformational selectivity averaged over the held-out targets, with the best reaching $\bar{S}=+0.677$. Our anonymous code repository can be found at https://anonymous.4open.science/r/AlloGen_NeurIPS-04CB.

Key--value (KV) cache quantization is widely used to reduce Large Language Model (LLM) inference memory, yet existing evaluations solely focus on measuring perplexity and accuracy without assessing the safety impact. In this study, we explore alignment preservation under KV cache quantization. Across eleven instruction-tuned models (3.8B--72B) and five benchmarks (1,894 prompts), we find that low-bit quantization can silently destroy safety alignment (Mistral-7B loses 15.2\% of its refusals at only $1.03\times$ perplexity, and no universal safe bit-width exists), with sharp model-specific phase transitions invisible to standard metrics. We identify that the root cause is geometric: safety features occupy a low-dimensional activation subspace $10^2$-$10^3\times$ more vulnerable to quantization noise than the full representation space perplexity averages over. Inspired by this observation, we propose **Per-Channel Reduction** (PCR), a diagnostic that classifies each model into one of three mechanistic failure modes: (i) *outlier-crushes-safety*, where safety lives in non-outlier channels collaterally damaged by outlier-driven scale factors; (ii) *outlier-as-safety*, where safety overlaps outlier channels and finer granularity cannot rescue it; (iii) *multi-layer dilution*, where safety is distributed across many layers and per-layer fixes fail. PCR predicts the correct mitigation direction on all nine primary models and one held-out model from an independent family (20 calibration prompts). PCR generalizes across unseen prompts, models, and production quantizers (KIVI: up to 97.2\% recovery), succeeding where attention-based allocation methods fail. The resulting training-free protocol (${\sim}35$ GPU-minutes) recovers up to 97\% of lost alignment at minimal memory overhead, addressing vulnerabilities confirmed in production vLLM serving with FP8 KV cache on NVIDIA GPUs.

Reliable prediction of material properties from atomistic structures requires machine learning models that respect SE(3) symmetry. Frame-based methods address this challenge by constructing equivariant coordinate systems that align SE(3)-equivalent structures into unified representations. While attractive for their flexibility and efficiency, existing frame-based methods face two major limitations: a single frame type is often insufficient for heterogeneous atomic environments, and frame constructions can become unstable near highly symmetric or degenerate configurations. To address these issues, we propose the Multi-Frame Adaptive Network (MFAN), which leverages multiple frame types and introduces a local adaptive mechanism to combine them according to atomic environments, enabling different geometric references across atoms and complementary local-global information for each atom. Building on this multi-frame architecture, we further incorporate a frame-quality-aware weighting scheme that downweights unreliable frames before degeneration, thereby improving the continuity of the resulting representation. Experiments on crystal property prediction benchmarks demonstrate the leading performance of MFAN, with consistent gains from adaptive multi-frame learning. Additional force prediction experiments show improved robustness in continuity-sensitive settings.

Large Language Models (LLMs) present a novel paradigm for power system dispatch (PSD), enabling operators to derive optimal dispatch strategies from multi-objective requirements and grid dynamics. However, isolated paradigms focusing on modeling or solving often yield decision bias or physical infeasibility. Meanwhile, end-to-end paradigms couple modeling-solving with reflection-feedback loops, obscuring whether failures stem from inaccurate modeling, invalid solutions, or insufficient feedback, thereby causing ineffective iterations. To address these challenges, we propose Agile Dispatch Agent (ADA), an agent based on two-time-scale evolution that decouples the optimization process into two asynchronous loops. On the fast time scale, the Planner actively acquires information to clarify complex requirements, while the Solver selects algorithms to adapt to the problem mathematical characteristics. On the slow time scale, the Summarizer module updates the modeling feasible region based on evaluations from the Judger. Under a finite-horizon Planner and standard two-time-scale stochastic approximation assumptions, our analysis provides local tracking guarantees for the fast decision process and the slow knowledge evolution. Experiments on L2RPN benchmarks with ambiguous requirements and renewable dynamics show that ADA outperforms state-of-the-art baselines.

We study connections between the sample cost of replicability, which requires an algorithm to produce consistent outputs in different runs, and the sample complexity in the high success probability regime for distribution testing algorithms. Specifically, we consider a stronger notion of replicability, which we term as strong replicability, and show that any $\rho$-strongly-replicable testing algorithm can be converted in a blackbox manner to a tester with at most $\exp(-\tilde{\Omega}(\rho^{-2}))$ failure probability under the same sample budget. Our conversion is optimal for many natural testers. As applications of our conversion theorem, we obtain tight sample lower bounds for strongly replicable testers for coin bias and closeness testing.

Systematicity is the capacity to generalize compositionally. Existing benchmarks primarily test behavioral systematicity, while representation-level analyses often leave open whether recovered signals are merely decodable or causally used during generation. We therefore introduce $\mathrm{Bounded\mbox{-}RS}$, an evaluation artifact and protocol for adjudicating these possibilities under bounded hypotheses: unsupported, present-only, causal-use, or underdetermined. The artifact is a controlled semantic-parsing setting with matched recombination tests, exact intervention pairs, and evidence routines for recoverability, interchange, necessity, and repair during generation. We apply $\mathrm{Bounded\mbox{-}RS}$ to the two prominent representation-level explanations of systematic generalization, binding and geometry. Within the family instantiations evaluated here, and under the same behavioral anchor and state semantics, the binding family satisfies the criteria for $\mathit{Causal\mbox{-}use}$, whereas the geometry family is adjudicated as $\mathit{Unsupported}$ under its frozen closure basis. Detector-truthfulness tests do not validate a geometry detector, so the geometry conclusion remains a claim about the tested family rather than about all possible geometric mechanisms. $\mathrm{Bounded\mbox{-}RS}$ thus provides a reusable way to make disciplined representational claims from behavioral systematicity tests.