NeurIPS 2026 Papers

Domain Incremental Learning (DIL) aims to continuously adapt to new domains while retaining the knowledge of previous domains. Existing DIL methods are generally built upon an idealized assumption of clean supervision. However, real-world data streams are often affected by label noise, giving rise to the more challenging Noisy Domain Incremental Learning (N-DIL) scenario. Under this setting, not only is intra-domain knowledge acquisition hindered, but inter-domain knowledge conflicts are also exacerbated, which amplifies catastrophic forgetting. To address these challenges, we propose a novel Cross-Domain Knowledge Separation and Positive Transmission (ST-Prompt) framework. Specifically, to mitigate inter-domain knowledge conflicts, a Cross-domain Prompt Knowledge Contrastive Isolator is developed to enhance domain-wise knowledge separation, thereby mitigating the cross-domain knowledge interference during inference. Furthermore, to improve intra-domain knowledge acquisition, a Synergistic Knowledge Transmission scheme is introduced, which extracts reliable knowledge from previous domains to facilitate noisy data learning in the current domain. These two components are mutually reinforcing, jointly promoting effective cross-domain knowledge learning and utilization. Extensive experiments on diverse benchmarks demonstrate that ST-Prompt achieves state-of-the-art performance. Our code will be released.

Cross-distribution generalization in longitudinal behavioral data remains a persistent obstacle. Models trained on one cohort degrade on new populations, and prior approaches operating through feature-distribution alignment or model-capacity scaling have not substantially improved cross-distribution performance. We diagnose the limitation as insufficient representational constraints during training. We propose holding the language model frozen, so that the coherence criterion is defined by its pre-trained language prior and cannot shift during training. We introduce PRISM, a frozen-backbone framework that decomposes each behavioral trajectory into temporal, spectral, and semantic streams and integrates them through directed cross-attention with instance-specific gating. Training uses a dual-path objective combining discriminative and coherence constraints on a shared representation. On GLOBEM, a widely used cross-cohort benchmark for behavioral health prediction, PRISM achieves 79.93\% out-of-distribution accuracy, 27.13 points above the previous best, with an in-distribution to out-of-distribution accuracy gap of 1.24 points. PRISM also achieves the highest OOD accuracy on three additional datasets (LifeSnaps anxiety, MFAFY engagement, CrossCheck schizophrenia symptoms), with consistently smaller ID-to-OOD gaps than fine-tuned VLM baselines. Ablations identify the frozen coherence constraint as the component responsible for the distribution-invariance pattern.

Multi-Level Intermediate Representation (MLIR) underlies modern ML compiler infrastructure, including TensorFlow, JAX via StableHLO, PyTorch's Inductor backend, and IREE. Yet it appears only in trace amounts in code-LM pretraining corpora. MLIR is also extensible by design: new dialects ship per application domain, so maintaining a fine-tuned model per dialect does not scale. We ask whether inference-time priors derived mechanically from each dialect's Operation Definition Specification (ODS) can substitute for gradient-based adaptation. We make two contributions. First, we release four natural-language-to-MLIR benchmarks across three dialects (MLIR-Spec-150, Linalg-Spec-30, StableHLO-Spec-30, and StableHLO-Held-Out-200), totaling 410 in-scope NL→MLIR pairs, plus a 25-program StableHLO-OutOfGrammar stress set (435 benchmark instances). Thirty of the Spec prompts, ten per dialect, also carry hand-authored input and expected-output references for functional testing (MLIR-Functional-Reference-30). All artifacts ship under Apache-2.0 with Gebru datasheets and Croissant 1.1 metadata. Second, on top of these benchmarks we build a three-layer schema-derived constraint stack: a context-free grammar over op signatures (C1), type-domain splits from an ODS-extracted type lattice (C2), and an SSA-scope validator driving five-retry rejection sampling (C3). Porting the stack from arith+func+memref+linalg to StableHLO required no new constraint-layer code. Empirically, on dialects whose verifier semantics are dominated by structural constraints, SmolLM2-1.7B with token-level schema-derived constraints matches or exceeds 15B–34B code LMs that run the strongest constraint enforcement their serving stack admits (post-hoc rejection), at a matched retry budget and 8–25× the per-generation speed: on linalg it reaches 80.0% verify-valid (three runs, each 80.0%, n=125), 21 to 44 pp above CodeLlama-34B, Granite-Code-34B, and StarCoder2-15B with non-overlapping CIs, and the lead survives a same-family fp16 precision control. The comparison is between systems, not models: under the baselines' post-hoc protocol, SmolLM2 itself falls from 52.0% to 24.5% on arith+func, a +27.5 pp masking contribution. On arith+func and on the templated parametric StableHLO-Held-Out-200, where verifier semantics turn on attribute values rather than structure, the same baselines match or beat the SLM; we scope these explicitly as non-win cells. We release benchmarks, decoder, every per-prompt generation, and a pinned reproducibility Dockerfile.

Multimodal geometry reasoning requires models to jointly perceive diagrams and perform symbolic derivations. Test-time verification methods such as majority voting assume voter errors are roughly independent, but in geometry this assumption breaks down: all candidates perceive the same diagram, so perceptual biases cascade into correlated errors that produce confident but wrong consensus. We observe that neuro-symbolic candidates contain compositional structure that can serve as a verification resource. We introduce SKETCH, a format decomposing each candidate into a neural pathway (natural-language claim) and a symbolic pathway (visual grounding → typed DSL → deterministic execution). These pathways are governed by different error processes—symbolic errors are correlated across candidates through shared perception, while neural errors are more dispersed. Our method, Compositional Consensus Verification (CCV), gates consensus on cross-channel agreement: because the two pathways fail for different reasons, accidental agreement is rare, making each concordant vote far more precise than a raw majority vote. Across four geometry benchmarks, CCV achieves 81.0% accuracy (+10.4 pp over majority voting) without external scorers or additional inference. Seven non-compositional baselines—including trained PRMs—cluster at 70–71%, while compositional cross-layer features jump to 79–81%, confirming that structural diversity is the core verification resource, and that a principled gating algorithm effectively unlocks its potential.

Predicting genetic perturbation responses across cell lines is challenging not only for models, but also for the metrics used to evaluate them. We introduce a strict cross-cell-line CRISPRi stress test on Replogle Perturb-seq, training on K562 and RPE1 (or additionally HepG2) and evaluating on held-out Jurkat. We pair this setting with two necessary controls: target-control-copy, which predicts no perturbation effect, and shifted-control baselines, which translate target-line controls by a source-derived mean shift. Across classical baselines, recent deep models, CellFlow, and context-conditioned flow-matching variants with five frozen single-cell foundation-model encoders plus a PCA/null context, we find that aggregate distributional metrics can be strongly driven by target-line cell-state variability rather than perturbation-specific response. Target-control-copy is competitive on aggregate distributional metrics, while shifted-control approaches CellFlow on Energy and substantially outperforms it on perturbation-specific Pearson. Used as fixed per-cell context for a shared downstream flow head, none of five frozen scFM encoders matches a PCA/null encoder in this strict cross-line setting. Finally, anchored mean-preserving residual-flow probes reveal a metric-specific trade-off: full-space residual flows improve $W_2$ but degrade per-gene moment matching, whereas low-rank constrained flows preserve moments but lose most $W_2$ gain. No tested method simultaneously dominates perturbation-specific response and biologically relevant distributional fidelity. These results argue for perturbation-aware cell-level evaluation in cross-cell-line virtual-cell benchmarks.

Selecting a pretrained encoder for a downstream task without retraining or labels is a standard preprocessing step in transfer learning. The dominant label-free score, RankMe, was validated within architecture families and is widely applied across them. We show that its cross-architecture use is unsound: rankings are systematically driven by the penultimate-layer dimension rather than representational quality, and on heterogeneous model pools the score performs worse than uniform random selection. We give a random-matrix explanation establishing this failure as inherent to any rotation-invariant score, and propose Subspace-Corrected Ranking (SCR), a family of label-free corrections that breaks rotation invariance via alignment with a fixed reference encoder. Across three architecturally distinct pools and 23 aggregated task units, the proposed SSR-dagger reduces mean regret from +0.165 (RankMe) to +0.071 (95% CI [+0.045, +0.099], p < 1e-4). Paired with a within-family score in a two-stage pipeline, it matches this quality at one-third of the forward-pass cost. Five labelled architecture evaluations suffice to match the best fully label-free strategy; K=35 labelled evaluations reduce regret to +0.018 --- a 5.7x improvement at the same forward-pass cost.

Reinforcement learning with verifiable rewards (RLVR) post-trains language models on multi-step reasoning by assigning a single outcome reward uniformly across all tokens in a trajectory, regardless of which steps contributed to success or failure. Improving credit assignment can address this limitation by enabling targeted refinement of faulty reasoning steps, rather than updating entire trajectories uniformly. Resets are one such simple mechanism, enabling more precise credit assignment by returning to an intermediate state and resampling its continuation, so that outcome differences can be attributed to decisions made at that point. We propose two such methods: Random-Reset Policy Optimization (RRPO), where reset states are drawn randomly from reasoning steps, and Self-Reset Policy Optimization (SRPO), where the model self-localizes the erroneous step in an incorrect trajectory and resets there. We analyze these methods within a Conservative Policy Iteration (CPI) framework. We extend it with a credit-assignment oracle that targets improvable states, defined as those whose advantage exceeds a threshold, and show that it yields provable improvements over random resets. Across models and reasoning benchmarks, SRPO consistently outperforms standard GRPO and RRPO by sampling multiple suffix continuations at a self-localized reset and learning from their rewards, using only the model itself with no external supervision.

Foundation models are central to building general-purpose embodied agents. Minecraft has become the dominant testbed for this paradigm, but its computational and engineering overhead limits rapid iteration and controlled analysis. Crafter offers a lightweight alternative yet lacks the foundation model ecosystem required for such research. We introduce CrafterDojo, a suite of foundation models, datasets, and data-generation toolkits that establishes Crafter as a testbed for foundation-model-based embodied intelligence, providing CrafterVPT, CrafterCLIP, and CrafterSteve-1. To address Crafter's data scarcity, CrafterDojo includes automatic tools that generate diverse synthetic demonstrations and aligned video-language pairs, yielding the CrafterPlay and CrafterCaption datasets. Experiments show that these datasets support effective foundation model training and enable hierarchical agents that outperform non-hierarchical baselines on long-horizon tasks. We release all code, models, and datasets.

Automated structural health monitoring is essential to prevent catastrophic infrastructure failures. Precise, pixel-level defect segmentation is needed to accurately assess structural integrity, but progress in defect segmentation for civil infrastructures has been held back by an extreme scarcity of data, which requires costly expert annotation. The need for data is accentuated by algorithmic hurdles intrinsic to the problem, including center-bias and the need to rely more on shape when inspecting nearly textureless building materials. To remove the bottleneck, we introduce **Cracks in the Foundation (CiF)**, the largest and most detailed civil infrastructure (instance) segmentation dataset to date, comprising $\approx$150,000 high-resolution images meticulously curated over five years in collaboration with civil engineering experts. With the help of this unprecedented data source, we expose a blind spot of current visual AI: despite the advent of promptable Foundation Models (FMs) and Vision Language Models (VLMs), and despite the impressive abilities of today's specialised segmentation models, it turns out that dense image understanding in the built environment is nowhere near solved. Our evaluations indicate that even the most recent zero-shot FMs face significant challenges when deployed on real-world infrastructure and even the performance of specialised models with domain-specific supervision plateaus at $\approx$25\% mAP. CiF establishes inspection of civil infrastructure, an elementary and seemingly easy perceptual task, as an open challenge that reveals fundamental weaknesses of present-day models trained predominantly on internet images, literally and figuratively highlighting *cracks in the current foundation model paradigm*.

Post-training quantization (PTQ) compresses large language models to low bit-widths using a small calibration set, making calibration selection a consequential but under-specified part of the quantization pipeline. We study calibration selection for weight-only LLM PTQ backends such as AWQ and GPTQ. In this setting, calibration data does not define an inference-time activation clipping threshold; instead, it determines activation-conditioned quantities used for weight reconstruction, saliency estimation, and scale search. We identify a failure mode in which a calibration set misses rare, high-magnitude input channels of quantized linear modules, causing the backend to underrepresent reconstruction-sensitive directions. Motivated by this observation, we formulate calibration selection as weighted maximum coverage over module-level activation outlier channels. Each candidate sequence covers the module/channel pairs it activates above a reference threshold, and each covered pair is weighted by a dimensionally consistent surrogate for its potential contribution to squared output reconstruction error. The resulting objective is monotone submodular, so greedy selection admits the classical $(1-1/e)$ approximation guarantee for the coverage objective. We instantiate this formulation in COVERCAL, a backend-agnostic calibration selector operating on cached activation masks. Across LLaMA-2, LLaMA-3, and Mistral models, under AWQ and GPTQ INT4 backends and five downstream evaluations, COVERCAL improves over the reported fixed-pool calibration baselines, with the largest gains at small calibration budgets. Under AWQ at $K=128$, COVERCAL improves MMLU by $1.2$--$1.5$ points over random calibration; at $K=64$, it matches or exceeds random calibration at $K=256$ on LLaMA-3-8B. The contribution is not a new quantizer, but a module-correct coverage formulation, an efficient greedy selector, and empirical evidence that weighted outlier coverage is a useful calibration signal for existing weight-only PTQ backends.

This paper studies the task of estimating heterogeneous treatment effects in causal panel data models with covariate effects. We propose a novel **Co**variate-Adjusted **DE**ep C**A**usal **L**earning (**CoDEAL**) framework, that cohesively deal with the underlying heterogeneity and nonlinearity of both panel units and covariate effects. CoDEAL integrates nonlinear covariate effect components (parameterized by a feed-forward neural network) with nonlinear factor structures (modeled by a multi-output autoencoder) to form a heterogeneous causal panel model. The nonlinear covariate component flexibly captures complex covariate influences on outcomes, and the nonlinear factor decomposition enables CoDEAL to effectively capture both cross-sectional and temporal dependencies inherent in the data panel. This latent structural information is subsequently integrated into a customized matrix completion algorithm, thereby facilitating more accurate counterfactual imputation. Moreover, the use of a multi-output autoencoder explicitly accounts for heterogeneity across units and enhances the model interpretability of the latent factors. We establish theoretical guarantees on the convergence of the estimated counterfactuals and demonstrate the compelling performance of the proposed method using extensive simulation studies and real-world data applications.

Building hydrogen station networks, electric vehicle (EV) charging grids, or vaccine distribution systems requires coordinating two separately operated graphs - demand-side facilities and the supply-side networks they depend on - where each operator's payoff depends on the other's choices. We formalize this as the Co-Evolving Graph Game (CEGG), a setting that sits between facility location, interdependent-network analysis, and graph-based multi-agent reinforcement learning (MARL) but is not jointly addressed by any of them. We propose Cross-Graph Attention with Policy Mirror Descent (CGA-PMD), pairing cross-graph attention - which lets each agent read only the partner-graph nodes structurally coupled to its own - with a policy mirror descent update for stable training. Across three reward domains and a range of practical network scales, we find that the structure of the coupling selects which method wins: at small scale most methods are competitive, but as scale grows or coupling becomes irregular, only CGA-PMD remains productive while own-only, unmasked-attention, and Proximal Policy Optimization (PPO)-based variants all fail in distinct ways. Our theoretical analysis explains why: PMD's Kullback-Leibler (KL) regularization admits an iteration-uniform cumulative drift bound while PPO's clip update does not, predicting CGA-PMD's stability in the hard regime where PPO-based methods collapse.

Generative modeling over discrete structures underpins applications across deep learning, from biological sequence design and code generation to large language models, yet generation often remains sequential, relying on autoregressive decoding or iterative refinement. We introduce Coupling Models, a one-step discrete generative model that learns a direct coupling between discrete sequences $x$ and Gaussian latents $z \sim \mathcal{N}(0,I)$. Unlike recent distillation methods that compress a pretrained multi-step sampler into fewer steps, Coupling Models train a purpose-built decoder to invert this coupling and generate samples in a single forward pass. The method also avoids complex continuous flows over the simplex and hand-specified data-to-noise couplings. Empirically, Coupling Models improve the strongest one-step baselines in each domain, reducing LM1B text-generation perplexity by 33% at its lowest-perplexity operating point, Fly Brain enhancer-design FBD by 18%, and MNIST-Binary FID by 46%. These results suggest that effective one-step discrete generation depends strongly on how data and noise are coupled before decoding.

We investigate the problem of learning neural operators for coupled multi-physics PDEs, where multiple physical processes co-evolve and influence one another via bidirectional coupling. For example, in subsurface CO$_2$ storage, fluid flow, heat transfer, and geo-mechanical deformation evolve in a tightly coupled manner. Prior work, including neural operators and generative PDE solvers, mostly focuses on single-physics settings or relies on iterative inference, limiting their effectiveness in coupled PDEs. This paper introduces coupled-flow, a one-step generative neural operator that revisits coupled operator learning as solving a distributional transport problem. It parameterizes process-specific average velocity fields conditioned on the full coupled state, enabling cross-process interactions to emerge naturally from the transport dynamics. We further establish a Wasserstein-2 generalization bound that ties the prediction error of coupled-flow to its training losses through the regularity of the induced flow. Experiments on a wide variety of multi-physics benchmarks demonstrate that coupled-flow achieves significant performance improvements over coupled neural-operator and generative PDE baselines.

Physics-Informed Neural Networks (PINNs) solve forward PDEs by minimizing residual losses from the governing equations with initial and boundary conditions, but they often struggle with discontinuities such as shocks. In contrast, finite volume methods (FVM) handle discontinuities by enforcing integral conservation, which admits weak solutions. Motivated by this, we propose a \emph{Coupled Integral PINN (CI-PINN)} that augments a standard PINN with an auxiliary network for integral potentials and coupled integral constraints. This improves robustness near shocks while avoiding meshing and the numerical flux integration/reconstruction used in classical schemes. We validate CI-PINN on forward benchmarks including Burgers, Buckley--Leverett, the Euler system, and the Shallow-Water equations.

Guidance methods are essential tools for steering the generation process in diffusion models, improving the generalizability of stochastic robotic controllers and enabling high-quality controlled image synthesis. Despite strong parallels between flow-matching and diffusion, integrating guidance into flow-matching models remains quite challenging. Recent work has proposed general guidance frameworks for flow-matching, however, they are direct adaptations of diffusion guidance methods and can prove unstable. In this work, we conduct a theoretical analysis of guidance in flow-matching and show that a naive translation of diffusion-based guidance to flow-matching breaks the coupling distribution used during training, resulting in weaker control over sampling accuracy. Building on this insight, we propose **Coupled Guidance**, a flow-matching-specific guidance mechanism derived from a geometric perspective that leverages the coupling structure inherent to flow-matching training. To validate the effectiveness of our approach, we conduct extensive experimentation on a custom robotic *collision avoidance* task using Maniskill3, four robotic *dexterous manipulation* tasks from the Adroit benchmark, and three image generation tasks, namely *inpainting*, *super-resolution*, and *deblurring*, using the ImageNet-128 and CelebA-HQ-256 datasets. Our findings show that our framework improves sample quality and guidance stability across all tasks, without imposing any additional overhead.

Crowd counting is a vital computer vision task with wide-ranging applications, from public surveillance to urban planning. A persistent challenge is achieving scale invariance despite noisy labels, as humans often make counting errors with similar objects. Most existing methods struggle to maintain consistency and robustness when testing images differ in scale from the training data. We propose ScaLe-aware Invariance and Correcting Errors (SLICE) for robust crowd counting, a novel scale-aware loss function that improves scale invariance, enabling consistent and robust predictions under varying scales and annotation noise. The loss function combines two components: a scale-invariance term that applies multiscale supervision to enforce consistency across density maps of the same scene, and an error correction term that addresses inevitable annotation noise by modeling label uncertainty with probabilistic methods such as a mixture of Gaussians. SLICE is model-agnostic and integrates easily into existing crowd counting architectures without extra computational overhead during inference. Extensive experiments on five benchmark datasets show that SLICE greatly improves scale invariance and cross-dataset generalization, achieving the state-of-the-art results in challenging scenarios. Using CrowdDiff, Steerer, and CrowdFormer as baselines, SLICE lowers mean absolute error (MAE) by 5–7\% by addressing annotation noise and scale variation. Applying the scale-invariance term across multiple scales further cuts MAE by up to 15\%.

Reinforcement learning has rapidly become the dominant paradigm for training software engineering (SWE) agents on long-horizon, multi-turn tasks such as resolving real-world GitHub issues. Yet existing pipelines suffer from a fundamental credit-assignment bottleneck: rewards arrive only at the end of trajectories that may span dozens of file edits, shell commands, and test invocations, leaving every intermediate decision indistinguishable from luck. We observe that containerised SWE tasks provide a particularly useful form of environment forkability: a commit hash plus a container image specifies a real executable software state that can be restored cheaply and exactly, with auditable provenance and without a simulator-to-reality gap. We exploit this property by introducing Counterfactual Rollout Replay (CRR), a training-time procedure that re-executes the environment from selected decision points with alternative actions and assigns each selected step an advantage equal to the sampled on-policy versus counterfactual return differential. CRR requires no learned process reward model, no human rubric annotation, and no oracle hindsight signal; it is free of auxiliary process labels and reward-model training, while still incurring extra environment-replay compute. Applied to a 14B base model on SWE-Gym and evaluated on SWE-bench Verified and SWE-rebench, CRR-trained agents reach higher resolution rates with substantially fewer trajectories than outcome-only GRPO and complement orthogonal advances such as PRM-based scoring and trajectory search.

We study finite-horizon two-player zero-sum perfect-recall partial-information games and ask for the minimal finite-dimensional state sufficient to answer all rooted observation-measurable unilateral continuation queries. For each player $i$ and information set $I$, we consider the span $\mathcal L_i(I)$ of feasible counterfactual beliefs and the effective local query space $\mathcal E_i(I)$ generated by rooted continuation queries restricted to $\mathcal L_i(I)$. We prove that its dimension $d_i(I)$ is simultaneously the rank of the local counterfactual query operator, the minimum core-query basis size, the minimum exact local realization dimension, and the dimension of the canonical quotient $\mathcal L_i(I)/\mathcal N_i(I)$, yielding reduced counterfactual predictive-state representations, unique up to similarity, with exact first-return update operators. Aggregating the local ranks defines the deviation-conditioned predictive-state dimensions $d_t^i$, $d_t$, $d_\star$, and $D_\star$. We then prove that local rooted payoff-query control transfers to global exploitability, and under local design richness and bounded one-step parameter norms obtain a rollout-based upper bound in these minimal coordinates. We complement this with a minimax lower bound showing that $\Omega(d/\gamma^2)$ local rollouts are necessary even when the peak DCPSD equals $d$, and with an exponential separation theorem exhibiting games with $d_\star=m$ but observable continuation-trace complexity $2^m$. Finally, we show that DCPSD recovers finite-rank predictive-state, observable-equivalence, and linearly parameterized continuation-law models as special cases.