NeurIPS 2026 Papers

Deep learning systems are known to exhibit *implicit regularization* (alt. *implicit bias*), favoring simple solutions instead of merely minimizing the loss function. In some cases, we can analytically derive the implicit regularization -- connecting it to an equivalent penalty that augments the learning objective. However, modern deep learning systems are complex, carrying modifications to the training procedure and architecture (e.g. early stopping, minibatching, dropout) whose effects are not always directly interpretable. Although estimating the resulting implicit regularization could aid theorists in algorithm design and practitioners in interpreting their hyperparameter choices, this problem has received little direct attention. It is also tractable: regularization makes weight updates deviate from loss gradients, promising a signal for identifying implicit bias. Here we provide gradient matching methods that can be used to empirically estimate the implicit regularization. Our method works on networks with known regularization, recovering popular explicit penalties like $\ell_1$ and $\ell_2$. It also replicates known implicit effects, like the quadratic weight penalty induced by early stopping in gradient descent, demonstrating that it can be used to test theories of implicit regularization. Crucially, because our method is empirical, it can handle implicit regularization in arbitrary networks. We demonstrate this use by characterizing the effects of dropout in deep networks, showing implicit $\ell_2$ effects in this popular method. Our work shows that practitioners can use gradient matching to understand regularization in networks with implicit biases that are too complicated to derive analytically.

Flow and diffusion-based generative models achieve state-of-the-art quality, but require multiple forward passes at inference time, making them computationally costly. Consistency models overcome this by learning a flow map directly, enabling few‑step generation and dramatically reducing the computation needed at test time. However, formal statistical guarantees for such one‑step generators are scarce. Existing results either suffer from the curse of dimensionality or rely on impractical loss functions. We investigate whether one‑step flow-based models can escape the curse of dimensionality under the assumption that the data distribution is generated by a low-dimensional surface perturbed with Gaussian noise. In this setup, we derive approximation and generalization error bounds for the flow map estimate that circumvent the curse of dimensionality. Our findings provide the first principled explanation for practical success of one-step flow-based generative models.

Inference-time search (e.g., Tree of Thoughts, ToT) with large language models (LLMs) often concentrates on a small set of structurally or semantically similar trajectories, leaving alternatives underexplored—a failure mode we call \textit{reasoning basin collapse}. We introduce \textsc{BASIN}, a training-free selection method that penalizes repeated visits to the same reasoning basin, defined symbolically for arithmetic tasks or semantically via hypothesis clustering, thereby reallocating search across diverse reasoning strategies. Under matched inference budgets, \textsc{BASIN} improves over ToT by up to $+22$pp on Game of 24. To explain why this quality-agnostic penalty is effective, we introduce the \emph{redundancy gap} $\Delta$, which measures how differently search concentrates for correct versus incorrect predictions: standard ToT often operates near $\Delta \approx 0$, failing to distinguish correct from incorrect searches by their visit patterns, while \textsc{BASIN} consistently induces $\Delta > 0$, concentrating search on correct basins while dispersing incorrect ones. More broadly, \textsc{BASIN} suggests structure-aware selection as a simple and general approach to improving inference-time reasoning.

Offline goal-conditioned reinforcement learning remains challenging in stochastic long-horizon settings, where compounding value estimation errors hinder reliable goal reaching. While prior methods have sought to address this challenge through various approaches, a key challenge remains path mirage, where agents overcommit to spuriously successful trajectories in offline datasets induced by stochasticity, often leading to failure on difficult tasks. To address this issue, we propose Branch-aware Graph Planning (BGP), which captures stochastic branching structures by identifying high-variability points as graph nodes and designing goal-conditioned edges with segmented learning to avoid unreliable branches and reduce unnecessary stochasticity along paths. As a result, BGP enables reliable goal reaching even in highly stochastic long-horizon environments, and experiments on diverse OGBench tasks show that it substantially outperforms prior state-of-the-art offline goal-conditioned RL methods.

We derive a tight generalization bound for deterministic neural networks based on their last hidden layer representations. By evaluating representation quality with a parameter-free $k$-nearest-neighbor ($k$-NN) classifier, our certificate avoids traditional parameter-space complexity measures, KL divergences to a posterior, weight norms, and compression. The bound is tight: trained from scratch, it reaches an 8.0 percentage point gap to test error on CIFAR-10 (ResNet-18) and yields the first non-vacuous certificates we are aware of on CIFAR-100; with frozen DINOv2 representations, it certifies at 1.4\% on CIFAR-10 and 23.1\% on 1,000-class ImageNet. It decomposes additively into three empirical, independently measurable quantities: local class geometry ($k$-NN decoder error), decoder-network agreement, and training stability. This allows the bound to act as a diagnostic tool: we demonstrate that actively destroying local class geometry eliminates generalization while leaving dataset memorization intact, identifying class geometry as a causally relevant mechanism for generalization. Finally, we show that learned representations are significantly less sensitive to weight noise than linear readouts, suggesting why parameter-space certificates often struggle to remain tight.

Diffusion Transformers enable powerful in-context image editing from a reference image and a text prompt, but also make it easier to maliciously edit public images while preserving the private identity of the reference. Existing DiT-oriented protections, such as DeContext, mainly suppresses context-to-target attention to weaken reference utilization. However, this strategy can be insufficient for complex edits where prompt semantics already dominate generation. Instead, we study proactive protection for DiT-based editors from a conditional-flow perspective, and use a local geometric interpretation of in-context editing, where the denoising trajectory is empirically characterized as being steered by prompt-consistent and reference-consistent directions. Thus, effective protection should not simply corrupt the output, but should drive it away from the original context manifold. To this end, we propose Conditional Flow Hijacking, which adds imperceptible perturbations to the reference image. Instead of suppressing context attention, our method preserves the context pathway, but redirects the context-induced steering signals in the intermediate transformer layers, causing the final trajectory to drift away from the reference consistent solution region. Experiments on FLUX.1-Kontext with different datasets and prompts including attribute and complex scene editing show that our method significantly suppresses reference identity leakage while maintaining image quality and prompt alignment, outperforming prior protection baselines, especially under complex scene editing prompts. We further verify the robustness of the proposed protection method as well as its transferability across other models, highlighting the need for proactive defenses for in-context generative models.

We study semi-offline value estimation, where a small on-policy sample supplements a large off-policy dataset. This regime is common in recommender systems and pre-deployment A/B testing, and the challenge is to use the limited on-policy data to correct systematic bias in a given value predictor. Recent work on Bellman calibration (BC) gives a post-hoc, model-agnostic correction along the predicted value. We propose Error-Guided Bellman Calibration (GBC), which trains an error model on the on-policy sample and calibrates the predictor along this learned error axis. We prove a completeness-free refinement guarantee, a theorem showing that the correction adapts to the intrinsic dimension of the prediction bias, and a finite-sample bound on the value-estimation error that isolates our advantage in the refinement term. Across synthetic, CRM, and D4RL benchmarks, GBC improves over existing calibration baselines.

Designing continuous trajectories whose time-averaged occupancy provably matches a prescribed spatial density (the *ergodic coverage* problem) is central to UAV-assisted data collection and sensing, robotic exploration, and mobile monitoring. For flying agents in particular, this challenge is acute: trajectories must balance coverage fidelity against tight energy budgets, no-fly zones, and acceleration limits. Existing methods either re-optimize each trajectory online (with cost growing in the horizon and re-running for every target, agent, and realization) or rely on bespoke analytical constructions that must be re-derived for each new constraint. We propose a *pushforward* framework that decouples ergodicity from density matching: an analytic latent trajectory provides exact uniform ergodicity on a simple annular domain, and a single map, learned offline by optimal-transport conditional flow matching, transports this latent occupancy onto the prescribed target density. The composed trajectory is then asymptotically ergodic with respect to the learned pushforward distribution, with deviation from the target controlled by the flow-matching training loss. Once trained for a given target density and constraint set, the map serves an unbounded number of trajectories and a multi-agent fleet without per-agent retraining, and many differentiable operational constraints (no-fly zones, acceleration ceilings, or fairness penalties) enter as additive soft penalties in the training loss without re-deriving the design. We prove three results (an acceleration-energy bound, an $O(1/\sqrt{K})$ ergodic convergence rate in the number of trajectory cycles $K$, and an approximation-error bound) that combine into an end-to-end coverage bound estimable from CFM training diagnostics (certified given an architectural Lipschitz bound on $v_\theta$). Experiments on synthetic targets empirically support each theoretical envelope; on a real UAV-coverage dataset, the proposed method gives the strongest observed coverage--energy and coverage--constraint tradeoffs among the evaluated time-warping, optimization, and concurrent flow-matching baselines.

We consider the problem of synthesizing Clifford quantum circuits for devices with all-to-all qubit connectivity. We approach this task as a reinforcement learning problem in which an agent learns to discover a sequence of elementary Clifford gates that reduces a given symplectic matrix representation of a Clifford circuit to the identity. This formulation permits a simple learning curriculum based on random walks from the identity. We introduce a novel neural network architecture that is equivariant to qubit relabelings of the symplectic matrix representation, and which is size-agnostic, allowing a single learned policy to be applied across different qubit counts without circuit splicing or network reparameterization. On six-qubit Clifford circuits, the largest regime for which optimal references are available, our agent finds circuits within one two-qubit gate of optimality in milliseconds per instance, and finds optimal circuits in 99.2\% of instances within seconds per instance. After continued training on ten-qubit instances, the agent scales to unseen Clifford tableaus with up to thirty qubits, including targets generated from circuits with over a thousand Clifford gates, where it achieves lower average two-qubit gate counts than Qiskit's Aaronson-Gottesman and greedy Clifford synthesizers.

In many interactive settings, from social and institutional environments to multi-agent systems, agents must infer latent behavioral structure from limited signals while their own actions shape the data they observe. This endogeneity violates the stationary, exogenous assumptions underlying standard likelihood-based methods, leading to failures in adaptation despite accurate clustering. We propose \emph{Epistemic Social Learning} (ESL), a two-timescale framework that couples belief-based inference with adaptive representation learning under endogenous interaction. At a fast timescale, agents maintain Bayesian beliefs over shared behavioral prototypes. At a slower timescale, prototypes are updated from belief-weighted interaction data via the recursion $\Theta_{m+1} = \Theta_m + \gamma_m \widehat{H}_m$, aligning learned representations with the interaction distributions induced by agent behavior. We formalize ESL as a stochastic approximation process with controlled Markov noise and show that its dynamics track the differential inclusion $\dot{\Theta} \in \mathcal{G}(\Theta)$, whose limit sets correspond to self-consistent interaction regimes. Across repeated games with behavioral shifts, ESL reduces post-switch regret by $26\%$ over strong baselines while improving decision-relevant prediction. These results reveal a fundamental gap between latent identification and adaptive performance under endogenous interaction.

We introduce epistemic pairwise maximin share (EPMMS), a new fairness notion for fair division of indivisible goods. Two fundamental notions in this setting are envy-freeness up to any item (EFX) and pairwise maximin share (PMMS), with PMMS being stronger than EFX. While EFX has been extensively studied, far less is known about PMMS. Recent work shows that relaxing EFX via an epistemic perspective leads to substantial progress on the EFX problem, raising the question of whether a similar approach can advance our understanding of PMMS. Motivated by this, we initiate the study of EPMMS, the epistemic relaxation of PMMS. EPMMS is more challenging than EEFX: the key approaches underlying recent progress on epistemic EFX inherently fail to extend to EPMMS. We establish the following results. 1. For additive valuations, $4/5$-EPMMS allocations exist and can be efficiently computed. 2. For bivalued valuations, EPMMS allocations exist and can be efficiently computed; in fact, we obtain the stronger guarantee of epistemic groupwise maximin share (EGMMS), which also strengthens the existence of MMS allocations for this setting. 3. EPMMS allocations exist in two settings where MMS allocations need not exist: instances with three additive agents or two types of additive agents.

Zero-shot mutation effect prediction is widely used to evaluate protein language models (PLMs). However, because existing benchmarks mix additive and non-additive mutations, and non-additive effects (epistasis) are sparse yet critical, it remains unclear whether these models truly capture thermodynamic residue coupling. As a result, strong overall performance may mask weaknesses in modeling multi-residue interactions. To make this distinction explicit, we introduce EpistasisBench, a thermodynamically stratified zero-shot benchmark constructed from large-scale stability measurements. By isolating significant and sign epistatic double mutations, it separates additive effects from thermodynamic coupling and defines two complementary tasks: ∆∆G prediction restricted to epistatic subsets, and a direct ∆∆Gint scoring task that examines non-additive structure in model probability space. When evaluated across more than 17 state-of-the-art PLMs, EpistasisBench reveals a consistent pattern. Performance declines under strict epistasis filtering, strong single-site predictors fail on coupled mutations, and standard masked language model scoring shows no variation in ∆∆Gint, thereby preventing detection of non-additive effects. Together, these findings indicate that the limitation lies in common zero-shot inference schemes rather than model capacity. To demonstrate that non-additive signal can in fact be recovered under a more suitable inference formulation, we introduce a simple joint marginal inference mechanism that restores non-additive signal and improves performance without fine-tuning. Overall, EpistasisBench provides a physically grounded test for multi-residue interactions and highlights a key gap in current zero-shot evaluation of protein language models. The dataset is available at https://www.openml.org/d/47245

Modern systems are often expected to transfer across tasks that were not specified during training, leading to the question: what data facilitates generalization in new, unanticipated settings? One hypothesis is that data with more structural information could contain shared circuits and subprograms that could be recycled in a wider array of downstream settings. Epiplexity, a recently proposed measure of the structural information a compute-bounded learner can extract from data, provides a mechanism to reason about this relationship. In this paper, we show how to operationalize epiplexity as an online training signal for data selection and generation. For selection, we fit scaling laws to the training loss curves of natural data domains to predict the expected epiplexity gain as a function of training tokens, and use this signal to adaptively determine the sampling weights over domains during training. For synthetic data generation, we define a generator's reward as the change in learner epiplexity over a buffer of previously generated data and use REINFORCE policy gradients to guide the generator toward an epiplexity-maximizing distribution. In both cases, higher epiplexity predicts improved downstream performance on zero-shot and fine-tuning based tasks, supporting the hypothesis that data rich in structural information yields representations that transfer across domains.

Mapping fine-scale 3D chromatin landscapes is critical for understanding gene regulation, yet the resolution of Hi-C data remains fundamentally constrained by both physical bottlenecks and sequencing costs. Although deep learning has shown promise in super-resolving Hi-C data, existing methods suffer from absolute coordinate loss during localized patching and a lack of structural and epigenomic priors in unconditioned generation. We propose a novel epigenomics-guided flow matching framework EpiFlow for 3D genome super-resolution. EpiFlow introduces two key innovations: (1) an EpiCond module that deterministically maps 1D epigenomic features to 2D spatial constraints via orthogonal expansion with built-in positional grounding, avoiding quadratic attention complexity; and (2) a Toeplitz-regularized classifier-free guidance mechanism that enforces distance-decay priors using a learnable symmetric prototype matrix. Extensive evaluation shows that our method achieves state-of-the-art cross-cell-type loop detection, improving recall by $2.8\times$ over raw Hi-C on unseen H1-hESC (Loop $F_1$ of $0.325$ vs. $0.199$) and outperforming all baselines on same-cell HFFc6 ($F_1=0.520$). Aggregate Peak Analysis (APA) confirms biological validity, and strong generalization to a held-out cell line demonstrates that the model learns transferable principles of 3D genome folding rather than cell-specific biases.

Latent world models based on Joint-Embedding Predictive Architecture (JEPA) are deterministic by design. While successful in fully observable scenarios, this paradigm breaks down when past observations and actions lead to multiple plausible future possibilities, e.g., due to occlusion. We introduce EpicWorldModel, a framework to train stochastic JEPAs for environments and tasks with inherent uncertainty under partially observability. We jointly train the EpicWorldModel predictor with its latent representation space to directly predict multiple potential future states using a flow-matching objective, when the goal-relevant scene content is absent from the conditioning history. We show that flow predictive variance approximates an upper bound on Expected Information Gain and serves as a useful exploration guidance for planning. By incorporating this uncertainty signal into Cross-Entropy Method (CEM)-based planning, our approach balances goal-reaching with exploration of uncertain regions where occluded or unseen goals are most likely to be located. We demonstrate the effectiveness of EpicWorldModel through a series of latent planning experiments with the best performance among all baseline methods, showing up to $22\%$ empirical improvement in success rate over LeWorldModel.

Video understanding is commonly built on action-triggered responses or scene-level aggregation, implicitly assuming clear temporal boundaries. This assumption breaks in dense-action streams such as gameplay and sports, where overlapping actions induce continuous latent dynamics without explicit transitions. The challenge is further amplified in streaming settings, where models operate under strict causal constraints and must progressively construct episodic memory for downstream reasoning. Existing segmentation-based strategies therefore often produce fragmented or semantically mixed episodes, leading to degraded understanding. We present EpiStream, an online framework for utility-aware semantic episode formation in dense-action streams. Instead of detecting event boundaries, EpiStream learns when an evolving video prefix should be committed into an episode unit that maximizes downstream utility under limited memory budgets. The framework consists of three steps: (i) utility function design for semantic coherence, inter-episode distinctiveness, and downstream objectives; (ii) utility-optimal commit advantage construction from teaching signals; and (iii) peak-aware advantage learning for training a causal online commitment policy from prefix-only observations. Extensive experiments show that EpiStream substantially improves temporal semantic coherence and consistently benefits challenging downstream tasks, including real-time advice and player intention prediction.

The simplex method is one of the most widely used algorithms for linear programming, but its practical efficiency is highly sensitive to pivot selection and difficult to predict due to structural uncertainty in the search process. We propose EpiPivot, which casts pivot selection as a controllable decision process under epistemic uncertainty. EpiPivot identifies this uncertainty as two-dimensional: temporal dependence across the optimization trajectory and joint uncertainty across candidate pivot rules, and addresses both through temporal attention and an Epistemic Neural Network with shared latent sampling. At decision time, Thompson sampling combines these uncertainty-aware predictions into robust pivot choices. EpiPivot achieves significant improvements over classical and learning-based baselines, reducing pivot cost by up to 79\% over the best-performing classical rule on large-scale instances where most classical rules fail to converge within the time limit.

As AI models evolve from text and vision to physical agents, embodied AI faces a fundamentally different attack surface. Prior attacks on embodied systems have largely focused on semantic or instruction-level manipulations, such as prompts, adversarial images, and action commands; by contrast, we study environment-only perturbations that require no access to the model or instructions. We reveal a new attack surface: the physical environment itself. An adversary who rearranges objects, without modifying instructions or accessing models, can cause hazardous consequences. In this work, we propose EnvTrap, a diagnostic pipeline that constructs paired safe, trap, and null-trap (benign but misleading) embodied scenarios. We demonstrate that environment-only perturbations raise hazardous-action rates to an average of 86.5\% across multiple vision-language-action (VLA) models, with similar vulnerability patterns confirmed in world models. On a consequence-prediction task, model accuracy remains near chance, while human evaluators succeed easily. We further propose a consequence-aware defense that reduces trap trigger rates by an average of 76.3\% across VLA models in simulation and by 61.7\% on physical robots. This vulnerability arises because current embodied models can recognize scene state but often fail to predict action consequences under altered layouts. Our findings establish environment integrity as a prerequisite for safe embodied AI deployment. Our code and data are available at https://anonymous.4open.science/r/Envtrap-1BA4/

Diffusion-based large language models (dLLMs) rely on bidirectional attention, which prevents lossless KV caching and requires a full forward pass at every denoising step. Existing approximate KV caching methods reduce this cost by selectively updating cached states, but their decision overhead scales with context length or model depth. We propose EntropyCache, a training-free KV caching method that uses the maximum entropy of newly decoded token distributions as a constant-cost signal for deciding *when* to recompute. Our design is grounded in two empirical observations: (1) decoded token entropy correlates with KV cache drift, providing a cheap proxy for cache staleness, and (2) feature volatility of decoded tokens persists for multiple steps after unmasking, motivating recomputation of the $k$ most recently decoded tokens. The skip-or-recompute decision requires only $O(V)$ computation per step, independent of context length and model scale. Experiments on LLaDA-8B-Instruct and Dream-7B-Instruct show that EntropyCache achieves 15.2×–26.4× speedup on standard benchmarks and 22.4×–24.1× on chain-of-thought benchmarks against vanilla baselines, with competitive accuracy and decision overhead accounting for only 0.5% of inference time.