NeurIPS 2026 论文列表

Resolution invariance is a critical capability for scientific machine learning methods (SciML), however, the ability to sample at any resolution does not guarantee stable emulation at all resolutions. This important distinction underpins one of scientific machine learning's greatest potential: the ability to efficiently emulate complicated and computationally expensive systems on inexpensive coarse grids. Stable numerical solvers are computationally expensive as they attempt to resolve small scales that receive and dissipate spectral energy. Learned coarse-grid forecasters remove the dissipative channel, allowing unresolved frequencies to re-emerge as grid-scale noise (aliasing). We explore stable resolution invariant emulation through physically principled control over entropy dynamics to reinforce the dissipative mechanism within the Neural Discrete Equilibrium (NeurDE) method. Entropically-stabilized NeurDE produces stable long-time rollouts for thousands of forecasting steps on large Reynolds number turbulent flows (Re=50,000) with very coarse grids ($\mathbb{R}^{16\times16}$). Entropically-stabilized NeurDE demonstrates extreme stability at coarse resolutions, outperforming the data-generating stabilized numerical method: a much more challenging baseline than SciML surrogates.

Temporal causal structure learning aims to recover lag-aware causal relations from multivariate time series. While prior knowledge has been widely used to improve static structure learning, its integration into lag-aware structure learning remains limited, because available priors are often lag-agnostic. Recent studies have explored lag-agnostic edge presence priors, but partial order priors, which can prune the ordering space and improve structure learning, remain underexplored. A key challenge is that variables may influence each other at different lags, leading to cyclic variable-level relations where ordering is no longer naturally defined. Moreover, directly applying static differentiable formulations of partial order constraints can be unstable due to repeated-walk accumulation on cycles. To address this gap, we represent partial order priors on the summary graph and follow the natural idea of the order relation that only allows a single direction of possible causality between two variables, which is equivalent to forbidding all reversed causality, direct or indirect. This converts partial order constraints into path prohibition constraints that can be extended to cyclic summary structures. For stable differentiable characterization, we propose a decoupled partial order constraint method. The method preserves the original lag-aware structure for data fitting, imposes partial order constraints through a normalized summary representation, and captures higher-order reachability with a closed-form connectivity characterization. Experiments further demonstrate its effectiveness in nonstationary settings.

We propose a stable algorithm for solving maximum coverage problems under a cardinality constraint of $k$ in a record-level stability model, where adjacent instances $I\sim I'$ differ by the incidence of one universe element. Our algorithm computes a negative-entropy-regularized maximizer of the capped concave relaxation over the hypersimplex and rounds it with a common-seed rounding map from hypersimplex correlated sampling. The resulting sensitivity bound is $\min\{2k, \alpha_k k/\lambda\}$, where $\lambda$ is the regularization parameter and $\alpha_k = O(\log k)$. For utility, the expected coverage has approximation factor $1-\frac{1}{e}$ with an additive error of order $O(\lambda k \log \frac{m}{k})$, where $m$ is the number of input sets. We also report experiments on synthetic and real-world instances.

Long-horizon forecasting of time-dependent partial differential equations (PDEs) is critical for characterizing the sustained evolution of physical systems. While neural operators have emerged as efficient surrogates, they typically learn implicit finite-time transitions from discrete observations. When deployed autoregressively, such propagators often suffer from rapid error accumulation and dynamic drift. To address this, we propose a neural forecasting framework that reformulates PDE rollout as learning a Structured Spectral Propagator (SSP) in a propagation-oriented latent space. Following an analysis-propagation-synthesis design, our framework: (i) maps physical states into a shared, time-consistent spatial representation; (ii) projects this space into a compact propagation state to isolate recurrent dynamics from fine-grained spatial details, thereby decoupling reconstruction fidelity from rollout regularity; and (iii) evolves retained spectral modes using a frequency-conditioned linear backbone complemented by a nonlinear spectral closure to account for truncated interactions. This explicit structuring endows the propagator with a strong inductive bias for coherent modal evolution. Extensive experiments demonstrate that SSP significantly outperforms state-of-the-art baselines, reducing relative $L_2$ errors by up to 48.9\% and exhibiting improved stability in temporal extrapolation beyond the supervised horizon.

Generative Flow Networks (GFlowNets) learn to sample states proportional to an unnormalized reward. Despite their theoretical promise, practical training is often unstable, exhibiting severe loss spikes and mode collapse. To address this, we first assess the sensitivity of GFlowNet objectives, demonstrating that a small Total Variation (TV) distance between the learned and target distributions does not preclude an unbounded training loss. Motivated by this mismatch, we establish converse guarantees by deriving loss-to-TV bounds that certify global fidelity from bounded trajectory balance losses. Lastly, we propose \emph{Stable GFlowNets}, which leverages our theory to stabilize training via adaptive reference flow and improves the trade-off among mode coverage, robustness, and certifiability.

Deep learning has emerged as a powerful paradigm for constructing nonlinear asset pricing factor models. However, temporal distribution shifts such as bull-bear transitions and industry rotations undermine the generalization of models trained under the i.i.d. assumption. As a result, existing models tend to exploit environment-specific correlations that are highly predictive in-sample but unstable across market regimes. To address this challenge, we propose $\textit{\textbf{CASH}}$, a causality-inspired asset pricing factor framework designed to learn invariant and minimally sufficient representations under temporal distribution shifts. CASH formulates an information-theoretic adversarial learning objective that systematically discards transient market noise by optimizing against an adversary designed to exploit spurious correlations, thereby distilling invariant factor-return relationships. The resulting invariant representation is then integrated into a conditional factor pricing model via a dynamic factor exposure network. Experiments on real-world stock datasets across multiple markets demonstrate that CASH consistently outperforms state-of-the-art baselines in out-of-sample return prediction and exhibits superior robustness under pronounced temporal regime shifts.

Training neural networks directly in a low-rank parameterization is an appealing route to reducing memory, compute, and storage simultaneously during both training and inference. Dynamic low-rank training (DLRT), which confines weights to a rank-$r$ manifold via the Galerkin projection of the gradient flow, is particularly attractive because it identifies efficient subnetworks on the fly without specialized initialization or post-factorization. However, DLRT fails to find trainable networks under high compression. In this paper, we derive the gradient flow of the best rank-$r$ approximation and point out that the offset of DLRT comes from a curvature-coupling term which is large and thus non-negligible under aggressive compression. Guided by this analysis, we propose a stable dynamic low-rank training method, named SDLRT, which maintains a lightweight compensation buffer that reinjects the top neglected singular directions. Additionally, we introduce a negative feedback on the truncation tolerance to stabilize each layer's rank. Experimentally, SDLRT reliably finds trainable subnetworks where DLRT collapses and as a PEFT adapter on DeBERTa-v3, it achieves the best average score on SuperGLUE at only $2.8\\%$ parameter overhead over LoRA.

RL+Search is the computational backbone of modern AI systems for imperfect-information extensive-form games. The Recursive Belief-based Learning (ReBeL) framework already mitigates one source of training fragility by sampling the pivot iteration uniformly from the iteration budget; we identify a second, hitherto unaddressed source: the budget itself is held fixed across all subgames, creating a rigid coupling between solver maturity and outer-loop learning that propagates variance through the recursive self-play and produces noisy, seed-dependent training trajectories. We propose \textbf{Stochastic Horizon Annealing (SHA)}, which randomizes only this remaining static degree of freedom: at each subgame the total CFR iteration count is drawn from a Gaussian centered at the expected budget. Combined with ReBeL's existing uniform pivot, the procedure smooths the learning objective across a continuum of solving depths and behaves as an implicit ensemble at zero additional cost. SHA is a one-line change to any ReBeL-style trainer. We give a complete theoretical treatment with three results, all proved in the main text: (i) SHA preserves the $O(N^{-1/2})$ convergence rate of linear CFR; (ii) the per-subgame target variance is bounded by $C_\beta^2\sigma^2/(4N^3)$, vanishing rapidly with the budget; and (iii) the recursive variance of the value-network targets scales as $\Theta(D)$ for SHA versus $\Theta(D^2)$ for the fixed-horizon ReBeL, where $D$ is the recursion depth. On Liar's Dice 1$\times$4f, 1$\times$5f, and 1$\times$6f (30 seeds each) at the ReBeL reference setting of depth-2 subgames and 1024 search iterations, SHA reduces mean final exploitability by 9--18\% and shrinks the 95\% confidence interval across seeds by 34--63\%, with larger games benefiting more as predicted by the theory, establishing stability as a first-class, theoretically supported contribution for RL+Search on imperfect-information games.

While reinforcement learning (RL) has been central to the recent success of large language models (LLMs), RL optimization is notoriously unstable, especially when compared to supervised fine-tuning (SFT). In this work, we investigate the stability gap between SFT and RL from a gradient-based perspective, and show that the convexity of the SFT loss with respect to model logits plays a key role in enabling stable training. Our theoretical analysis demonstrates that this property induces favorable gradient directionality during optimization. In contrast, Proximal Policy Optimization (PPO), a widely adopted policy gradient algorithm utilizing a clipped surrogate objective, lacks this stabilizing property. Motivated by this observation, we propose Logits Convex Optimization (LCO), a simple yet effective policy optimization framework that aligns the learned policy with an optimal target derived from the original RL objective, thereby emulating the stabilizing effects of logits-level convexity. Extensive experiments across multiple model families show that our LCO framework consistently improves training stability and outperforms conventional RL methods on a broad range of benchmarks.

Reinforcement Learning with Verifiable Rewards (RLVR) has become an effective post-training method for improving the reasoning abilities of Large Language Models (LLMs). However, existing methods mainly apply uniform optimization constraints across all tokens, ignoring their heterogeneous roles. Prior work shows that high-entropy tokens are closely tied to reasoning, while low-entropy tokens primarily encode factual knowledge, and recent approaches attempt to exploit this distinction by isolating token updates via masking or asynchronous training. We argue that such isolation breaks the sequential dependency structure of autoregressive generation, leading to suboptimal learning. To address this, we propose \textbf{Archer}, an entropy-aware RLVR framework with \textbf{dual-token constraints} that preserves joint optimization while modulating update strength across token types. Our method introduces response-level entropy normalization for stable token classification and applies differentiated clipping ranges and KL regularization to encourage exploration on reasoning tokens while preserving knowledge tokens. Experiments on mathematical reasoning and code generation benchmarks show that Archer consistently outperforms strong baselines across multiple model scales, improving both \textit{pass@1} and \textit{pass@K} performance. These results highlight the importance of respecting sequence-level dependencies when designing fine-grained RL optimization strategies for LLMs.

Few-shot object detection (FSOD) requires detectors to adapt to novel categories from only a few labeled instances, where prototype-based transfer methods have become a strong and efficient paradigm. However, we observe that such methods remain unstable in extreme low-shot regimes. We attribute this instability to two coupled factors: semantically unanchored query initialization, which yields high-variance cold-start optimization, and deterministic prototype matching, which over-trusts noisy or occluded support regions. To address these limitations, we propose Semantic Fine-Grained Prototype Distillation (SFPD), a language-conditioned probabilistic adaptation framework for prototype-based FSOD. SFPD introduces language-conditioned feature queries to provide semantic anchors for novel-class adaptation, uncertainty-aware prototype distillation to down-weight unreliable support evidence through heteroscedastic Gaussian modeling, and component-guided part-aware prototypes to refine fine-grained semantic-visual alignment. These modules act during adaptation and preserve the original detector path at inference, introducing no additional inference FLOPs. Experiments on PASCAL VOC and MS-COCO show that SFPD consistently improves a strong FPD baseline, with especially clear gains in the most challenging low-shot settings, e.g., a 3.9-point nAP50 improvement on VOC Split 2 under 1-shot. Further ablations, multi-seed evaluation, and convergence analysis indicate that SFPD improves both accuracy and adaptation stability.

Looped Transformers --- which repeatedly apply a shared transformer block --- are an architecturally natural fit for variable-length algorithmic tasks. Although they can exhibit strong length generalization beyond the length of training sequences, this behavior is brittle, yielding high out-of-distribution (OOD) variance, even across well-performing in-distribution solutions. We trace this variance to the spurious correlation in simple algorithmic tasks between sequence length and number of loops. Introducing stochasticity into the number of loops during training sharply reduces OOD variance and stabilizes predictions across inference-time loop counts. To improve upon heuristic randomization schemes, we further analyze RL-Halting as a learned stochastic schedule and find that it generally improves the accuracy--stability trade-off. Across binary addition, Dyck-1, Unique Set, and Copy, learned stochastic stopping often improves this trade-off but can also stabilize a suboptimal computation. Our work suggests that “when to stop” should be treated as a training-time design choice, not merely an inference-time computation-allocation rule.

AI-generated image detectors often generalize poorly to unseen generators because the classifier latches onto generator-specific feature directions that are predictive only on the training set. We show that simply projecting away generator-discriminative LDA directions fails: these directions also carry genuine real-vs-fake signal, so post-hoc erasure removes evidence along with shortcuts. We introduce StaR, a training-time regularizer that scores each LDA direction by leave-one-generator-out stability and penalizes the classifier only on unstable directions; stable directions are left available for classification. On ResNet-50, StaR reduces the across-seed standard deviation of the four-OOD average AUC by 24x relative to ERM (F-significant on every OOD) while preserving in-distribution-like performance; on CLIP ViT-L/14, StaR improves the four-OOD average AUC by 1.7 points over matched CLIP-ERM and 4.5 over EFFORT, with the largest gains on platform-shift benchmarks. Mechanistic analyses show that StaR rotates the classifier away from unstable generator directions while amplifying generator-discriminative structure in the features---a rotational, not erasive, effect that post-hoc projection cannot reproduce. We additionally find that the conventional ``argmax in-domain validation'' epoch-selection rule is sub-optimal: a deployable ``stop at first in-domain 0.99'' rule beats it by sim0.7 AUC points on average.

Federated Learning (FL) is a promising paradigm for distributed machine learning. However, FL often suffers from degraded generalization performance due to the inconsistency between local and global optimization objectives and client-side overfitting. In this paper, we provide a global-update stability analysis framework as an analytical tool to study generalization error and derive the stability bounds of mainstream FL optimization algorithms under non-convex settings. Our analyses reveal how the number of global update steps, data heterogeneity, and update rules influence their stability. We observe that momentum-based FL acceleration methods do not improve stability. To address this issue, we propose FedSEMA, a new FL algorithm that couples global momentum with a gradient-difference corrected Nesterov Accelerated Gradient (NAG) scheme and a hybrid proximal term to enhance stability. This design ensures updates follow a globally consistent descent direction while retaining the benefits of acceleration. Theoretical analysis shows that FedSEMA achieves an improved stability upper bound on non-i.i.d. datasets in the non-convex settings. Extensive experiments on real-world datasets demonstrate that FedSEMA significantly outperforms multiple baseline methods under standard FL settings, achieving faster convergence and state-of-the-art performance.

Self-training is a practical way to adapt CLIP with limited supervision, but its performance is highly sensitive to pseudo-label reliability. We identify an important failure mode in CLIP adaptation: temporal prediction instability, where a target sample repeatedly flips its prediction across epochs. We provide a perspective based on cross-modal anchoring mismatch: the decision geometry induced by pretrained text anchors can deviate from the local visual cluster structure of the target domain. This mismatch can make ambiguous target samples susceptible to prediction flips during adaptation, whereas target-aligned anchoring produces more stable pseudo-labels. Motivated by this observation, we propose Stability-Aware Self-Training (SAST), centered on Stability-Weighted Image Prototypes (SWIP). SWIP builds class-wise visual anchors from temporally stable target samples to provide more reliable adaptation references. We further add a lightweight text-side refinement to reduce confusion-induced instability, and combine the two streams with an adaptive fusion rule. Across six standard benchmarks and three adaptation settings, SAST consistently outperforms strong baselines. The results suggest that explicitly modeling temporal stability is an effective route to more robust CLIP adaptation under self-training.

We study the accumulation of test errors in language models performing multi-step reasoning. Although longer reasoning improves model capabilities by scaling test-time computation, test error compounds in autoregressive generation and can grow substantially across steps. The key factors determining the stability of such reasoning remain unclear. To address this question, we analyze the test error of multi-step reasoning under supervised fine-tuning. We first present a test error bound governed by the product of *input Jacobian spectral norms* across generation steps. This product, summarized as an error amplification factor, scales exponentially with steps and controls the reasoning stability. Then, we explicitly analyze this factor in transformers trained to predict linear and quadratic function weights. We prove that the transformer converges to a solution where this amplification factor decays, yielding near-zero test loss even over longer steps. Based on the analysis, we propose a training method to suppress the error amplification factor by combining (i) chain-of-thought length compression that reduces the reasoning steps, and (ii) quantization-aware training that provably regularizes the input Jacobian norms. We validate our method by fine-tuning language models on reasoning tasks that require executing graph algorithms and tracking variable states through logical operations. Across seven tasks, our method improves over baselines by 3.5\% on average, and by 8.2\% in length generalization evaluations. We further validate that it reduces the error amplification factor in fine-tuned models.

The widespread deployment of generative AI has made it increasingly difficult to distinguish synthetic content from real data. Consequently, synthetic data is inevitably incorporated into the training pipelines of future model generations, forming a *self-consuming training loop*. Prior work has studied the effects of such recursive self-consuming training, but analyses have largely been limited to *isolated* models, where a model consumes only its own synthetic data, or to simplified interactions between two models. This paper takes a first step toward understanding *networked* self-consuming generative models, in which multiple models consume synthetic data generated by one another through complex interaction pathways. We introduce a theoretical framework representing models as nodes in a directed, weighted graph, with edge weights governing the flow of synthetic data among models. Using this framework, we analyze the long-term behavior of networked models under retraining dynamics, establishing conditions for convergence and characterizing the resulting fixed points. We further investigate how the system's long-term stability and diversity are shaped by each model’s access to real data, cross-model data consumption, and the structure of the interaction graph.

AI systems increasingly shape how individuals access and evaluate information, intensifying concerns about misinformation. Computational simulation offers a controlled complement to direct empirical study, with fine-tuned LLMs enabling powerful agent-based simulations. However, fine-tuning often induces broad behavioral shifts including bias drift, degenerate heuristics, and loss of task competence, which undermine experimental validity. The central challenge is inducing localized decision shifts without global behavioral collapse. We study this through a stability analysis of supervised fine-tuning, attenuating loss on selected training slices to shift false-claim acceptance while preserving overall task competence. We introduce FrameRef, a large-scale dataset of semantically equivalent claims with controlled surface framings across five dimensions (Authoritative, Consensus, Emotional, Sensationalist, Prestige), with human validation confirming systematic framing effects on claim acceptance. We identify stability regimes in which targeted error shifts can be induced without degrading aggregate accuracy or calibration. A sequential exposure task over FrameRef shows that small framing-conditioned shifts compound into substantially different cumulative outcomes under feedback, but largely disappear when feedback is removed, confirming that interventions modify specific decision tendencies rather than global performance. We release code, LoRA adapters, and data at https://github.com/anonymousauthor2352/frameref.

Latent action models (LAMs) aim to learn compact representations of state transitions directly from visual observations, but they suffer from a fundamental ambiguity: the latent code can encode target-state information rather than true dynamics. Existing approaches address this issue through restrictive bottlenecks, which reduce leakage at the cost of limiting expressivity. We propose StaDy, a regularization framework that instead enforces conditional informativeness: the latent code should aid prediction only when paired with the source state, while remaining uninformative about the target on its own. Concretely, we introduce a likelihood-matching objective that aligns the decoder’s predictions conditioned solely on the latent variable with its unconditional predictions. This discourages target memorization without constraining latent capacity. Experiments in video modeling show that, unlike bottleneck-based methods, StaDy scales effectively with increased latent dimensionality, reducing static content leakage while improving the representation of more complex dynamics.