NeurIPS 2026 论文列表

Submission timing in human evaluation systems is universally treated as an irrelevant detail. We identify a systematic temporal leniency bias: across 129,023 reviews spanning three consecutive years of a large-scale ML venue, evaluators submitting closer to the deadline assign higher within-paper scores and produce less thorough assessments. The pattern holds in all 15 robustness specifications across all three years, and early evaluators achieve measurably lower Brier scores ($\Delta \approx 0.005$--$0.006$, stable across years). We propose two complementary corrections. Optimal Timing-Calibrated Aggregation (OTCA) derives a linear weight function $w(t) = a^{\star} t + 1$ by minimising paper-level decision loss, significantly outperforming heuristic alternatives (McNemar $\chi^2 = 46.3$, $p < 0.001$). Adversarial Temporal Debiasing (ATD) learns timing-invariant representations via Gradient Reversal, reducing timing discriminability by $37.4\\%$ while preserving predictive utility. Combined in a joint scheme, the two methods improve borderline decision accuracy by up to $11.25\\%$ over uniform averaging (McNemar $p < 0.001$), consistently across all three years. Both methods require only submission timestamps, already logged by every major evaluation platform, making deployment cost-free.

Zeroth-order optimization (ZOO) has emerged as a promising approach for fine-tuning Large Language Models (LLMs) when gradient access is unavailable or memory-prohibitive. However, ZOO relies on estimating gradients via finite differences between two function evaluations at perturbation scale $\mu$, making it inherently noisier than its first-order counterpart. This raises a fundamental question: \textit{under what conditions does ZOO yield reliable gradient estimates?} In this work, we provide a rigorous variance-theoretic answer through a generic noise-amplification framework and demonstrate that the reliability of ZOO gradient estimations is structurally determined by whether the two finite-difference evaluations are performed on \textit{matched data}. Mismatched evaluations, as induced by on-policy Reinforcement Learning (RL), produce a $1/\mu^2$ oracle-noise amplification that sets an irreducible noise floor, whereas matched evaluations, as realized by offline optimization methods such as Boltzmann Targeted SFT (BOLT), eliminate this amplification entirely via noise cancellation. We further derive convergence guarantees with explicit constants that cleanly separate the two regimes. Controlled experiments on math reasoning tasks validate these findings, using ZooBOLT and ZooGRPO --- zeroth-order adaptations of BOLT and GRPO --- as representatives of the matched- and mismatched-data regimes respectively. On GSM8K with Qwen2.5-1.5B, ZooBOLT recovers $96.1\%$ of first-order BOLT's performance gains, while ZooGRPO exhibits near-zero learning despite its first-order counterpart achieving strong results. This qualitative separation persists across harder benchmarks (MATH), larger model scales (Qwen2.5-7B), and different model variants (DeepSeek-R1-Distill-Qwen-1.5B). Additionally, ZOO methods reduce peak memory to near-inference levels, validating its practicality as a memory-efficient fine-tuning paradigm.

Self-attention without positional information is permutation equivariant, therefore positional encodings are required whenever a transformer must distinguish input order. However, injecting absolute position can also weaken length generalization. We formalize this tension for additive positional encodings by introducing positional spread, the total variance of the positional embedding matrix. For additive PE in softmax attention with mean pooling, we prove an upper bound demonstrating that the equivariance breaking component of risk is controlled by $\|\widetilde E\|_F^2$, along with a matching lower bound for self-separable additive PEs. We finally prove a spread-expressivity tradeoff: any additive PE that distinguishes absolute positions with minimum separation $\delta$ must have spread $\Omega(\delta^2 n)$. The theory predicts two regimes: when the target is near permutation invariant, lower spread improves length extrapolation; when the task requires absolute position, additional spread reduction destroys expressivity. We test this prediction using controlled synthetic tasks, spread-regularized additive PE, and long context retrieval and reasoning tasks. Across low positional sensitivity tasks, spread strongly predicts OOD accuracy; on absolute position tasks, performance demonstrates the predicted non-monotone tradeoff. For score-based methods such as RoPE and ALiBi, we include a displacement based comparison rather than matching bounds, clarifying both the reach and limits of the spread theory.

Frontier AI systems function as \emph{constitutional institutions}: each deployed model encodes an implicit ranking among safety, helpfulness, honesty, autonomy, and equity. We ask whether the supply of frontier constitutional types covers human demand. Combining a paraphrase-controlled audit of $23$ frontier LLM archetypes with a pairwise-tradeoff study of $1{,}649$ humans on the same instrument, we report three facts. Demand is broad: it spans all five values, with the largest constituency under one-third. Supply is narrow and drifting: the $23$-archetype hull occupies $0.10\%$ of the demand hull, and across six model families autonomy decreases in $5/6$, equity increases in $5/6$, and safety increases in $4/6$. The importance of this drift is not that the frontier moves but that it moves away from a value that is already undercovered, mechanically worsening the welfare floor for the users least well served by the static menu. The fix is sparse: a $2$-vertex menu $\{e_{\mathrm{HON}}, e_{\mathrm{AUT}}\}$ beats the full $23$-archetype frontier by $47$ percent on mean regret ($95$ percent); three vertex additions cut mean regret by up to $81$ percent and worst-group regret by up to $64$ percent. We formalize these findings as a budgeted-pluralism trilemma and show the binding regime is empirically realized.

Memory is becoming a core component of long-horizon AI agents, allowing agents to reuse past experience when operating web browsers, software tools, and other interactive environments. Existing work mostly treats memory as a supply problem, asking what experience to write, how to store it, and which entry to retrieve for the next task. Yet we still lack a clear account of how models consume retrieved memory across a multi-step action trajectory. This consumption process matters because it determines not only what memories should be retrieved, but also what models and control policies are needed to use them safely. To diagnose this process, we propose Entry--Propagation--Recovery (E-P-R), a trajectory-level framework that asks where memory first changes an action, whether that change carries forward, and whether the agent can recover after leaving a correct path. We instantiate E-P-R on WebArena and on MemTrapBench, a controlled benchmark we build to isolate these phases. We find that the main failure often begins at entry: agents adopt conflicting memory at the first exposed decision point even when the recommendation is task-wrong. Repeated exposure then amplifies this early error, while recovery after divergence is weak. Together, these effects create a compliance trap: across models, conflicting memory induces similar compliance rates, but once agents comply, their success rates collapse to a low floor. Stronger agents therefore suffer larger absolute damage because each compliance event erases more baseline capability. These results suggest that memory-augmented agents should be evaluated not only by retrieval quality or final success rate, but by how they consume memory throughout the trajectory.

When language models reason in chain-of-thought or collaborate in pipelines, they serialize structured information into natural language. How much compositional structure survives this bottleneck? We propose a round-trip protocol that answers this question empirically for tree-structured symbolic expressions. A generator converts a procedurally generated arithmetic expression into a word problem, a separate extractor recovers the expression from the word problem alone, and symbolic equivalence provides an exact oracle. Evaluating all pairwise combinations of fifteen models yields a communication matrix whose marginals separate generation quality from extraction quality. Three main findings emerge. First, the channel is lossy and asymmetric: swapping which model generates and which extracts shifts accuracy by up to 53.2 points, and the best pair reaches 82.0\% by combining different models on each end rather than the same model on both. Second, at least 84\% of round-trip failures originate at generation, and difficulty is driven by tree structure (operator count, depth, right-branching) rather than model family. Third, the channel is trainable: $\sim3600$ fine-tuning examples that share the evaluation's operators and tree shapes lift every open-weight model above untrained GPT-5, an upper bound under matched semantics. A disjoint-domain regime with new operators and vocabulary also raises every open-weight model, confirming the gain is not an artifact of matched semantics, though a gap to the frontier remains. Together these results identify compositional serialization as a primary limiting factor when models communicate hierarchical structure through natural language.

Group Relative Policy Optimization (GRPO) is widely used to post-train reasoning language models, and its rollout group size $G$ is typically treated as a compute hyperparameter. We argue that $G$ also sets a difficulty threshold below which prompts produce no learning signal. Under binary verifier rewards, a rollout group only produces a learning signal when it contains both a success and a failure, and the rate at which a prompt yields such mixed groups turns on sharply near $p_c\simeq\ln 2/G$. Below that threshold the prompt is invisible to the relative-advantage estimator regardless of optimizer state. We verify this gate directly in pass@256 rollout traces from Qwen3-1.7B, Qwen3-8B, and Llama-3.1-8B, where empirical mixed-group rates match the binomial prediction within $1.4$–$1.9$ percentage points on average per model. Sweeping fixed-$G$ GRPO with $G\in\{8,16,32,64,128\}$ then shows that difficulty-binned per-prompt improvement after training tracks the frontier shape with binned $R^2$ between $0.962$ and $0.984$ across all three model families. Treating $p_c$ as a moving target, we instantiate a frontier-tracking group-size schedule that improves the best fixed-$G$ baseline by $4.2$–$5.3$ points on MATH-500 and $2.4$–$3.4$ points on GSM8K under matched rollout budget. Ablations attribute the gain primarily to frontier-aware switch timing.

Memory-augmented reasoning agents typically store a single realized reasoning trajectory as the unit of experience. This abstraction is brittle under stochastic decoding: the same task can induce multiple plausible reasoning paths with different decompositions, assumptions, and failure modes. We introduce Branch-Structured Distributional Memory (BranchDIME), a framework for constructing memory from branch-structured trajectory sets rather than isolated traces. BranchDIME samples short reasoning prefixes, clusters them into prefix-induced branches, expands representative prefixes into full trajectories, and distills the resulting set into reusable memories containing positive strategies, anti-patterns, and contrastive rules. Across GPQA-DIAMOND and MATH500, BranchDIME improves over single-trajectory memory and alternative trajectory-selection baselines at comparable inference latency, yielding relative gains of 8.7% and 2.2% over single-trajectory memory, respectively. Coverage analysis shows that BranchDIME spans all discovered branches, compared with 20.0% coverage for consensus selection and 64.9% for random selection, while also achieving the highest downstream accuracy. BranchDIME further transfers across tasks and models, improving over single-trajectory memory by up to 15.4% on AIME25 and OlymMATH. These results support a shift from trajectory as experience to trajectory distribution as experience: memory should capture the local reasoning landscape induced by stochastic generation, not merely one sampled path.

Certifying authorities for safety-critical software audit recovered trace matrices module by module, not on average. Every existing distribution-free calibration delivers a marginal certificate, and we show the gap to the auditor's quantity is structural and unbounded. Under within-block exchangeability with cross-block heterogeneity in the conditional null score distribution, every procedure controlling marginal $\mathrm{FDR}$ at level $\alpha$ on a bipartite trace matrix with $K$ source modules admits worst-module false discovery proportion at least $1{-}\alpha{-}O(1/K)$ with probability at least $1{-}2/K$. We name this the block catastrophe and close it with $\mathrm{CERTRA}$, which wraps any black-box scorer, computes a within-block rank-indicator conformal $e$-value, and applies $e$-$\mathrm{BH}$ separately within each source module. The construction yields finite-sample per-module $\mathrm{FDR}$ control under arbitrary intra-module dependence, with matching power up to a rank-quantization factor. We release IndTrace-2026, a 12{,}406-link composite across aerospace, automotive, and medical-device specifications with professional ASIL/DAL/SIL annotation (Cohen's $\kappa{=}0.86$). $\mathrm{CERTRA}$ holds worst-block $\mathrm{FDP}$ at 0.118 where split-conformal drifts to 0.413, retains 0.741 of true links versus 0.572 for the strongest fair-operating-point baseline, and reduces audit deficiency 3.3-fold. Per-module distribution-free calibration is what an industrial audit can sign.

We study how temporal correlations in the data can make certain sparse learning problems efficiently learnable by gradient-based methods. Our focus is on Boolean k-juntas, a canonical sparse learning problem known to pose barriers for gradient-based methods under independent uniform samples. We show that this picture changes when the samples are generated by a lazy random walk on the hypercube. In this setting, the temporal dependencies can be exploited by a two-layer ReLU network trained using stylized-SGD with a temporal-difference loss, which compares target and predicted increments across consecutive samples. For every fixed k, the resulting sample complexity is essentially linear in the ambient dimension d. By contrast, we show that for large-batch gradient methods using standard convex pointwise losses, temporal correlations do not provide the same advantage.

Humans systematically misrepresent probability in a stereotyped inverse-S pattern. It has been documented for decades, but its origin remains unexplained. We propose a Bayesian encoding–decoding account in which probabilities are represented by noisy internal signals and decoded by Bayes-risk minimization. For bounded probability stimuli, we show that distortion decomposes into boundary regression, likelihood repulsion, and prior attraction, yielding a key prediction: the classic inverse-S-shaped weighting pattern implies a U-shaped allocation of encoding precision with greater sensitivity near 0 and 1. Across judgment of relative frequency, lottery pricing, and risky choice, this U-shape is recovered from data without imposing any functional form on the encoding, and our framework outperforms deterministic weighting functions, bounded log-odds models, uniform-encoding Bayesian accounts, and matched efficient-coding models on held-out data. In a new dot probability estimation experiment with bimodal stimulus statistics, the recovered prior tracks the new distribution while the recovered encoding remains U-shaped. Together, these results identify the inverse-S-shaped probability weighting function as the joint product of a stable U-shaped encoding and a flexible prior, integrated by optimal Bayesian decoding.

The Bayesian learning rule (BLR) exploits the relation between Bayesian updating and the Kullback-Leibler divergence (KLD) to derive efficient additive updates for exponential-family variational posteriors. The KL geometry is powerful but restrictive: combined with posterior families with parameter-dependent support it can lead to undefined objectives. We ask which non-KL geometries preserve the additive BLR computation. Previous works that use $f$-divergences suggest that this is only possible with KLD. Here, we show that this is not true. We show that additive information-geometric optimization algorithms can be derived for the class of mixed-space Bregman or Fenchel-Young discrepancies. These discrepancies induce generalized maximum entropy families, dual coordinates, and generalized BLR canonical gradient updates, recovering the classical BLR. The generalized BLR leads to new algorithms that \textit{cannot} be derived from the KL-based BLR. For the Tsallis negentropy with quadratic statistics, the generalized BLR yields bounded-support and heavy-tailed $q$-Gaussian posteriors and an easy Newton-like implementation. Experiments up to CIFAR-100 illustrate controllable support and tail behavior and competitive performance.

Approximate machine unlearning aims to remove the influence of specific training data from a trained model without retraining from scratch. We identify a previously undocumented confound in how unlearning is evaluated on BatchNorm-based architectures: a single forward pass over retain data, an operation that modifies no weight, can deterministically rewrite the model's normalization state and reverse the apparent surface-metric forgetting. We formalize this operation as a weight-preserving fixed-point operator and prove that any pre-versus-post gap it induces is provably attributable to BN running statistics rather than to any modification the unlearning method made to the weights. This attribution claim cleanly separates measurement failure (BN artifact) from encoder failure (residual weight-encoded information, recently documented in concurrent work), and the same operator framework yields a unique decomposition of linear-probe elevation into BN-measurement-bias and encoder-geometry components. Empirically, the artifact reverses headline forget accuracy by up to $78$pp across nine published methods on standard benchmarks; an attacker with as few as $10$ unlabeled images recovers most of the masked accuracy; and a strict GroupNorm control reduces the artifact to zero across all methods.

This paper introduces the BV4 Benchmark, a collection of high-dimensional spectral datasets for unsupervised anomaly detection in vacuum environment data streams. Collected via Optical Emission Spectroscopy (OES), the benchmark comprises nine experiments representing practical scenarios structured according to established anomaly detection literature and validated by domain experts to address challenges such as spatial anomalies, temporal anomalies, and concept drift. BV4 datasets are composed of 2048 distinct wavelengths recorded over time, accompanied by timestamps and ground truth labels, enabling rigorous evaluation of machine learning models under dynamic environmental changes within vacuum chambers. We demonstrate the utility of BV4 through a simple comparative evaluation against previous datasets for anomaly detection in spectral data streams and discuss the current limitations and future improvements of the benchmark to support meaningful evaluative claims in the area.

Large-scale wildlife monitoring in forested environments using drones remains challenging due to occlusion, limited visibility, and scarce annotated data. We present a comprehensive airborne wildlife dataset comprising 386 paired RGB and thermal aerial video sequences recorded across diverse temperate forests and forest-adjacent habitats in Austria. Each frame is geo-referenced with precise global coordinates (longitude, latitude and altitude), enabling learning and evaluation in both image space and geographic space. The dataset includes 10 animal species captured under varying seasonal, environmental, and illumination conditions, with annotations supporting tasks such as object detection, multi-object tracking and fine-grained classification. The multispectral data and spatial metadata enable research on world-coordinate trajectory analysis, spatial population modeling, geo-aware perception, and advanced methods such as Airborne Light Field Sampling. Thermal subsets of this dataset have been used to develop and validate methods for wildlife monitoring, while the RGB data has mainly stayed untouched. To address this, we present a cross-modal pipeline to transfer thermal annotations to RGB frames next to the dataset. With this dataset we aim to provide a blueprint to promote research on multimodal, geo-referenced perception in ecology.

Despite the success of Neural Operators (NOs), architectural hyperparameters such as depth are still typically selected by empirical grid search, with limited connection to the governing PDE. We introduce the \textit{alternation depth}, $\alpha_F$, a structural quantity of the PDE right-hand side that counts the minimum number of non-pointwise/pointwise coupling stages needed to build its leading nonlinear spatial structure. We use $L_{\min} = \alpha_F + 1$ as a principled \emph{depth-efficiency lower-bound guideline} for IVP solution maps: a starting depth, not a predictor of the empirical optimum $L^*$. We provide partial formal support: (*sufficiency*) an $L$-block architecture can approximate any operator in a formally defined $L$-alternation class under compactness and regularity assumptions; (*separation*) for a constructed hard family at finite resolution and polynomial activations, using fewer blocks forces width to grow as a power of the discretization size. Empirically, a 736-run study over nine in-scope IVP datasets plus one Darcy stress test, using FNO-family variants and one kernel-integral baseline (KIN), is consistent with depth improvements appearing at or above $L_{\min}$ in the tested settings. The observed optimum can be larger because of spectral approximation, pointwise approximation difficulty, data regime, and optimization. We summarize the resulting practitioner guidance in four capacity-allocation rules.

Prior work localizes the alignment tax---instruction tuning's degradation of structured generation---to transformer layers, but cannot determine which projections within a layer carry the functional change. We identify a consistent sub-layer hierarchy across three primary SwiGLU decoder-only families (7--9B parameters, with Yi-1.5-9B supplementary): among equal-parameter MLP projections, $W_{down} > W_{up} > W_{gate}$ (2.5:1.5:1)---an architecturally grounded ordering (reproduced under continual pretraining) not resolved by concurrent work. Within attention, $W_V$/$W_O$ gradient magnitudes exceed $W_Q$/$W_K$ by 1.76--3.91$\times$. Three independent methods---weight patching, gradient analysis, and training-time exclusion---converge on the same within-layer structure; a continual-pretraining control confirms the within-MLP hierarchy is architectural (not alignment-specific), but the layer-level distribution of harmful modifications is---this task-specific signal enables targeted rollback (8.9$\times$ over architectural-prior heuristics on held-out data, $p=0.002$). Surgical Alignment Reversal (SAR), a training-free rollback of attribution-ranked components, recovers 47% of the alignment tax (cross-entropy metric, Qwen; Llama/Mistral: 2--3$\times$ random) on held-out cross-dataset data (in-domain ratios 2.8--5.3$\times$); Output-Constrained DPO (OC-DPO) largely eliminates the tax while Q/K exclusion does not---both validated on attribution-independent data. Safety evaluation shows no significant degradation: refusal-rate $|\Delta| \le 1.4$ pp ($n=1{,}000$), no significant ASR increase under adversarial attack ($|\Delta\text{ASR}| \le 6$ pp, $n=50$); Yi-1.5-9B is excluded due to safety--structure overlap ($-13.6$ pp). Attribution replicates at 14B; gradient asymmetry persists at 72B.

Layer-wise visual-text similarity in Multimodal Large Language Models (MLLMs) is widely interpreted as evidence that the language model progressively integrates visual content into a shared representation space. This reading rests on the assumption that scalar alignment scores reflect content-level cross-modal interaction. To test this assumption, we apply controlled interventions to the visual stream. Across 13 MLLMs from five families spanning 0.5B to 72B parameters, replacing projector-output visual tokens with Gaussian noise sharply reduces task accuracy, yet four standard scalar measures (CKA, SVCCA, MIR, and the leading principal-angle cosine) fail to consistently separate the corrupted stream from the original. We call this failure the \emph{alignment illusion} and trace it to the shared language-model pathway: anisotropic MLP down-projections pull visual and text tokens toward common output directions, producing \emph{weight-induced alignment}. Because this component is essentially one-dimensional, we introduce the principal-angle gap (PA gap), defined as the difference between the top two principal-angle cosines, which separates weight-induced similarity from multi-directional visual structure. Under graded visual corruption, the PA gap tracks task accuracy more consistently than the scalar scores we consider; under a structured but irrelevant image, it further exposes regimes in which internal geometry and task accuracy come apart. Internal visual-text alignment in MLLMs is therefore best read as a geometric diagnostic of the visual stream inside the language model rather than a direct proxy for content-level cross-modal interaction, and is most informative when calibrated by controlled task evidence.

Scientific discovery is constrained not only by what is true, but by what is cognitively available to the researchers currently exploring a field. Many directions are coherent in light of the literature yet unlikely to be proposed because no existing community occupies the right combination of concepts, methods, and intuitions. Modern language models inherit this bias, recombining high-density regions of the literature when prompted for novel ideas. We introduce a framework that targets the complementary region, which we call the \textbf{alien space of science}, where directions are plausible under the structure of existing knowledge but unlikely under the distribution of existing researchers. Our method first decomposes papers into granular conceptual units and clusters them into a shared vocabulary of \emph{idea atoms}. It then learns two complementary models over this vocabulary. A \emph{coherence model} scores whether a combination of atoms forms a viable research direction, and an \emph{availability model} scores whether any existing author community is positioned to produce a given combination. Sampling alien directions then reduces to ranking atom combinations that maximize coherence while minimizing availability. On a corpus of 16{,}068 peer-reviewed LLM papers from NeurIPS, ICLR, ICML, and major NLP venues, the resulting sampler explores a $3.5\text{--}7\times$ broader effective atom vocabulary than frontier LLM ideation baselines without sacrificing coherence, and produces ideas that match or exceed those baselines under blind LLM, human, and downstream experimental evaluation. By separating scientific plausibility from community availability, our framework points toward AI ideation that complements rather than merely accelerates human science, expanding exploration into coherent directions the current community is unlikely to pursue.