NeurIPS 2026 Papers

Mathematical reasoning benchmarks for large language models face three persistent limitations. First, datasets scraped from public competitions are vulnerable to training-data contamination and saturate quickly. Second, most existing benchmarks cover only part of the olympiad-to-research spectrum. Third, benchmarks that withhold problems indefinitely to mitigate leakage trade away transparency and reproducibility. We introduce \gls{soohak}, a benchmark of 1{,}141 newly authored problems by 86 mathematicians designed to address all three. \gls{soohak} consists of three difficulty splits, constructed adversarially against panels of small, mid-size, and frontier baseline LLMs, spanning olympiad-style problem solving through research-adjacent material; a separate 99-item Refusal split probes recognition of ill-posed prompts. The dataset is temporarily embargoed and evaluated by request, with a committed public release in late 2026. On \splitthree{}, Gemini-3-Pro, GPT-5, and Claude-Opus-4.5 reach Avg@3 of 30.39\%, 26.37\%, and 10.39\%, respectively; the best Pass@3 is only 44.12\%, and 129/340 problems remain unsolved by any closed model evaluated, indicating substantial room for improvement before LMs can meaningfully assist in mathematical research. By contrast, the \calibrationsplits{} are nearly saturated by both closed and open-weight systems. Notably, while GPT-OSS-120B, Kimi-2.5, and GLM-5 each reach Pass@3 between 87.4\% and 88.3\% on \splitone{}, within 6 percentage points of the leading closed model (Gemini-3-Pro, 93.85\%), they trail by over 20 percentage points on \splitthree{}. This pattern suggests open-weight pipelines are well targeted to olympiad-style benchmarks but transfer poorly to research-adjacent material. Finally, through a timed human study with 25 participants of diverse mathematical expertise, we confirm that the difficulty is genuinely mathematical rather than artifactual, with aggregated teams covering 50.6\% of a 79-problem sample.

Graph Neural Networks (GNNs), as representative deep models designed for graph data, effectively learn graph topological information and push the performance boundaries across various graph tasks. However, the computational and memory burden of GNNs poses significant challenges for scaling to large real-world graphs. Spiking Graph Neural Networks (SGNNs), which integrate biologically plausible learning via unique spike-based neurons, have emerged as a promising energy-efficient alternative. Different layers communicate with sparse and binary spikes, which facilitates computation and storage of intermediate graph representations. Despite the proliferation of SGNNs proposed in recent years, there is no systematic benchmark to explore the basic design principles of these brain-inspired networks on the graph data. To bridge this gap, we present **SGNNBench** to quantify progress in the field of SGNNs. Specifically, we elaborately investigate the design space of SGNNs to facilitate the development of a general SGNN paradigm. Regarding efficiency, we empirically compare these baselines w.r.t. model size, memory usage and theoretical energy consumption to reveal previously overlooked energy bottlenecks. Furthermore, SGNNBench establishes a unified and reliable benchmark to comprehensively evaluate 9 state-of-the-art SGNNs across 20 datasets covering diverse graph settings and tasks.

Large language model (LLM)-driven evolutionary frameworks have emerged as a promising paradigm for iterative search in complex optimization and structured generation tasks. However, existing methods typically rely on heuristic variation operators and lack an explicit mechanism to extract directional information from observed performance feedback, limiting both efficiency and reliability. To address this, we propose SGEvolve, a semantic gradient-guided evolutionary framework for LLM-driven optimization. The key idea is to treat the performance differences between parent and offspring as zeroth-order evidence of an underlying ascent direction. Based on this view, we formulate semantic gradient extraction as a Maximum A Posteriori (MAP) estimation problem, where the LLM infers the most probable direction of improvement conditioned on observed evolutionary steps. The inferred gradients are then used to guide mutation and crossover, biasing generation toward more promising regions of the search space. We evaluate SGEvolve on a diverse set of tasks, including mathematical optimization, algorithm design, and real-world industrial problems. Experimental results demonstrate that SGEvolve consistently outperforms existing open-source baselines, achieving superior performance in both solution quality and search efficiency.

Edge of Stability (EoS) refers to the phenomenon where full-batch gradient descent (GD) training of neural networks with step size $\eta$ pushes the largest eigenvalue of the Hessian, i.e. the sharpness, to $2/\eta$ and hovers there. \citet{damian2023selfstab} explains the hovering behavior of the sharpness by *self-stabilization*, a mechanism driven by third-order structure of the loss, and shows that GD implicitly follows projected gradient descent (PGD) on the set where the sharpness is below $2/\eta$. For mini-batch stochastic gradient descent (SGD), the sharpness stabilizes *below* $2/\eta$, with the gap widening as the batch size decreases. However, no theoretical explanation exists for this suppression. In this paper, we introduce *stochastic self-stabilization* that extends the self-stabilization framework to SGD. Our key insight is that gradient noise injects variance into the oscillatory dynamics along the top Hessian eigenvector, strengthening the sharpness-reducing force and shifting the equilibrium below $2/\eta$. Following the approach of \citet{damian2023selfstab}, we define *stochastic predicted dynamics* that tracks the deviation of SGD from the PGD trajectory, and prove a stochastic coupling theorem that relates the SGD sharpness and loss to those of PGD. Based on our predicted dynamic, we derive a closed-form equilibrium sharpness gap that scales with the variance of the gradient noise projected onto the top eigenvector of the Hessian. This formula predicts that smaller batch sizes yield flatter solutions, and recovers GD when the batch equals the full dataset.

Neural networks are known to be susceptible to over-reliance on spurious correlations. However, the precise mechanism by which models exploit shortcut features is not fully understood, and algorithms to mitigate this behavior rely on as yet unjustified assumptions about the learned representations. In this work, we provide the first end-to-end theoretical characterization of spurious feature learning for two-layer ReLU neural networks trained by online minibatch SGD on the logistic loss. We consider data drawn from the high-dimensional Boolean hypercube with a quadratic signal function (namely XOR) and a linear spurious correlation. We show that SGD learns the spurious feature first, and exponentially fast. Moreover, the optimization dynamics couple the spurious and signal features, with a stronger spurious component inhibiting signal feature learning. Our analysis reveals precise phase transitions in the learning dynamics. In the first phase, alignment between the signs of the spurious feature and second-layer weight drives rapid growth of the spurious feature. When the spurious correlation is maximally strong, we show theoretically that the spurious feature dominates even at the sample complexity threshold where XOR would be learned in isolation (i.e., if the spurious feature was absent). In contrast, when the correlation strength is constant, we provide preliminary empirical evidence that the model can eventually learn the XOR signal, although the spurious feature is not forgotten.

Recent mixed-policy optimization methods for LLM reasoning that interleave or blend supervised and reinforcement learning signals report improvements over the standard SFT-then-RL pipeline. We show that numerous recently published research papers rely on a faulty baseline caused by two distinct bugs: a CPU-offloaded optimizer bug in DeepSpeed that silently drops intermediate micro-batches during gradient accumulation (affecting multiple downstream frameworks including TRL, OpenRLHF and Llama-Factory), and a loss aggregation bug in OpenRLHF that incorrectly weights per-mini-batch losses. Together they suppress SFT performance, with the optimizer bug accounting for most of the gap and the loss aggregation bug contributing a smaller additional effect. Once corrected, the standard SFT-then-RL pipeline surpasses every published mixed-policy method we evaluate by +3.8 points on math benchmarks with Qwen2.5-Math-7B and by +22.2 points with Llama-3.1-8B. Even a truncated variant with just 50 RL steps outperforms mixed-policy methods on math benchmarks while using fewer FLOPs.

Cognitive neuroscience research indicates that human spatial navigation relies not merely on abstract global map representations, but rather on anchoring local observations to spatially stable, decision-relevant landmarks to effectively align the current scene with the cognitive map. Inspired by this mechanism, we propose SFPR, a structural fingerprint-based LiDAR-to-OpenStreetMap (OSM) place recognition framework. This framework goes beyond the traditional coarse retrieval paradigm that relies exclusively on global descriptor similarity and further introduces fine-grained discrimination based on structural fingerprints to achieve more accurate place recognition. Specifically, we propose a structural fingerprint-aware attention mechanism that utilizes sparse but spatially stable anchors as saliency prompts, directing the network's attention toward potential fingerprint regions while dynamically generating global descriptors. Subsequently, we formulate the extraction and matching of structural fingerprints as an alternating optimization problem guided by this fingerprint-aware attention. Through iterative optimization, we extract structural fingerprints with high fingerprint-aware attention and geometric consistency from anchors. These structural fingerprints are further fed back into the training stage as geometric priors, effectively suppressing false matches characterized by similar global features but contradictory local structures. Experiments demonstrate that SFPR effectively overcomes the retrieval bottleneck in macroscopically homogeneous scenes, achieving a relative improvement of 30.20\% in Top-1 Recall@1m compared to the current state-of-the-art method. Code are publicly available at https://anonymous.4open.science/r/SFPR.

Vision-Language Models (VLMs) show great potential for Industrial Anomaly Detection (IAD) but often overlook subtle high-frequency defects due to their semantic bias. Additionally, one-class adaptation risks rank collapse and catastrophic forgetting. To address these issues, we propose a unified framework termed SF-DST. First, we design Spatial-Frequency Dual-Stream Transformer, which explicitly captures lost details via a parallel discrete cosine transform branch. Crucially, it incorporates Asymmetric Cross-Modal Modulation (ACMM), a mechanism that leverages spatial semantics as a context gate to selectively retrieve and inject spectral features, effectively compensating for the texture blindness of VLMs. Second, Anomaly-Aware LoRA (A-LoRA) enforces Orthogonal Regularization on the adaptation subspace. This geometric constraint ensures subspace diversity to prevent mode collapse, while optimizing a hypersphere-based metric for precise anomaly discrimination. Extensive experiments on MVTec-AD, VisA and MMAD benchmarks demonstrate that our method significantly outperforms state-of-the-art approaches, successfully bridging the gap between large-model generalization and industrial-grade precision.

Large language models (LLMs) are rapidly shifting toward agents that solve tasks through diverse interfaces, including web and graphical user interfaces (GUIs). Among these, the terminal command line provides a text-based, general-purpose interface, covering tasks from system operations to data science and machine learning. However, scaling terminal-agent training remains challenging, as it requires diverse and coherent task instructions, executable environments, and reliable verification, while lacking naturally grounded supervision data. In this work, we propose SETA, a scalable framework for generating verifiable terminal environments for reinforcement learning (RL). The framework consists of two pipelines sharing a unified verification mechanism: SETA-Synth converts diverse sources into standardized RL environments, and SETA-Evol further expands from existing environments with adaptive control of difficulty and diversity. Together, we construct and release SETA-Env, the largest open-source verifiable terminal RL dataset to date, containing over $4{,}500$ environments. We evaluate our dataset by training a Qwen3-8B model with GRPO on SETA-Env, achieving $12$\% pass rate on Terminal-Bench 2.0, the best reported result for an RL-trained model at the 8B scale. The results demonstrate that SETA-Env provides high-quality training environments for terminal agents and serves as a valuable resource for advancing research on terminal-based agent learning.

Reconstructing speech from non-invasive brain signals offers a promising pathway for restoring communication in individuals who are cognitively intact but unable to speak. Existing EEG-to-speech approaches formulate this task as \emph{acoustic reconstruction}, optimizing waveform fidelity while ignoring whether the generated speech preserves high-level semantic content. In this work, we revisit this formulation and argue that EEG signals carry not only acoustic but also semantic information. We identify two key limitations of prior methods: (1) the neglect of spatial relationships between EEG electrodes, and (2) the failure to exploit the semantic structure of the N400 paradigm, where congruent and incongruent trials reflect distinct semantic processing. We propose \textbf{SENSE} (\textbf{S}emantic-\textbf{E}EG \textbf{N}eural \textbf{S}peech Synth\textbf{E}sis), which combines a graph-based EEG encoder over electrode geometry with EEG Semantic Conditioning (ESC), aligning EEG to a pretrained semantic space using only congruent trials. On the N400 dataset, SENSE consistently outperforms prior methods on both acoustic and semantic metrics, and model-internal channel attribution suggests distributed reliance on auditory, sensorimotor, and centro-parietal regions, consistent with known speech-perception neuroscience. In the unseen-subject setting, SENSE trained on only two subjects already surpasses the strongest baseline trained on all eighteen subjects in word error rate, and matches it on acoustic metrics with as few as eight subjects.

Chunking is a hidden bottleneck in dense retrieval and retrieval-augmented generation: it determines the units that can be indexed, retrieved, and ultimately used as evidence. Yet most systems still segment documents using fixed token windows or globally thresholded semantic similarity drops, treating chunking as an engineering heuristic rather than a statistical decision. We propose **SEISMOS**, a statistical signal detection framework for semantic chunking. **SEISMOS** models consecutive sentence-embedding cosine similarities as a document-level signal and derives boundary decisions from the null hypothesis of no semantic transition. Across four development BEIR corpora, we establish three corpus-invariant properties: bimodal document structure, rapid survival decay of shallow local minima, and positive lag-1 autocorrelation. These properties show that semantic boundaries should be detected by variance-normalized deviations, not by document means or global similarity thresholds. Accounting for autocorrelation leads to a normalized discrete Laplacian detector that identifies significant semantic valleys through a single interpretable decision rule. Evaluated on five BEIR benchmarks, **SEISMOS** consistently improves over fixed-length, recursive, and production semantic chunking baselines under one fixed operating configuration. Without corpus-specific retuning, the same detector transfers to a held-out TREC-COVID corpus. Our results suggest that the boundaries needed for effective retrieval are already encoded in embedding-similarity signals, and that principled, efficient, LLM-free chunking can be obtained by detecting them statistically.

Diffusion transformers (DiTs) have emerged as a dominant architecture for text-to-image generation, yet their performance drops when generating at resolutions beyond their training range. Existing training-free approaches mitigate this by modifying inference-time attention behavior, often through Rotary Position Embeddings (RoPE) extrapolation combined with attention scaling. However, these strategies apply a uniform and content-agnostic scaling across RoPE components with distinct frequency characteristics, inducing a trade-off between preserving global structure and recovering fine detail. We introduce \textbf{SEGA}, a training-free method that dynamically scales attention across RoPE components according to the latent’s spatial-frequency structure at each denoising step. This adaptive scaling improves both structural coherence and fine-detail fidelity. Experiments show that SEGA consistently improves high-resolution synthesis across multiple target resolutions, outperforming state-of-the-art training-free baselines.

Most video anomaly understanding (VAU) systems still infer anomaly judgments from evidence collected in a feed-forward way under predefined rules, typically sampled frames, or events clustered from such frames. This protocol fails when the collected evidence is incomplete under imperfect predefined rules, because once reasoning begins the model can no longer search for the missing evidence. We address this limitation with SEEK-VAU, which reframes VAU as an agentic search-and-verify interaction protocol under a bounded search budget: instead of passively reasoning over pre-collected evidence, the policy actively gathers evidence across the three stages of an anomaly event chain (precursor, trigger, and aftermath) and verifies its sufficiency before finalization, turning event-chain completeness into an explicit design rather than an outcome of predefined rules. Although search-and-verify spans multiple interaction turns, the bounded search budget caps both the per-turn input tokens and the cumulative visual context, keeping inference efficient. To make this behavior learnable, we introduce evidence-faithful counterfactual verification (EFCV), which rewards selected evidence that remains supportive, compact, and necessary under counterfactual verification. We further introduce SEEK-Bench, featuring video-level episodes with temporal interval annotations, semantic QA and event-chain stage labels. Together, SEEK-VAU and SEEK-Bench establish a strong foundation for evidence-faithful and actively verifiable VAU. The code and data will be released.

Action chunking improves exploration and accelerates value propagation in long-horizon reinforcement learning, but naively applying off-policy methods to the temporally extended action space at reduced decision frequency offsets these gains, leading to poor sample efficiency. Existing action chunking methods address these issues through computationally expensive critic-only approaches or by relying on offline data. We introduce SEAR, a sample-efficient off-policy algorithm that enables online reinforcement learning with action chunks by addressing both challenges. To handle high-dimensional action sequences, SEAR employs a causal transformer critic trained with multi-horizon targets that provide a training signal for every prefix of an action chunk, effectively increasing the useful gradients per sample. To restore decision frequency, SEAR operates with a receding horizon and random replanning, ensuring uniform state coverage while combining the fast value propagation of large chunks with the reactivity of small ones. SEAR outperforms state-of-the-art online reinforcement learning methods including SimbaV2 on challenging Metaworld manipulation tasks. Beyond online RL, applying SEAR to the offline-to-online method QC improves its performance on the demanding OGBench cube-triple tasks.

Low-Rank Adaptation (LoRA) enables efficient adaptation of large pre-trained models to downstream tasks by parameterizing weight updates with low-rank matrices. In this paper, we investigate the limitations of the LoRA parameterization from a geometric perspective. Specifically, we show that when a full fine-tuning gradient is backpropagated to the low-rank matrices, it undergoes anisotropic scaling driven by their singular values. We argue that this phenomenon is undesirable because it distorts the full fine-tuning gradient by skewing it toward dominant singular directions while suppressing others. Our analyses demonstrate that anisotropic gradient scaling reduces the effective rank of the low-rank matrices' gradients and results in suboptimal alignment between the full fine-tuning gradient and its low-rank approximation in LoRA, thereby exacerbating the gap to full fine-tuning. To address these limitations, we propose a new low-rank parameterization, SDS-LoRA, which structurally decouples singular values from the backward pass. Our method ensures that the full fine-tuning gradient backpropagates only through the orthonormal bases of the low-rank matrices' subspaces, independent of their scales. Convergence analysis demonstrates that while LoRA’s convergence rate degrades with the condition number of the low-rank matrices, SDS-LoRA remains independent of it. Experimental results across natural language and vision benchmarks show that SDS-LoRA improves loss convergence and reduces the gap to full fine-tuning, significantly enhancing adaptation performance.

Conventional real-valued neural networks struggle to explicitly capture phase-frequency coupling, while long-horizon time series forecasting requires models to characterize non-stationary amplitude variations, phase shifts, and frequency-dependent temporal evolution. This mismatch limits the stability and expressiveness of neural models on complex forecasting tasks. To address this challenge, we propose SDHilb, a structure-preserving forecasting framework that lifts real-valued observations into an adaptive complex state space and models temporal evolution as a process jointly governed by amplitude and phase information. Through the Hilbert transform, phase-related structures are explicitly encoded, leading to a coupled evolution of real and quadrature components that motivates Schrödinger-Hamiltonian structured dynamics in the lifted complex space. Based on this formulation, SDHilb integrates Schrödinger-guided linear dynamics with an adaptive multi-scale Hilbert transform and introduces three key components: (1) a structure-preserving linear dynamic module with symplectic discretization, (2) an adaptive multi-scale Hilbert module for constructing data-dependent quadrature components and refined time-frequency decomposition, and (3) a theoretically grounded parameter reuse mechanism derived from the closed-form recurrence structure. Extensive experiments demonstrate that SDHilb achieves competitive or superior performance on both short- and long-term forecasting tasks while reducing horizon-specific parameterization through recurrence-guided parameter reuse.

Vector quantization (VQ) with autoregressive (AR) token modeling is a widely adopted and highly competitive paradigm for time-series generation. However, such models are fundamentally limited by exposure bias: during inference, errors can accumulate across sequential predictions, leading to pronounced quality degradation in long-horizon generation. To address this, we propose **SDFlow** (**S**imilarity-**D**riven **Flow** Matching), a non-autoregressive framework that operates entirely in the frozen VQ latent space and enables parallel sequence generation via flow matching. We tackle three key challenges in making this transition: (1) eliminating exposure bias by replacing step-wise token prediction with a global transport map; (2) mitigating the high-dimensionality of VQ token spaces via a low-rank manifold decomposition with a learned anchor prior over the latent manifold; and (3) incorporating discrete supervision into continuous transport dynamics by introducing a categorical posterior over codebook indices within a variational flow-matching formulation. Extensive experiments show that SDFlow achieves state-of-the-art performance, improving Discriminative Score and substantially reducing Context-FID, particularly for challenging long-sequence generation. Moreover, SDFlow provides significant inference speedups over autoregressive baselines, offering both high fidelity and computational efficiency. Code is available at *https://anonymous.4open.science/r/SDFlow-D6F3/*

Search-augmented reasoning agents interleave internal reasoning with calls to an external retriever, and their performance relies on the quality of each issued query. However, under outcome-reward reinforcement learning, every search decision in a rollout shares the same trajectory-level reward, leaving individual queries without step-specific credit. Recent process-supervision approaches address this gap by drawing step-level signals from outside the policy, relying either on a much larger teacher model, or on sub-question annotations produced by a stronger external system. In contrast, we propose \textbf{SD-Search}, which derives step-level supervision \emph{from the policy itself} through hindsight self-distillation, requiring neither an external teacher nor additional annotations. In SD-Search, a single model plays two roles that differ only in conditioning: a student that sees only the context available at inference time, and a teacher that additionally conditions on a compact hindsight block summarizing the search queries and final outcomes of a group of rollouts sampled from the same question. Since teacher knows how each rollout unfolded and which ones succeeded, its query distribution implicitly marks which decisions were worth making, and the student is trained to recover this behavior by minimizing the token-level Jensen--Shannon divergence to the teacher at search-query positions. This layers a dense, step-level signal on top of GRPO's coarse trajectory reward. Crucially, this signal requires only a single teacher forward pass per rollout at training time, all absorbed within the standard RL loop without external model inference, auxiliary annotation pipeline, or additional training. On seven single-hop and multi-hop QA benchmarks, SD-Search reaches $0.428$ average Exact Match with Qwen2.5-3B, matching the leading process-supervision baseline without its external teacher, and $0.476$ with Qwen2.5-7B, surpassing all outcome-reward and process-supervision baselines on the average.

Few-shot adaptation of vision--language models must balance two competing goals: learning fine-grained visual distinctions from very limited supervision, and preserving the efficiency that makes frozen CLIP-like models attractive for deployment. Existing lightweight adaptation methods, including prompt tuning, LoRA, and cache-based adapters, typically supervise global image embeddings and therefore underuse token-level structure. Conversely, structure-aware methods can exploit local evidence but often introduce additional inference-time computation. We propose SD-LoRA, a structural distillation framework that uses token-level graph reasoning only during training and absorbs its effect into a lightweight LoRA-adapted CLIP model. During training, a heterogeneous graph teacher models patch-patch, global-local, and visual-text relations over high-resolution ViT tokens. Rather than keeping this graph at test time, SD-LoRA transfers its structural bias into the shared CLIP-LoRA backbone through implicit distillation and Prototype Predictive Alignment, which aligns graph-conditioned representations with support-derived class prototypes. At inference, the entire graph teacher is discarded; prediction uses only a single CLIP-LoRA forward pass and a cache-based classifier. Across 11 few-shot benchmarks, SD-LoRA improves over a strong CLIP-LoRA baseline by 1.3/1.2/2.4/1.9/2.0% in the 1/2/4/8/16-shot settings, while adding no graph computation at test time. Further results on base-to-novel generalization, cross-dataset transfer, OOD robustness, and calibration show that training-time structural supervision improves not only accuracy but also transfer reliability.