NeurIPS 2026 Papers

Compositional energy-based models can generalize to larger combinatorial reasoning problems by reusing a learned factor energy across many local constraints. In our paper, we show that a key bottleneck in compositional reasoning is not composition itself, but the non-convex geometry of the learned energy landscape. To solve this problem, we introduce Convex Compositional Energy Minimization (CCEM), a framework that parameterizes each factor with an input-convex neural network and optimizes the composed energy over a tight convex relaxation of the feasible set. Because convexity is preserved under summation, the global relaxed objective remains convex, enabling deterministic projected first-order optimization. CCEM is trained in two stages: factor-level contrastive learning to shape local energy basins, followed by end-to-end refinement through an unrolled projected solver. Our experiments show that our models trained on small subproblems or a single problem size transfer to larger instances without retraining.

Test-time compute scaling, where language models "think longer" via extended chain-of-thought (CoT), is the dominant paradigm for improving reasoning. Prevailing intuition holds that extended reasoning *converges* toward an answer; we show the opposite. Across three reasoning-RL families (DeepSeek-R1, Qwen-QwQ, Llama-3.1-Nemotron) and three benchmarks (GSM8K, MATH, GPQA), step-to-step SAE feature turnover and entropy *rise* monotonically across the chain (Cohen's $d = 0.61$–$0.68$, $p < 10^{-3}$). Yet the final answer is already linearly decodable from the residual stream within the first $\sim 20\%$ of the chain. We call this pattern **commit then explore**: the answer becomes stably decodable early, and the bulk of test-time compute is then spent exploring *around* an already-decodable answer. Targeted ablation shows post-commit features have a $2$–$6\times$ smaller teacher-forced log-probability effect (matched-mass control: $1.5$–$4.6\times$) and a $2$–$3\times$ smaller free-regeneration answer-flip rate than pre-commit features. A terminal-reward RLVR credit-assignment account predicts this asymmetry, and the dynamic is absent in base models: Qwen-2.5 and Llama-3.1 bases fail our detector at every layer, and a held-out quantitative prediction derived from R1 generalises to Qwen-QwQ at Pearson $r = 0.96$. Leveraging the structure, we build a Reasoning Monitor that saves $26$–$34\%$ of inference compute at $\geq 92\%$ accuracy retention, dominating four adaptive-compute baselines. Extended chain-of-thought, under this view, is not the computation of an answer; it is the exploration of its neighborhood.

Most Vision-Language Models (VLMs) are built by extending pretrained Large Language Models (LLMs) with visual modules and multimodal alignment. However, this multimodal scaling often degrades the language-side reasoning ability originally encoded in the base LLM. While the base LLM retains usable reasoning ability after scaling, the aligned VLM itself cannot reliably access this ability. Therefore, recovering degraded reasoning capability in VLMs may be more effective when using the base LLM as the source rather than relying on the VLM alone. Motivated by this, we propose **LIFT** (**L**anguage-side reason**i**ng **F**acilitation and **T**ransfer), a lightweight vector-intervention method that transfers reasoning capability from the base LLM to the VLM without retraining the backbone. LIFT defines **Reasoning Vectors** as answer-token hidden-state differences between a Reasoner path with an explicit reasoning trace and a Solver path without it, and injects these vectors into language-side activations of the target VLM. LIFT further supports learnable vector adaptation while keeping the VLM backbone frozen. We evaluate LIFT on two VLMs across six reasoning benchmarks, comparing Reasoning Vectors extracted from the base LLM and from the aligned VLM under matched protocols. Results show that LLM-derived vectors consistently outperform VLM-derived vectors, confirming that the base LLM is a more effective source for recovering reasoning. LIFT partially recovers degraded reasoning through lightweight language-side interventions. Further analyses show that Reasoning Vectors influence intermediate reasoning behavior rather than merely altering final answers.

Large Language Models (LLMs) have become increasingly capable as tool-using agents, with benchmarks spanning diverse general agentic tasks. Yet rigorous evaluation of scientific tool use remains limited. In chemistry, recent agents can plan syntheses and invoke domain-specific tools, but evaluations often rely on curated demonstrations, expert assessment, or LLM-as-judge scoring rather than exact, judge-free ground truth. We address this gap with chemical procurement cost estimation, a practical task in which an agent must ground chemical identities, retrieve supplier quotes, select valid purchasable packs, normalize quantities, and compute cost from a reaction description. We introduce ChemCost, a benchmark of 1,427 evaluable reactions grounded to a frozen pricing snapshot covering 2,261 chemicals and 230,775 supplier quotes, supporting scalar scoring and stage-level diagnosis of grounding, retrieval, procurement, and arithmetic failures. To evaluate robustness, we further construct controlled noise-injected views that perturb chemical aliases, quantity expressions, missing fields, and input formatting. Experiments with frontier, open-weight, and chemistry-specialized LLM agents show that tool access is necessary but insufficient for solving the task. The strongest agents reach only 50.6% accuracy within 25% relative error on clean inputs and degrade substantially with realistic noise. Stage-level analysis further shows that failures arise from brittle parsing, ineffective evidence integration, invalid pack selection, and non-convergent tool use.

We study long-context balanced entropic optimal transport (OT) attention on TPU hardware through a stopped-base, fixed-depth tail-refinement surrogate. After a stopped $T$-step Sinkhorn solve, we unroll a short refinement tail and differentiate that surrogate exactly. For the production $R=2$ case, the backward pass contains four staircase plan factors. We prove an exact one-reference-tile schedule: the $R=2$ score cotangent is a single reference plan tile times an explicit modifier field built from vector cotangents and dual differences. This yields block-wise cost $O((T+R)LW)$, $O(Ld)$ input storage, and $O(L)$ additional HBM usage for fixed head dimension $d$ and band width $W$. We also formalize the current \texttt{dustbin\_block} path as the same balanced surrogate on an augmented support, so the schedule lifts to the gap-aware transport path used in our TPU runs. We provide a local surrogate-bias bound, an a posteriori bias certificate, and a projective contraction certificate for strictly positive active blocks. On synthetic masked problems, the optimized kernel matches exact autodiff of the same centered surrogate to within $10^{-5}$--$10^{-10}$. On TPU v6e-8, a four-configuration Pfam screen completes end-to-end, and a promoted balanced $R=2$ run sustains roughly $8.5$ examples per second through a three-hour budget, reaching step $1437$. Held-out Pfam test shards improve reconstruction from $3.17$ to $0.99$ and sparse CE from $5.86$ to $5.69$ relative to step $0$. These results support exact fixed-depth backward theory, a theorem-matching gap-aware bridge, and trainability evidence for the production path.

Neural denoisers trained with pointwise distortion losses such as MSE often suppress the temporal regularities that downstream tasks depend on---a failure that becomes acute under non-Gaussian, impulsive, or non-stationary disturbances. We propose a training framework that lifts a family of classical complexity descriptors---permutation entropy, sample entropy, Lempel-Ziv complexity, and Higuchi fractal dimension---into end-to-end differentiable objectives via calibrated soft surrogates with straight-through gradients. Three complementary mechanisms emerge from this lifting: (i) early fusion injects local structural priors as auxiliary inputs, (ii) a complexity-consistency regularizer matches the multi-scale complexity of denoiser outputs to clean references, and (iii) a curriculum sampler prioritizes examples whose noise-induced complexity inflation is largest. We prove that additive noise strictly inflates ordinal uncertainty for bounded piecewise-smooth signals, and that, under bounded surrogate calibration, complexity consistency yields a strictly larger expected decision margin than MSE-only training. On OFDM/QAM communication signals under AWGN, $1/f$ colored, and $\alpha$-stable impulsive noise, our framework reduces post-detection bit error rate by 29--47% over Transformer (PatchTST, iTransformer), GNN, and CNN baselines, with statistically significant gains across five seeds and 95% bootstrap confidence intervals. The improvements transfer to over-the-air conditions on the public RadioML 2018.01A benchmark (single-carrier QAM portion) and a new 50-session in-house SDR testbed, yielding 22--26% relative BER reduction without fine-tuning. The framework is architecture-agnostic and adds $\leq 3.5\%$ FLOPs over the underlying backbone.

Distilling "System 2" reasoning into compact student models is bottlenecked by the scarcity of fully-correct teacher traces: as task difficulty grows, rejection sampling discards an increasingly large fraction of teacher generations. "Wrong" (W) traces are abundant, but training on them indiscriminately causes *policy poisoning* the student internalises the teacher's hallucinated arithmetic alongside any useful reasoning structure. We propose **Atomic Reasoning Units (ARU)**, a framework that resolves this trade-off via strict syntactic constraints. ARU enforces a Backus--Naur-Form (BNF) grammar that decomposes each reasoning step into a Premise, an Operation, and a Result, algorithmically disentangling reasoning planning from arithmetic execution. *Counterfactual Logic Verification* (CLV) symbolically re-executes W-traces to recover the $68\%$ that have valid plans but faulty arithmetic, and *Syntactic Loss Masking* (SLM) trains the student on this verified structure while suppressing the gradient on hallucinated result tokens. Across the $3 \times 8 \times 6$ grid we evaluate ($\{0.6, 1.7, 8\}$\,B Qwen3 students $\times$ eight benchmarks $\times$ six baselines), ARU is the strongest method in every cell, improving GSM8K by $+11.1$ and MATH-500 by $+9.8$ over the closest baseline at $1.7$\,B; on the held-out AIME competition set ARU's Maj@$8$ is $14.8$ versus $9.5$ for PoT (the closest baseline). We further document a small-scale capability gap in which ARU at $0.5$\,B matches Gold-SFT at $\geq 1.5$\,B on GSM8K (paired-bootstrap $p<0.05$ after Bonferroni correction), with a pre-registered ProofWriter control showing the gap collapses on pure-logic reasoning---consistent with arithmetic offloading as the dominant mechanism, though we do not claim the control uniquely isolates it. The code is available at https://anonymous.4open.science/r/ARU-286D.