NeurIPS 2026 Papers

Moving beyond evaluations that collapse performance across heterogeneous prompts toward fine-grained evaluation at the prompt level, or within relatively homogeneous subsets, is necessary to diagnose generative models' strengths and weaknesses. Such fine-grained evaluations, however, suffer from a data bottleneck: human gold-standard labels are too costly at this scale, while automated ratings are often misaligned with human judgment. To resolve this challenge, we propose a novel statistical model based on tensor factorization that merges cheap autorater data with a limited set of human gold-standard labels. Specifically, our approach uses autorater scores to pretrain latent representations of prompts and generative models, and then aligns those pretrained representations to human preferences using a small calibration set. This sample-efficient methodology is robust to autorater quality, more accurately predicts human preferences on a per-prompt basis than standard baselines, and provides tight confidence intervals for key statistical parameters of interest. We also showcase the practical utility of our method by constructing granular leaderboards based on prompt qualities and by estimating model performance solely from autorater scores, entirely eliminating the need for additional human annotations.

Fine-tuning large language models (LLMs) on downstream tasks often weakens their previously aligned safety behavior, creating an inherent trade-off between task adaptation and safety preservation. Existing approaches attempt to mitigate this issue by identifying safety-relevant neurons, layers, or update directions in the original parameter space. However, safety-related information is often distributed across many directions in this space, causing downstream updates to overwrite parameters that encode safety behavior. We propose WSR-Tune, a Weight Space Reparameterization-based fine-tuning framework that preserves safety during downstream adaptation. Our approach constructs a safety-conditioned basis and reparameterizes the weight matrices such that safety-relevant information becomes concentrated in a small subset of directions. We then identify safety-critical directions and freeze them in the reparameterized space, while updating the remaining complementary directions for downstream adaptation. By explicitly structuring the parameter space to disentangle safety and task information, our method reduces interference between objectives and enables simultaneous preservation of safety and improvement in downstream performance. Experimental results show that our method achieves a more favorable trade-off between downstream performance and safety retention, demonstrating its effectiveness for reliable LLM fine-tuning.

EEG foundation models (FMs) offer strong population-level priors but degrade on individual subjects due to anatomy-driven distribution shifts. We propose Rest-Tuning, a task-agnostic calibration framework that personalizes a population FM using just 1–3 minutes of unlabeled resting-state EEG. Rest-Tuning employs masked teacher-student alignment to produce a subject-calibrated backbone, reused as initialization across multiple tasks from the same subject. Across two multi-subject datasets and four EEG FMs, Rest-Tuning outperforms task-only adaptation, with up to 13.3% absolute gain in downstream task accuracy and over 40% average reduction in labeled task data. Calibration saturates quickly and requires under 3 minutes of compute per subject. Representation analyses show reduced subject-to-population distance and smoother LoRA adapter interpolation paths, indicating improved compatibility between rest- and task-adapted solutions. These results establish resting-state EEG as a practical, fast, and reusable calibration interface, enabling population EEG FMs to function as reliable, subject-specific BCIs and serve as reusable initialization for downstream task adaptation.

Onboard satellite models often require frequent updates, but the weights adapted to earlier data distributions can quickly become outdated. However, updating large-scale model parameters in orbit presents significant challenges due to the limited uplink bandwidth of Low Earth Orbit (LEO) satellite systems, particularly for hyperspectral satellite imagery, where high-dimensional spectral–spatial inputs lead to increased model size and update costs. Existing full fine-tuning methods are thus expensive to retrain and difficult to deploy under strict communication constraints. To address this challenge, we propose NE-LoRA, a parameter-efficient adaptation framework for bandwidth-constrained onboard hyperspectral model updates. NE-LoRA combines a primary low-rank branch with a nonlinear auxiliary branch to capture both global update trends and complex spectral–spatial variations. Additionally, we introduce a differentiated training strategy for multi-matrix adapters, motivated by the asymmetric initialization and gradient dynamics of different adapter matrices. Experiments on four hyperspectral datasets and three representative backbone models demonstrate that NE-LoRA consistently outperforms LoRA-based baselines and remains competitive with, and in several cases superior to, full fine-tuning. Across the evaluated settings, NE-LoRA updates only a small fraction of the total parameters on average while preserving low deployment overhead, offering a favorable accuracy–communication trade-off for onboard hyperspectral adaptation.

In a 100-body robot co-design benchmark, the height threshold that resets a falling robot changes which bodies appear optimal. We train PPO controllers for DERL-evolved UNIMAL morphologies under Standard (τ = 0.20), Strict (τ = 0.50), and Reset-free (τ = 0) protocols, then evaluate every policy under a common protocol. Strict and Reset-free training share only 5 of their top-10 bodies and have low rank agreement (Spearman ρ = 0.21). The effect is family-specific, not a uniform penalty: averaging per-body retention ratios, Reset-free training reduces Floor bodies' per-step reward by 45% relative to Standard training but reduces variable-terrain bodies by 16%. Under Strict, the losses are 22% for Floor and 34% for variable-terrain bodies, while multi-task bodies are most stable across protocols. Blocked ANOVA and morphology-level permutation tests support the protocol-by-family interaction. A SAC validation panel planned before analysis finds the same Strict-versus-Reset-free family ordering under a different optimizer. Selecting bodies by their minimum reward across the three training protocols recovers ≥91% of every single-protocol oracle's top-k average reward (k ∈ {5, 10, 20}; all bootstrap lower bounds ≥84.3%). Co-design benchmarks should state the reset rule, audit ranking sensitivity, and report protocol-robust selections when choosing bodies.

Adapting language models on long-form corpora can improve domain behavior while also making protected passages easier to reproduce verbatim from short prefixes. We study this tension through \emph{relative energy gain}, the token-level log-density ratio between an adapted model and its pre-adaptation anchor on an exact protected continuation. This quantity separates ordinary predictability from update-induced copying pressure and accumulates over suffixes as a likelihood-ratio advantage for extractable memorization. Building on this view, we introduce Energy-Gated Copyright Regularization (EGCR), a training objective that augments maximum likelihood with a soft relative-energy budget on protected tokens. EGCR uses a differentiable gate combining local persistence of relative gain with anchor surprisal, so regularization is concentrated on distinctive spans whose probabilities rise in a memorization-like way while ordinary next-token supervision is retained elsewhere. Across BookMIA adaptation stress tests, training-prefix extraction, CopyBench transfer, and gate ablations, EGCR reduces exact-match and overlap-based reproduction while preserving held-out language-modeling and book-related utility. Token-level visualizations further show that the gate focuses on a small set of expressive positions associated with verbatim recovery. These results suggest that relative-energy barriers offer an effective and inspectable mechanism for copyright-aware language model adaptation.

Wasserstein distributionally robust optimization (DRO) is a prominent paradigm for robust decision-making in the face of distributional uncertainty. Given a nominal data distribution, DRO selects a decision which minimizes the risk for a worst-case data distribution within a prescribed Wasserstein radius $\varepsilon$ of the nominal distribution. In this paper, we investigate the quality of DRO decisions when evaluated on a worst-case distribution within a 2-Wasserstein $\varepsilon$-neighborhood of the nominal distribution, as measured by excess risk or ex-ante regret. Beginning with multivariate linear regression and extending to ridge regression, we first identify the asymptotic complexity of robust decision-making in the $\varepsilon \to 0$ limit, characterized by a problem-specific condition number $\kappa$. In this regime, a wide spectrum of simple algorithms, including DRO, achieve the instance-optimal rate of $O(\kappa \varepsilon^2)$. For fixed $\varepsilon > 0$, we prove that DRO still achieves the minimax rate if the problem is of an appropriate ``low rank''. On the other hand, we identify significant failure modes where DRO is provably suboptimal by dimension-dependent factors. To resolve this, we introduce a new condition number reduction (CNR) procedure which achieves the optimal rate and admits tractable approximation algorithms. We support these theoretical results with numerical experiments comparing the performance of various estimators including DRO and CNR.

General-purpose behavior models increasingly internalize broad procedural knowledge at training time, making adaptation less a problem of learning entirely new behaviors than of reconfiguring existing competence for new goals, object relations, and skill interfaces. We study this as reconfigurability, the ability to realize new behaviors by restructuring components and relations cost-efficiently, under *sparse governing information*, where a task specification provides high-level goals or abstract directives but omits the concrete bindings and compositions required for execution. We propose RecPo, a framework that represents skills task-agnostically through multiple procedure views and materializes a new task by composing only the transition relations consistent with the task-relevant views. RecPo maintains a procedure knowledge base of local transition relations within each view. At deployment time, it interprets a task demonstration through the same views, converts observed view-level changes into a compositional query, and resolves the query into an executable skill sequence. On an extended Franka Kitchen benchmark with held-out long-horizon tasks outside the stored compositions, RecPo achieves 91.7\% ordered subtask completion, substantially outperforming non-reconfigurable skill retrieval baselines. Real-robot case studies further show deployment-time adaptation through recomposition rather than task-specific retraining.

Recent 3D foundation models, such as DUSt3R, MASt3R, VGGT, $\pi^3$, and Depth Anything 3, provide strong feed-forward depth and pose estimates on pinhole imagery, but degrade sharply under fisheye camera geometry. We show that this failure is partly caused by a pinhole camera bias in the positional encodings of pretrained 3D foundation models, and propose RayTun3R, a lightweight camera adaptation approach. It keeps the pretrained network fixed and adapts only lightweight components tied to token position and camera geometry. RayTun3R learns parameter-efficient residual corrections to absolute and rotary positional encodings, together with parameter-free tokenization and corrections to prediction-grid coordinates that remove residual pinhole assumptions. The resulting adapter contains only 10,752 trainable parameters and can be learned from a short temporal segment using geometric losses. Once adapted, RayTun3R transfers effectively to the remaining frames of the sequence without incurring additional runtime costs. Across diverse fisheye datasets with fields of view from $110^\circ$ to $200^\circ$, our adapter reduces rotation error by $2-12\times$ relative to the unadapted model, outperforms LoRA while using $\sim14\times$ fewer trainable parameters, improves pose over adaptation-free baselines while avoiding their multi-view inference cost, and remains competitive on depth accuracy.

The growing context lengths of large language models (LLMs) have made key-value (KV) cache memory a critical bottleneck during inference. Vector quantization (VQ) with commutative codebooks provides an effective Rotary Position Embedding (RoPE)-aware solution by enabling reconstruction-free attention over quantized key indices. However, existing commutative VQ methods still parameterize scalar codebook coefficients across RoPE subspaces and codewords largely independently, leaving parameter-side redundancy underexploited. We propose $\textbf{RankVQ}$, a low-rank parameterized commutative vector quantization method for KV cache compression. Instead of factorizing the deployed codebooks, RankVQ factorizes the underlying scalar coefficient matrices used to construct key and value codebooks. This design preserves the pseudo-symmetric structure required for RoPE commutativity on the key side, while improving the quality and compactness of offline codebook construction. The low-rank factors are discarded after codebook construction, so RankVQ keeps the same online inference form as standard commutative VQ and introduces no additional decoding overhead. Experiments on LongBench, InfiniteBench, and GSM8K show that RankVQ achieves a stronger accuracy--compression trade-off than directly comparable KV quantization baselines, with particularly clear gains in the 1-bit regime.

Deep learning models trained to optimize average accuracy often exhibit systematic failures on particular subpopulations. In real-world settings like healthcare, the subpopulations most affected by such disparities are frequently unlabeled, partially observed, or not known in advance. Existing group-robust methods typically assume prior knowledge of the relevant subgroups, using group annotations for training, validation, or model selection. We propose Misclassification Aware Rank-Limited Adaptation (MARLA), a parameter-efficient method for improving worst group performance without explicit subgroup annotations. MARLA leverages an ERM-trained model by calculating the model's misclassification probability scores on a held-out adaptation set to identify a low-dimensional subspace where errors concentrate. We then learn a rank-restricted additive correction to the classifier logits within that subspace. Across seven real-world datasets, we evaluate group robustness under three settings: no knowledge of subgroup relevance, partial knowledge of subgroup relevance, and full knowledge of subgroup relevance. MARLA improves worst-group performance while remaining fast, parameter-efficient, and practical to tune.

Graph representation learning has become a standard approach for analyzing networked data, with latent embeddings widely used for link prediction, community detection, and related tasks. Yet a basic design choice, the latent dimension, is still treated as a brittle hyperparameter, fixed before training and tuned by held-out performance. Learned factors are also identifiable only up to rotation and rescaling, so the nominal rank rarely coincides with the quantity that governs model behavior. We propose Spectral Prefix Extraction and Capacity-Targeted Representation Analysis (Spectra), which replaces rank as the unit of analysis with the spectrum of a learned positive semidefinite kernel, trace-normalized so that spectra are comparable across fits. The normalized eigenvalues form a distribution on the simplex, and their Shannon effective rank acts both as a summary of learned capacity and as a controllable training-time coordinate: a single scalar shapes this realized dimension during training, and bisection targets any desired value within the rank cap. To theoretically support that, we show local regularity and monotonicity of the realized-dimension profile. Across collaboration, social, biological, and infrastructure networks, Spectra traces performance--capacity frontiers that make the trade-off between predictive accuracy and realized dimension visible. It performs competitively with strong link-prediction baselines, yields aligned lower-capacity views of the same fitted model through spectral prefixes, and provides a principled handle on capacity in the overparameterized regime. Capacity thus becomes a property of the fitted model rather than a hyperparameter of the training.

Large language models are increasingly deployed as long-lived agents that must adapt across users, tasks, domains, modalities, and feedback regimes without access to model weights. Existing black-box adaptation methods typically optimize a single prompt, maintain an undifferentiated memory, or rely on repeated rollout-heavy search. However, these designs struggle when streams of input are nonstationary, feedback is sparse, and failures from one task family can contaminate behavior on another. We introduce RIZZ (**R**outing **I**nteractions to Near **Z**ero-interference **Z**ones), a continual adaptation framework for compound language-model systems that learns entirely through verifier-gated memory, routing, and prompt compilation. RIZZ organizes input streams into dynamically spawned memory branches. At inference time, either while online or offline, a context-aware router selects or creates a branch that retrieves branch-local, global, graph-structured, and working-memory context, which is compiled into a bounded prompt together with retrieved task evidence. After the model acts, task verifiers score the output, and only verified interactions can update memory, promote reusable rules, demote harmful rules, or create anti-patterns. This yields a black-box agent that improves through persistent natural-language feedback while explicitly controlling interference. RIZZ targets the regime where adaptation must occur online under context budgets. Finally, we demonstrate the effectiveness of our framework against state-of-the-art baselines on competitive benchmarks.

Single-step inorganic retrosynthesis is evaluated by whether a complete precursor set is recovered, yet many data-driven systems first rank individual precursors and then rely on threshold or count heuristics to assemble sets. This mismatch is especially severe when the true set size is unknown: individually plausible precursors can combine into incomplete, over-complete, or otherwise wrong near-miss sets. We reformalize the task as variable-size set-level ranking and introduce RECIPE, a target-conditioned framework that separates precursor recall from final set-level scoring. The Precursor Candidate Generator learns formula-level compatibility to build a high-recall precursor pool, while the Complete-Set Reranker compares variable-size candidate precursor sets directly. On the Retrieval-Retro year-split benchmark, the generator improves Combo@20 from 69.00 to 72.75. Compared with Retrieval-Retro, the reranker improves Combo@1 by 11.30 points to 71.70, Combo@20 by 20.82 points to 89.82, and Combo MRR by 14.14 points to 77.43. A preliminary 20-case out-of-distribution evaluation shows the same direction of improvement. These results suggest that set-level prioritization remains important after strong precursor recall, and that optimizing ranked complete precursor sets can reduce the inspection burden in inorganic synthesis planning.

Transformers with self-attention modules as their core components have become an integral architecture in modern large language and foundation models. In this paper, we study the evolution of tokens in deep encoder-only transformers at inference time which is described in the large-token limit by a mean-field continuity equation. Leveraging ideas from the convergence analysis of interacting multi-particle systems, with particles corresponding to tokens, we prove that the token distribution rapidly concentrates onto the push-forward of the initial distribution under a projection map induced by the key, query, and value matrices, and remains metastable for moderate times. Specifically, we show that the Wasserstein distance of the two distributions scales like $\sqrt{\log(\beta+1)/\beta}\exp(C t)+\exp(-ct)$ in terms of the temperature parameter $\beta^{-1}\to 0$ and inference time $t\geq 0$. For the proof, we establish Lyapunov-type estimates for the zero-temperature equation, identify its limit as $t\to\infty$, and employ a stability estimate in Wasserstein space together with a quantitative Laplace principle to couple the two equations. Our result implies that for time scales of order $\log\beta$ the token distribution concentrates at the identified limiting distribution. Numerical experiments confirm this and, beyond that, complement our theory by showing that for finite $\beta$ and large $t$ the dynamics enter a different terminal phase, dominated by the spectrum of the value matrix.

Large language models (LLMs) have demonstrated remarkable performance in video question answering (VQA). To improve responsiveness and answer quality, modern LLM-based VQA services pre-extract multiple LLM-compatible modalities from videos before response generation. However, the performance impact of modality combinations (MCs) and extraction knob settings has barely been studied. Our analysis shows that both factors substantially affect answer quality and Time-to-First-Token (TTFT) latency. Furthermore, the best configuration depends on both video content and query semantics, necessitating adaptive strategies. We propose Prysma, the first modality adaptation framework for LLM-based VQA. Prysma intelligently adapts the modality configurations to optimize answer quality, subject to a Service Level Objective (SLO) on average TTFT. It combines an offline modality orchestrator and an online semantic adapter. The offline orchestrator constructs a video knowledge base to warm-start tuning, employs a latency-aware configuration pruner to reduce search space, and runs a two-stage tuner to optimize both video- and query-level configurations. The online adapter selects configurations in real time based on query semantics and tuning history. Extensive evaluations demonstrate that Prysma improves answer quality by up to 82% over SOTA methods under the same SLO setting.

Noise search at inference time improves text-to-image diffusion quality but requires many multi-step teacher rollouts per query. Distilled few-step models are now routinely released for efficient inference, but their potential as proxies inside noise search is uncharacterised. We measure proxy--teacher rank agreement across three distillation regimes and three backbones spanning DiT, MMDiT, and UNet, and find a sharp asymmetry: eight closed-form image statistics preserve rank from the distilled proxy to the teacher (Spearman $\rho$ up to $0.80$), while learned semantic scorers decorrelate ($\rho \le 0.12$). Null controls localise the transfer to distillation training rather than few-step denoising or initial noise characteristics. We therefore reformulate noise search as a \emph{joint constraint} and propose ProxySearch: the proxy filters candidates by statistical match in $1$ step, the teacher argmaxes a user-chosen semantic reward over the survivors without a tunable weight trading semantic reward against the statistical target. \textsc{ProxySearch} matches or beats teacher-only argmax on compositional accuracy and human preference, while tightening the statistical target by $\sim 23\%$ in paired statistical $\ell_1$ relative to teacher-only argmax at matched compute on FLUX-$1024$.

We study representation learning in low-rank Constrained Markov Decision Processes (CMDPs), where both value and reward functions are approximated by a set of unknown representation vectors, and the transition dynamics admit a low-rank factorization. The objective is to maximize expected cumulative rewards while complying with constraints on the expected cumulative utility. To achieve this, we propose \texttt{MFRLC} (Model-Free Representation Learning for Low-Rank CMDP), a model-free algorithm that uses a primal-dual scheme to effectively balance reward regret and constraint violations. To the best of our knowledge, \texttt{MFRLC} is the first representation-learning method for low-rank CMDPs that combines the Least-Squares Value Iteration with Upper Confidence Bound (LSVI-UCB) and primal-dual techniques. \texttt{MFRLC} further enhances value function estimation with bonuses for function approximation and uniquely learns the mapping from representations to value functions directly through representation learning. We prove that \texttt{MFRLC} achieves both regret and cumulative constraint violation of order $\widetilde O(H^3 d^2 K^{3/4}|\mathcal{A}|^{3/2}/\gamma)$, making it provably sample-efficient and highly adaptable to complex environments due to the reliance on function approximation.

Representation learning has substantially improved the efficiency of unconstrained continuous-state Markov Decision Processes (MDPs), but its extension to Constrained MDPs (CMDPs) is difficult because the transition representation must be learned while constraint violation is controlled. In this paper, we study representation learning for low-rank CMDPs with continuous state spaces, and propose REP-PD-TEG for the soft cumulative-violation setting. In the feature learning period, the algorithm interacts with the environment using a decaying-$\epsilon$-greedy mixture execution policy, and uses the collected samples to learn transition features by maximum likelihood. Based on the learned features, it constructs uncertainty bonuses and plans with an optimistic Lagrangian objective to obtain the return policy. We prove regret and soft-violation guarantees for the return policies, and quantify the influence of the decaying-$\epsilon$-greedy exploration. In particular, REP-PD-TEG achieves the optimal \(\widetilde O(\sqrt K)\) return policy bound, and gives a \(\widetilde O(K^{3/4})\) overall sample bound when exploratory actions are charged. For the stricter hard cumulative violation, where feasibility must be verified episode by episode, we propose REP-PD-H-TEG. It plans under a bonus-corrected lower-confidence utility certificate with the learned representation, and performs a stabilized dual search to obtain the return policy, yielding regret and hard-violation bounds. Together, these results give the first theoretically guaranteed representation-learning method for low-rank continuous-state CMDPs with both soft cumulative guarantees and certified hard-violation control.