NeurIPS 2026 论文列表

Knowledge editing has emerged as a promising approach for efficiently updating embedded knowledge in large language models (LLMs). It first computes an ideal target hidden state that steers the LLM toward the new output, and then edits the model parameters so that the original hidden state is mapped to this target. While most existing methods focus on the second stage of parameter editing in preserving other knowledge, we find that the upstream construction of the target hidden state also plays an important role in allocating the model's limited capacity space. We first establish a theoretical connection between interference in the parameter space and interference between target hidden states, showing that the two are positively correlated. Building on this insight, we propose a simple yet effective $\ell_1$ regularization method that prunes away unnecessary ``tentacles'' of each knowledge vector, retaining only the essential components. With fewer extraneous tentacles, each edit affects less other knowledge, enabling better coexistence. Extensive experiments across multiple LLMs and benchmark datasets verify the effectiveness of our method.

Large language model (LLM) post-training enhances latent skills, unlocks value alignment, improves performance, and enables domain adaptation. Unfortunately, post-training is known to induce forgetting, especially in the ubiquitous use-case of leveraging third-party pre-trained models, which is typically understood as a loss of parametric or factual knowledge. We argue that this accuracy-centric view is insufficient for modern foundation models and instead define forgetting as systematic model drift that degrades behavior and user experience. In this context, we introduce __CapTrack__, a capability-centric framework for analyzing forgetting in LLMs that combines a behavioral taxonomy with an evaluation suite centered on capability-specific metrics. Using CapTrack, we conduct a large-scale empirical study across post-training algorithms, domains, and model families, including models up to 80B parameters. We find that forgetting extends beyond parametric knowledge, with pronounced drift in robustness and default behaviors. Instruction fine-tuning induces the strongest relative drift, while preference optimization is more conservative and can partially recover lost capabilities. Differences across model families persist, and no universal mitigation emerges.

Many prediction problems involve a hidden predictive state: a latent task, environment, rule, or data-generating mechanism that changes the query-optimal act. For a fixed query, recovering the full hidden predictive state is often stronger than necessary. The relevant target is the Bayes-act distinction among hidden predictive states that preserves oracle-gain adaptation. This question is central for in-context learning (ICL): a prompt may reveal part of an unseen task or rule, but an ICL system needs only to recover the prediction-relevant quotient state label for the query. We formalize this target through the canonical predictive quotient (CPQ), a problem-relative Bayes-act quotient of hidden predictive states. The theory characterizes the value of adaptation, the quotient that preserves full oracle gain, the sharp zero/positive-gain boundary, and a Bayes barrier showing why context-only prediction cannot recover a positive oracle gain at the risk level. We give two exact examples. A finite-state example makes the quotient structure, sharp positivity boundary, and a strict-loss mechanism explicit. A linear-Gaussian prior-family witness gives closed forms for a collapsed Gaussian baseline and a state-aware oracle, a restricted zero-gain boundary, and an exact prediction-level disagreement identity for the restricted gain. Together, the results give a decision-theoretic target for ICL: not full latent complexity, but the prediction-relevant quotient state label. We support the theory with exact checks, realized bridge experiments over prediction-relevant quotient state labels, and external validity tests on released pretrained in-context predictors.

Dynamic Sparse Training (DST) is an effective paradigm for sparse to sparse training of neural network connectivity under a fixed parameter budget. Recent advances in network-science for AI introduced epitopological learning methods, such as Cannistraci-Hebb Training (CHT), demonstrating that network automata applied to the mere network topology enables gradient-free predictions of the sparse connectivity evolution that can trigger significant increase in task performance. However, these methods were currently developed for standard MLP-like fully-connected architectures and need significant rethinking to be extended to convolutional neural networks (CNNs) due to element-wise weight sharing and spatially repeated interactions. In this work, we bridge this gap by introducing epitopological learning DST for CNNs through channel-wise network modeling to address the prohibitive complexity of element-wise modeling, which treats each kernel-channel as a single network node. Results show orders-of-magnitude improvements in time complexity of link-regrowth when applied at channel-level with respect to element-level. We also enhance the channel-wise formulation with lightweight contextual modulation to improve expressivity. Extensive experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet with ResNet and VGG architectures demonstrate that our method achieves performance comparable to dense training using only 30% of the parameters, which significantly reduces computational cost. Furthermore, our approach improves robustness under noisy inputs. These results highlight the effectiveness of topology-aware sparse training and the benefit of decoupling graph construction from sparse update rules in convolutional networks.

Active computer vision promises efficient, biologically plausible perception through sequential, localized glimpses, but lacks scalable general-purpose architectures and pretraining pipelines, leaving Active-Vision Foundation Models (AVFMs) underexplored. We introduce CanViT, the first task- and policy-agnostic AVFM. CanViT uses scene-relative RoPE to bind a retinotopic Vision Transformer backbone and a spatiotopic scene-wide latent workspace, the canvas. Efficient interaction with this high-capacity working memory is supported by Canvas Attention, a novel asymmetric cross-attention mechanism. We decouple thinking (backbone-level) and memory (canvas-level), eliminating canvas-side self-attention and fully-connected layers to achieve fast sequential inference and scalability to high output resolutions. We propose a label-free active vision pretraining scheme, policy-agnostic passive-to-active dense latent distillation: reconstructing scene-wide DINOv3 embeddings from sequences of low-resolution glimpses with randomized locations, zoom levels, and lengths. We pretrain CanViT-B from a random initialization on 13.2 million ImageNet-21k scenes--an order of magnitude more than previous active models--and 1 billion random glimpses, in 166 hours on a single H100. On ADE20K segmentation, a frozen CanViT-B achieves 38.5% mIoU in a single low-resolution glimpse, outperforming the best active model's 27.6% with 20x fewer inference FLOPs as well as its FLOP- or input-matched DINOv3 teacher. Given additional glimpses, CanViT-B reaches 45.9% ADE20K mIoU. On ImageNet-1k classification, CanViT-B also sets a new active-vision state of the art, with 84.5% top-1 accuracy after fine-tuning. CanViT generalizes to longer rollouts, larger scenes, and new policies. Our work narrows the wide gap between passive and active computer vision, demonstrating the potential of task- and policy-agnostic AVFM pretraining.

Relational out-of-distribution errors are easy to describe and hard to evaluate cleanly. An image may be natural, a caption may be fluent, and all mentioned objects may be common, yet the caption may describe the wrong relation in the image. Many evaluations intended to test this behavior can be solved without checking the pairing at all, because negatives differ in caption style, template, length, object frequency, or image statistics. We propose counterfactual pairing cycles, a blocked evaluation primitive that reuses the same two images and the same two captions on both sides of the comparison, changing only which image is paired with which caption. This makes image-only, text-only, and additive component shortcuts cancel exactly in finite samples. The same contrast also gives a training loss for relational outlier exposure and a diagnostic for bilinear vision-language scores. Across synthetic controls, GQA, and COCO, cycles separate coarse image-caption matching from stricter role-binding failures. In a shortcut-contaminated COCO outlier-exposure stress test, a model selected by a perfect shortcut validation AUROC transfers poorly to exact relational cycles, while cycle training improves CycleAcc from \(0.576\) to \(0.911\). The main lesson is simple: to test whether a model understands a pairing, the benchmark should keep the parts fixed, change the pairing, and ask only about coupling.

Legal judgment prediction has made substantial progress in recent years, yet existing work evaluates whether models reach correct conclusions rather than whether they reason correctly. The core of legal decision-making is the mapping of case facts to the elements of an applicable legal provision, a step that prior benchmarks neglect. We introduce the first benchmark for evaluating legal reasoning, built on 375 judgments of the European Court of Human Rights (ECtHR) discussing Art. 11 ECHR on freedom of assembly. We exploit that the court routinely splits the decision in a stage reporting the facts of the case, and an evaluation stage in which it uses these facts selectively for justification. We give the (preprocessed) list of facts to 10 downstream LLMs and evaluate their justifications. The pipeline is validated by three expert legal annotators. Downstream models are reasonably accurate in predicting whether Art. 11 has been violated, and which elements of doctrine are critical, but perform substantially worse at attributing specific facts to doctrinal elements. Essentially they make the right decision, but for the wrong reasons. This weakness is invisible under standard evaluation metrics. Our benchmark, pipeline, and evaluation framework are released. They can easily be adapted to any legal rule that follows the Civil law tradition, i.e. to legal reasoning that requires the mapping of facts to an abstract legal rule.

AI benchmarks increasingly leverage item-level statistical models, particularly item response theory (IRT), to estimate capabilities, rank systems, select informative examples, and diagnose benchmark quality. However, AI benchmark data often departs from the data regimes encountered in human testing, for which standard IRT estimation tools were originally developed: these benchmarks typically involve fewer evaluated models, far more items, and capability distributions that may be skewed, clustered, or multimodal. We examine how these regime mismatches challenge the reliability of IRT modeling for AI evaluation. Using item parameters and capability distributions derived from six widely used LLM benchmarks, we simulate response matrices under three common IRT models and compare four estimators used in recent benchmark studies: marginal maximum likelihood, Markov chain Monte Carlo, variational inference, and a neural pseudo-Siamese estimator. Across 18,000 simulation runs, we systematically evaluate computational feasibility, scalability, and the reliability of IRT inferences about model rankings, predicted performance, and item characteristics. Results show that classical estimators can become computationally infeasible for large benchmarks, whereas scalable estimators can produce unreliable item-level and ranking inferences with small or non-normally distributed model sets. This study identifies when latent trait models reliably support or risk distorting AI benchmarking claims, and what sample sizes and diagnostics are needed for trustworthy use.

Since Vision-Language Models (VLMs) are the core of the perceptual and reasoning capability in Vision-Language-Action models, which are deployed in environments shared with humans, evaluating VLMs' safety reasoning from video is critical for preventing human harm. Current video-based benchmarks, however, evaluate only whether a model can recognize dangers that are already visible, and overlook two capabilities that VLMs require: reasoning under hierarchical principles in which human safety takes precedence over obedience to user instructions, and anticipating harm rather than identifying it after it has occurred. We introduce VESPER, a video benchmark that targets both capabilities by scenarios in which a robot is executing a user-assigned task when a person unexpectedly moves into harm's way, requiring the model to predict the potential harm and decide whether to continue or stop. The benchmark comprises 458 videos covering matched safe and dangerous variants, built through a human-in-the-loop pipeline from diverse taxonomies. We evaluate 21 VLMs, and the results indicate that the current VLMs fail to properly decide the actions under such a scenario.

Passive image provenance asks whether pixels alone can reveal where an image came from: a human, an aggregate AI class, or a particular generator. This becomes a robustness problem once a source image can be edited before the verifier sees it. We study the problem as source--target verification under adversarial distribution shift. Our first result gives the exact best-case limit for any image-only verifier: the largest robust target-acceptance gap equals the minimum total-variation distance between the target distribution and the set of attacked source distributions. This quantity depends on the source, target, and edit class, not on the verifier architecture. Our second result explains why deployed public verifiers can fail before this statistical limit is reached. If the verifier can be emulated on the attack region to error $\varepsilon$, then a surrogate black-box attack reaches target acceptance within $2\varepsilon$ plus optimization error of the white-box optimum; score-revealing logistic and softmax heads over public features are identifiable, and approximate score access gives stable recovery bounds. Experiments on same-prompt real/diffusion benchmarks support this separation. Public CLIP-based interfaces collapse under targeted attacks, and stronger clean or adversarial retraining does not restore positive operational separation. In our evaluated settings, positive empirical upper bounds on the robust gap appear only under private low-bandwidth binary interfaces with abstention. The main lesson is simple: robust passive provenance requires both a source--target statistical analysis and an interface-aware evaluation of what the verifier reveals.

Model merging techniques, which aggregate independently finetuned models into one to combine their capabilities, have become a topic of significant interest in recent years, with a broad array of methods having been proposed to tackle this problem. Simultaneously, an emerging trend in distributed learning has been the use of methods such as local SGD and DiLoCo, which greatly reduce communication costs by periodically aggregating the independently trained local models. However, these communication-efficient methods have been shown to degrade in performance relative to the FLOP-matched data-parallel gold standard as the number of independent local models grows and as the number of local training steps before global communication is increased. In this work, we draw an explicit analogy between the pseudo-gradient aggregation step in local SGD/DiLoCo and task arithmetic-based model merging, establishing a straightforward way to utilize merging methods in the context of distributed optimization. We then evaluate multiple state-of-the-art model merging methods in this setting and identify one method in particular, Iso-C, as a promising approach for improving DiLoCo. We find that DiLoCo SGD with Iso-C aggregation outperforms not only simple pseudo-gradient averaging but even the momentum-based DiLoCo, despite lacking a momentum mechanism itself. Building on this finding, we propose IsoLoCo, which adapts Iso-C for distributed training by equipping it with Nesterov momentum. Our empirical evaluations on language model pre-training across varying numbers of local workers show that IsoLoCo significantly outperforms DiLoCo, with the gap between them widening as the number of workers increases. This advantage remains present across model sizes and inner step counts, confirming that merging-inspired aggregation is an effective strategy for low-communication distributed training.

Source labels tell us where a sample came from, but not how that source oriented its coordinates. In a sparse Gaussian multi-source model with source-specific sign gauges, that distinction changes the learning problem. Without calibration, pooling across sources erases the mean signal, and the source-labelled experiment reduces to sparse PCA. With task-independent relative-gauge metadata, synchronization aligns sources and the same observations become a sparse mean problem. We formalize the value of metadata through aligned mass, which quantifies how much first-order signal a metadata-induced alignment restores. Source labels create only limited aligned mass, whereas valid calibration can create aligned mass proportional to the full sample size. This yields a statistical separation in micro-source regimes and, under standard sparse-PCA hardness assumptions, a computational separation as well. Experiments confirm the predicted phase transition, the failure of invalid metadata, reuse of one calibration graph across many tasks, and controlled real-feature settings with imposed source gauges.

Most Vision-Language Models (VLMs) are built by extending pretrained Large Language Models (LLMs) with visual modules and multimodal alignment. However, this multimodal scaling often degrades the language-side reasoning ability originally encoded in the base LLM. While the base LLM retains usable reasoning ability after scaling, the aligned VLM itself cannot reliably access this ability. Therefore, recovering degraded reasoning capability in VLMs may be more effective when using the base LLM as the source rather than relying on the VLM alone. Motivated by this, we propose **LIFT** (**L**anguage-side reason**i**ng **F**acilitation and **T**ransfer), a lightweight vector-intervention method that transfers reasoning capability from the base LLM to the VLM without retraining the backbone. LIFT defines **Reasoning Vectors** as answer-token hidden-state differences between a Reasoner path with an explicit reasoning trace and a Solver path without it, and injects these vectors into language-side activations of the target VLM. LIFT further supports learnable vector adaptation while keeping the VLM backbone frozen. We evaluate LIFT on two VLMs across six reasoning benchmarks, comparing Reasoning Vectors extracted from the base LLM and from the aligned VLM under matched protocols. Results show that LLM-derived vectors consistently outperform VLM-derived vectors, confirming that the base LLM is a more effective source for recovering reasoning. LIFT partially recovers degraded reasoning through lightweight language-side interventions. Further analyses show that Reasoning Vectors influence intermediate reasoning behavior rather than merely altering final answers.

Language models (LMs) raise an alluring alternative to vector-based retrieval: \emph{generating} a relevant answer, entirely based on an in-context corpus. In spite of this appeal, prior studies consider proprietary systems or the smaller-scale reranking task, leaving corpus-scale in-context retrieval largely unexplored. In this work, we present the first systematic study of in-context retrieval on two scales practical retrievers demand: \emph{million-token} corpora and \emph{length-generalization} far beyond training-time sizes. We first introduce BlockSearch, a 0.6B LCLM retriever whose architectural and training modifications improve over prior LM baselines and length-generalize up to $10\times$ beyond its training regime. However, its retrieval still collapses under more extreme extrapolation. We trace this failure to an \emph{attention dilution} effect: as the corpus grows, irrelevant documents dominate the softmax denominator, minimizing the normalized mass on the gold document even as its pre-softmax score stays high. Motivated by this analysis, we introduce \emph{length-aware} adjustments to the attention softmax and \emph{document-level sparse attention}, improving retrieval at million-token scale, better performance than the concurrent $7\times$-larger MSA system, and comparable performance to dense retrieval. Together, our results position in-context retrieval as a viable alternative to classical retrieval, emphasizing attention control under extreme context growth as a new challenge.

Large language models have demonstrated impressive retrieval-augmented capabilities. However, a crucial area remains underexplored: their ability to appropriately adapt responses to the certainty of the retrieved information. It is a limitation with real consequences in high-stakes domains like medicine and finance. We evaluate eight LLMs on their *context-certainty obedience*, measuring how well they adjust responses to match expressed context certainty. Our analysis reveals systematic limitations: LLMs struggle to recall prior knowledge after observing an uncertain context, misinterpret expressed certainties, and overtrust complex contexts. To address these, we propose an interaction strategy combining prior reminders, certainty recalibration, and context simplification. This approach reduces obedience errors by 25% on average, without modifying model weights, demonstrating the efficacy of interaction design in enhancing LLM reliability. Our contributions include a principled evaluation metric, empirical insights into LLMs' uncertainty handling, and a portable strategy to improve context-certainty obedience across diverse LLMs.

Production training of large models alternates between model edits and hybrid-parallel strategy planning. Fixed-configuration planners solve each revision from scratch, while unvalidated reuse across edits risks returning a different plan than full re-search would. We introduce a dependency-keyed per-layer primitive interface for re-planning: closed-form layer primitives expose communication, memory, and time to the solver, and the same templates expose the input dependencies needed for reuse-equivalent re-planning. CONCORD attaches dependency sets and value signatures to primitive evaluations, enabling dependency-keyed re-planning to reuse only signature-valid primitive-evaluation work while preserving the same logical candidate set and feasibility predicate as cache-disabled full search. On production MoE strategy-quality runs—including a 4096-device MoE-438B deployment and a 32k-token long-context point—CONCORD's single-call plan achieves 1.14–1.24× training step-time speedup over an expert-tuned Megatron-style baseline (1.70× on the long-context point). Across a 9-fixture sweep at 256–4096 cards, CONCORD's warm re-planning achieves median 2.74× planner wall-clock speedup over cache-disabled full search across 54 in-dispatch cells. Full-search equivalence holds throughout: candidate and feasibility counts match in every cell, the top-1 plan matches in 54/54 cells, and stale-hit mismatches (cached value disagreeing with cold recomputation) are 0/162 over paired trials.

Structure prediction models are increasingly deployed as zero-shot digital screens in antibody design, operating under the assumption that high folding confidence implies true binding. However, rigorous validation of this premise has been limited by small test sets, narrow target diversity, and the risk of data leakage from training-adjacent structures. Here we introduce POISK: Patent-extracted Organization of Immunoglobulin Sequence Knowledge, a large-scale dataset derived from the US patent literature, comprising 89074 antibody-antigen records extracted from 10110 patents using an LLM-based pipeline. From this resource, we curate a balanced benchmark set of 22494 complexes: 11247 patent-validated binders without a known resolved structure paired with putative non-binders constructed from sequence-similar human repertoire antibodies. Using this dataset, we benchmark six leading structure prediction models (AlphaFold-Multimer, Boltz-2, Intellifold v2, OpenFold3, Protenix-v1, and Protenix-v2) on their ability to discriminate binders from decoys. The best-performing model, Protenix-v2, achieves 87% pairwise accuracy on the most confident half of predictions and correctly identifies the true binder from a pool of 50 candidates in 32% of trials; this represents a substantial enrichment over random selection, yet remains far from reliable single-candidate identification. Structural consensus across models also discriminates binders from decoys, with pairwise accuracy reaching 73.1% when three or more models converge on a pose DockQ $\geq$ 0.23, covering around 54.2% of pairs). Yet epitope localization remains poor: the best model contacts annotated epitope residues in only 45% of cases, indicating that high confidence reflects learned statistical associations rather than accurate physical binding modes. Our results suggest that current folding models provide useful but limited signal for antibody screening and should be complemented by orthogonal methods. We release POISK as a community resource for future benchmarking and model training.

Training instabilities such as loss spikes are frequently the result of stochastic gradient noise. Because of rare expressions in language training data, and multiple layer composition, the noise impact is heavy-tailed and survives mini-batch averaging. Gradient clipping is designed to address this where vector-norm clipping ignores matrix structure in weight updates, while spectral normalization (e.g. Muon) respects structure at additional cost. We show an entry-wise \emph{heavy-tailed noise} appears similar to real stochastic gradient noise. Furthermore, through a first-order perturbation analysis, we identify a \emph{localization} property under which a simple entry-wise method will give spectral normalization. Exploiting this, we derive a tractable surrogate for the Bayes-optimal entry-wise estimator under a Gaussian signal prior. We establish $O(\epsilon^{-4})$ convergence guarantee under Cauchy-contaminated noise. Empirically, we find that smooth shrinkage improves Adam on NanoGPT pretraining, saving ${\sim}$8% of training tokens. We further find that applying the entry-wise clipping before spectral normalization yields a ${\sim}$2% token saving on top of Muon.

We present an encoding method that maps each input token to a randomized sequence of binary values (0s and 1s). By operating at the finest representational granularity and applying compression, this approach reduces the effective vocabulary to just two symbols, yielding a compact, task-agnostic representation. Across eleven main tasks spanning six modalities (human text, genetic/clinical data, programming code, system logs, biological signals, and tabular medical data), it achieves up to two orders of magnitude stronger privacy guarantees under six gradient-based attack settings compared to seven encoder baselines and four defense baselines, including up to 549× improvement on FILM and 62× on TAG. At the same time, it reduces embedding model size by up to 98%, while maintaining competitive accuracy and computational efficiency. It also improves multilingual generalization, achieving up to a 4× increase in accuracy in low-resource settings.