NeurIPS 2026 Papers

Larger language models solve harder problems, but do they also produce proportionally better explanations? We study in-context explanation transfer across 40 open-weight models (360M--72B) and 5 receivers (360M--7B), using problems drawn from 13 benchmark sources and evaluated with domain-specific metrics. Our central finding is an \emph{articulability ceiling}: under equal token budgets, mid-range models are better teachers than frontier models. Transfer peaks at 170--220 words and then drops sharply beyond 300 words, falling below even the shortest explanations. Large models partly compensate through verbosity: longer explanations yield higher total transfer on average, but each additional word carries less pedagogical value, so a 360M model delivers more transfer value per word than a 72B model. We also find that RLHF improves pedagogical clarity: instruct models score better on LLM-judge evaluation than base models despite lower lexical overlap, and teacher correctness dominates all other factors. Code and logs are released with the paper.

Convolutional neural networks trained on ImageNet are known to rely heavily on local high-frequency texture, an inductive bias that translates into fragile robustness against distribution shifts in real-world environments. Event cameras, in contrast, record only changes in scene brightness and are therefore well suited to capturing contour information; however, due to the absence of diagnostic benchmarks in the event domain, the inductive bias that event-camera data instills in vision models has remained unexplored. In this work, we use knowledge distillation from the event domain to the RGB domain so as to exploit the rich evaluation toolkit available in the RGB domain and systematically dissect this inductive bias. Our experiments show that distillation from the event domain induces, in the RGB domain, notable color invariance, shape bias, and robustness to high-frequency noise. We identify the underlying mechanism as the model suppressing its dependence on high-frequency texture while simultaneously acquiring a strong dependence on edge-based object shape. This hypothesis is supported by changes in how color and spatial information are processed at the early layers, together with a "spectral trade-off" in which robustness to the absence of high-frequency components coexists with vulnerability to contamination of the relied-upon frequency bands and to disruption of geometric structure. We further show that this inductive bias differs markedly from existing robustification methods and that it functions as a strong prior for diverse downstream tasks that demand shape-based reasoning, such as medical imaging. The code and experimental configurations for reproducing our experiments are submitted alongside this paper as supplementary material.

Self-play can train language models without external supervision. However, existing methods require rule-checkable answers, leaving open-ended tasks dependent on curated prompts or frontier-model judges. We introduce SCOPE, a data-free self-play framework for open-ended tasks that co-evolves two policies: a Challenger that generates document-grounded tasks, and a Solver that answers them through multi-turn retrieval. A frozen copy of the initial model serves as the self-judge, which writes task-specific rubrics from the source document and grades Solver responses against them. Across three 7-8B instruction-tuned models (Qwen2.5, Qwen3, OLMo-3), SCOPE improves open-ended performance by up to +10.4 points on eight benchmarks and matches or exceeds GRPO_data trained on ~9K curated prompts. Although trained only on open-ended tasks, SCOPE also improves held-out short-form QA by up to +13.8 points on seven held-out benchmarks, surpassing GRPO_data on all three models. Ablations show that co-evolving the Challenger is necessary to keep tasks near the Solver's frontier, that gains arise from improvements in both retrieval and synthesis with the relative contribution varying by task, and that rubric generation quality is the bottleneck for self-judging.

Comics and manga tell stories through characters whose appearances change from panel to panel, yet readers recognise them hundreds of pages later without ever consulting a cast list. Character re-identification models rarely face the same challenge: existing evaluations give them a fixed set of identities before the story begins. **Re:Cognize** instead asks models to recognise characters as the story unfolds and learn who is who along the way. Across three manga corpora, we evaluate five Re-ID backbones and **MECHA**, a memory-augmented baseline we introduce, and find that the central difficulty lies somewhere unexpected. Recognising familiar characters is nearly solved. Knowing which matches to trust is not. When models build a gallery and grow it from their own predictions, their performance deteriorates, even though correct identity labels would improve it by over twenty points. This raises a simple question: when should a model trust its own match? We derive a single comparison that determines when gallery updates improve recognition and use it to design **VOUCH**, which updates its memory only when the page provides supporting evidence. Without fitting additional parameters, VOUCH recovers between one-third and two-thirds of the improvement available from perfect labels. RE:COGNIZE measures whether a model can recognise characters as it reads; VOUCH shows how to remember them reliably. We focus our methodological claims on identity maintenance for previously encountered characters, while separately evaluating new-character emergence as a diagnostic under a fixed reference rule to distinguish recognition from discovery.

Learning to collaborate with previously unseen partners is a fundamental generalization challenge, known as Ad Hoc Teamwork (AHT). Existing methods often adopt a two-stage pipeline: first, a fixed population of teammates is generated, and second, an AHT agent is trained to collaborate with them. This separation limits coverage of behaviors and ignores whether the generated teammates are informative for the AHT agent to learn from. On the other hand, AHT agents are typically trained under the assumption that the training teammate set is uncontrollable, despite the fact that its composition strongly influences generalization. This paper presents a unified framework for AHT by reformulating the problem as an open-ended learning process between an AHT agent and an adversarial teammate generator. We introduce ROTATE, a regret-driven, open-ended training algorithm that alternates between improving the AHT agent and generating teammates that probe its collaboration deficiencies. Experiments across Overcooked and Level-Based Foraging tasks demonstrate that ROTATE significantly outperforms baselines on an unseen set of teammates, establishing a new standard for robust, generalizable teamwork.

Remote sensing object detection often proceeds in a domain-incremental manner, where detectors continuously encounter new domains arising from changes in regions, resolutions, sensors, and modalities. This challenge is particularly severe when passive optical red-green-blue (RGB) imagery and active synthetic aperture radar (SAR) imagery coexist, because their distinct imaging mechanisms create large domain gaps. Existing methods mainly rely on feature alignment or regularization, but often overlook semantic associations across heterogeneous domains. The problem is more pronounced for DETR-like detectors, where decoder queries act as object-centric semantic slots and are easily disrupted by cross-domain adaptation. Under a query-as-resource view, we propose Q-cost for remote sensing domain-incremental object detection. At the feature level, Q-cost uses modality- and domain-aware prototypes to guide a two-level mixture-of-experts adapter and generate encoder prompts for domain-aware semantic aggregation. At the semantic level, it models decoder queries as limited resources with domain-dependent activity costs, and applies activity-aware gating to regulate query updates according to current and historical domain demands. Experiments on multiple remote sensing benchmarks show that Q-cost effectively balances new-domain adaptation and old-domain retention under severe domain and modality shifts. Our code is provided in the supplementary material.

Sparse object detection remains difficult when weak target evidence occupies only a few pixels and shares local statistics with surrounding clutter. This failure is not merely due to limited model capacity, but to a representation bottleneck: spatial domain features mix target edges, clutter textures, and scene context before they can be reliably separated. We propose PRISM, a wavelet domain detector that treats this problem as frequency separated representation learning followed by priority ordered feature interaction. PRISM decomposes each scene into directional high frequency evidence and global low frequency context, models the two streams with dedicated encoders, and fuses them through a learned content adaptive causal scan that lets likely target regions guide feature interaction before clutter dominated regions participate. Multi scale features are then reconstructed by inverse wavelet synthesis with adaptive gates and passed to a standard detection head. On low altitude UAV maritime benchmarks, including SeaDronesSee and AFO, PRISM achieves state-of-the-art performance with the largest gains on the smallest targets. Without architectural modification, it also transfers to aerial urban imagery on VisDrone, suggesting that priority ordered fusion of directional frequency evidence is a generalizable design principle for sparse visual signal detection.

Vision-Language Models (VLMs) exhibit remarkable zero-shot generalization, yet they often encode unwanted or hazardous stylistic domains such as idealized textbook diagrams in medical AI or cartoon vehicles in autonomous driving. Approximate Domain Unlearning (ADU) aims to selectively erase a model's recognition of a target visual domain while preserving accuracy on the remaining domains. However, existing ADU methods operate under a flawed closed-vocabulary assumption: they evaluate unlearning solely on the specific object classes seen during the unlearning fine-tuning phase. Consequently, these methods do not unlearn the domain itself; they merely overfit to seen class-domain pairs, leaving the domain easily recognizable for unseen classes and providing a false sense of removal. We argue that true domain erasure must be class-agnostic. To address this, we formalize Open-Vocabulary Domain Unlearning (OVDU), a rigorous protocol that mandates domain forgetting must transfer to held-out classes. To solve the OVDU challenge, we propose a surgical parameter-editing framework. First, a Fisher Information mask isolates domain-sensitive weights, mathematically protecting foundational zero-shot generalization. Second, our Targeted Manifold Scattering (TMS) objective uses preference-based mining to locally scatter the forget domain's stylistic geometry. Evaluated across PACS, OfficeHome, and DomainNet, our method vastly improves open-vocabulary generalization over existing baselines. Crucially, it delivers exceptional sample efficiency, outperforming peak 8-shot baseline results with only 4 shots.

Random Feature Models (RFMs) are attractive PDE surrogates because their linear-in-features form reduces training to regularized least squares. Standard RFMs, however, use one stationary spectrum over the whole domain, creating a bandwidth dilemma on heterogeneous PDE solutions: fine scales resolve sharp local gradients but oscillate in smooth regions, while coarse scales miss interface-dominated derivatives. We propose \textbf{LDD-RFM}, a learnable domain-decomposition framework whose local experts remain linear RFMs and whose nonlinear variables specify compact-support partition-of-unity (PoU) gates and local spectral scales. For fixed gates and spectra, expert coefficients are eliminated by ridge regression through variable projection, yielding a differentiable reduced objective with explicit linear-solve and partition-smoothness controls. We prove an $H^1$ error decomposition in which PoU sharpness enters through a product-rule term. Controlled regression and PDE experiments show that LDD-RFM reduces $H^1$ errors by $10\times$ over global and fixed-partition RFM baselines under matched feature budgets; exact-solver, timing, and robustness studies check that these gains are not artifacts of the linear solver or evaluation protocol.

Many real-world coding challenges are open-ended and admit no known optimal solution. Yet, recent progress in LLM coding has focused on well-defined tasks such as feature implementation, bug fixing, and competitive programming. Open-ended coding remains a weak spot for LLMs, largely because open-ended training problems are scarce and expensive to construct. Our goal is to synthesize open-ended coding problems at scale to train stronger LLM coders. We introduce FrontierSmith, an automated system for iteratively evolving open-ended problems from existing closed-ended coding tasks. Starting from competitive programming problems, FrontierSmith generates candidate open-ended variants by changing the problems’ goals, restricting outputs, and generalizing inputs. It then uses a quantitative idea divergence metric to select problems that elicit genuinely diverse approaches from different solvers. Agents then generate test cases and verifiers for the surviving candidates, and these problems then join the seed pool for the next round. On two open-ended coding benchmarks, training on our synthesized data yields substantial gains over the base models: Qwen3.5-9B improves by +8.82 score on FrontierCS and +306.36 (Elo-rating-based performance) on ALE-bench; Qwen3.5-27B improves by +12.12 and +309.12, respectively. Moreover, FrontierSmith-generated problems elicit long-horizon code-agent behavior comparable to human-curated open-ended tasks, causing agents to spend substantially more turns and tokens than on closed-ended problems.

We introduce EgoSurgHands, an egocentric 3D hand-pose dataset and benchmark for open-surgery training tasks captured with Meta Project Aria glasses on standard surgical training pads. To our knowledge, EgoSurgHands is the first public benchmark for surgical training that is simultaneously egocentric, sensor-grounded, and human-validated. EgoSurgHands contains 41,909 training hand-instance rows across 55 procedure recordings and 9,476 inter-annotator-agreement (IAA)-validated rows across 13 held-out procedure recordings from 4 wearers, 13 distinct tasks, 3 glove colors, and 3 stages of training. Each frame includes Aria intrinsics, SLAM extrinsics, MPS (Machine Perception Services) landmarks, per-joint confidence, timestamps, and eye gaze. Validation 3D keypoints are initialized from Aria MPS and independently corrected by four expert annotators under a multi-rater IAA workflow, yielding evaluation ground truth independent of any tested model's training signal. Using EgoSurgHands, we benchmark six state-of-the-art hand reconstructors under zero-shot transfer: HaMeR, WiLoR, WildHands, JointTransformer, SimpleHand, and MeshGraphormer. Ego-pretrained models consistently outperform non-egocentric baselines: WildHands improves over the non-ego HaMeR reference by 4.51 mm PA-MPJPE, while JointTransformer achieves the best zero-shot result at 14.95 mm PA-MPJPE. These results show that generalist egocentric pretraining transfers to surgical egocentric tasks, but also reveal a substantial remaining surgical-domain gap. As a lightweight diagnostic adaptation experiment, a 58K-parameter two-headed (pose + translation) residual corrector further reduces error on frozen backbones, reaching 10.40 mm PA-MPJPE and 75.8 mm absolute MPJPE on WildHands ($-$49.2\% and $-$85.8\% vs zero-shot). We release the dataset, benchmark code, checkpoints, and multi-rater annotation framework at [URL withheld for double-blind review].

Cross-domain open-vocabulary learning poses a unique and underexplored challenge, requiring models to generalize across both domain shifts and category shifts. To tackle this, we propose MAP: a parameter-efficient Multi-source domain-Adaptive Prompt tuning framework that leverages multiple labeled source domains to improve learning in novel, unlabeled target domains with unseen categories. MAP consists of two key components: Multi-Source Prompt Learning (MSPL) and Unsupervised Target Prompt Learning (UTPL). MSPL disentangles domain-invariant category semantics from domain-specific visual patterns by jointly learning shared and domain-aware prompts. UTPL enhances generalization in the unlabeled target domain by enforcing prediction consistency under text-guided style augmentations, introducing a novel entropy-minimization objective without relying on pseudo-labels. Together, these components enable effective alignment of visual and textual representations across both domains and categories. In addition, we present a theoretical analysis of the proposed prompts, examining their behavior through the lenses of fidelity and distinction. Extensive experiments on challenging CDOV benchmarks demonstrate that MAP achieves state-of-the-art performance with significantly fewer additional parameters.

Modern visual recognition systems increasingly rely on unlabeled images collected from diverse visual domains. A common goal in dataset construction is to avoid obvious marginal imbalance, such as skewed class totals or uneven domain sizes. However, marginal balance can be misleading in multi-domain data: even when per-class totals and per-domain sample sizes are balanced, sample allocations can be highly skewed across domain-class pairs. We call this hidden conditional skew Domain-Conditioned Class Imbalance (DCI), where class imbalance is revealed only after conditioning on domain context. For controlled evaluation, we fix class and domain marginals and vary only the joint domain--class allocation, making the effect of domain-conditioned imbalance directly measurable. To address this challenge, we propose HyperIDAC, a hypernetwork-based classifier for multi-domain imbalance. HyperIDAC infers domain priors from visual features and lightweight zero-shot predictions, then dynamically generates instance-specific classifier parameters and domain-conditioned class embeddings. This design yields domain-sensitive decision boundaries while leveraging CLIP semantics to support underrepresented domain--class pairs. Furthermore, a multi-level pseudo-labeling module combines teacher–student training with historical label tracking and a fallback for low-confidence samples, mitigating bias reinforcement in unlabeled settings. Extensive experiments on four multi-domain benchmarks under three regimes show that HyperIDAC substantially outperforms state-of-the-art methods, highlighting the value of domain-aware hypernetwork-based adaptation under DCI.

Chemical reaction datasets such as USPTO suffer from substantial incompleteness, frequently missing byproducts, co-reactants, and stoichiometric coefficients. This limits their applicability and reliability in downstream applications. Here, we introduce CompleteRXN, a large-scale supervised benchmark for reaction completion under realistic missing-data conditions. We construct a dataset of aligned incomplete and atom-balanced reactions by mapping USPTO records to curated mechanistic reactions. We evaluate representative baselines, including a novel encoder-decoder reaction completion model with constrained decoding, the Constrained Reaction Balancer (CRB), and a recent algorithmic method, SynRBL. On our CompleteRXN benchmark, the CRB achieves high performance across splits of increasing difficulty, reaching 99.20% equivalence accuracy on the random split and 91.12% on the extreme out-of-distribution split. SynRBL produces many balanced and chemically plausible completions, but with lower accuracy on the benchmark test splits. Across all methods, performance degrades with increasing incompleteness. We observe a substantial drop when evaluating on reactions outside the benchmark (full uncurated USPTO), highlighting the gap between benchmark performance and practical robustness and motivating future work.

Recently, reinforcement learning (RL) has been widely applied during post-training for diffusion large language models (dLLMs) to enhance reasoning with block-wise semi-autoregressive generation. Block size has therefore become a vital factor in dLLMs, since it determines the parallel decoding granularity and affects the rollout trajectories during RL optimisation, e.g., GRPO. Instead of investigating the effect of block size during inference on individual domains, this paper studies block size from a domain conflict perspective for dLLM RL post-training in multi-domain scenarios. The main contributions are: (1) a formulation of domain block size conflict in multi-domain RL for dLLMs, which will largely affect the post-training effectiveness for rollout-based RL methods; (2) a novel dataset, \textbf{Block-R1-41K} is constructed with a best-improved training block size for each sample, which also induces a Block Size Conflict Score to quantitatively measure the domain conflict; (3) a new benchmark, Block-R1, for flexible RL post-training for dLLMs in both single and cross domain; and (4) a simple yet powerful cross-domain post-training method with sample-level best-improved training block sizes. Extensive experiments on 13 distinct datasets, 7 latest RL algorithms, and various different dLLM backbones are covered in Block-R1. The benchmark is open-sourced at https://anonymous.4open.science/r/Block-R1-2026, with the dataset released at https://huggingface.co/datasets/dLLM-R1/Block-R1-41K.

Sharpness-Aware Minimization (SAM) is widely used in domain generalization to promote flat minima that generalize to unseen distributions. Existing SAM-based methods typically penalize the sharpness of the *average* loss across domains. We show that this prevailing formulation—termed AvSAM—has a fundamental geometric limitation: it can favor solutions with misaligned domain-wise dominant Hessian eigenvectors. As a result, such solutions may appear flat on average while remaining sharp within individual domains. To address this limitation, we propose an alternative objective that penalizes the average *domain-wise* sharpness. We show that this objective decomposes into the AvSAM objective plus a non-negative residual term, which we call the *Cancellation Gap*. This gap is minimized if and only if the domain-wise dominant eigenvectors are aligned up to sign, indicating that it mitigates AvSAM's tendency to favor misaligned solutions. However, we further uncover a degeneracy: the Cancellation Gap can become small without improving alignment when sharpness is highly imbalanced across domains. To mitigate this failure mode, we propose Cancellation Gap Minimization (CGM), which augments the objective with a squared coefficient-of-variation ($\mathrm{CV}^2$) regularizer to discourage sharpness imbalance. Experiments show that CGM achieves the best average performance across standard domain generalization benchmarks among strong baselines. Diagnostic analyses further confirm reduced domain-wise sharpness and improved alignment of dominant Hessian eigenvectors.

Language models (LMs) obtain much of their capability from the diversity of their training data. This is possible because neural architectures like transformers are remarkably flexible and can handle a variety of domains such as natural language, code, and mathematics. This raises the question: are transformers optimal for any specific domain? In this position paper, we present empirical evidence that the inductive biases of transformer-based LLMs should be re-examined, since, while broadly useful, they are often suboptimal on subsets of their training data. **Methods.** To support this position, we use a method that learns new activation functions within a transformer as a tool to explore the space of inductive biases. We train the resulting modified architectures on various data subsets. A large-scale evaluation shows that vanilla transformers are relatively easy to improve, but mostly on a per-domain basis. For example, an architecture optimized for a natural language dataset like FineWeb performs well on another such dataset like TinyStories, whereas architectures optimized for mathematics perform poorly on natural language. We also identify the mechanisms responsible for these improvements. **Implications.** These results call for re-evaluating core design choices in language models. We do not advocate a return to handcrafted, domain-specific models. Instead, we emphasize that different domains can benefit from different learning mechanisms. This will require new designs that preserve cross-domain interactions, analogous to how reasoning in the brain arises from the interaction of multiple specialized networks. We argue that this perspective may be key to improving reliability and data efficiency.

Calibration measures whether a model's predicted confidence aligns with its empirical accuracy, and is central to the reliable deployment of large language models (LLMs) in high-stakes domains such as medicine and law. While much recent work focuses on *improving* LLM calibration, the equally important question of how to *evaluate* it in realistic settings remains underdeveloped. Open-ended question answering (QA), the most common deployment setting for modern LLMs, is where existing evaluation methods fall short: logit-based metrics need restricted output formats and internal probabilities; verbalized confidence is self-reported and often overconfident; and sampling-based methods rely on task-specific extraction rules without a clear finite-sample target. We introduce **Sem-ECE** (**Sem**antic-Sampling **E**xpected **C**alibration **E**rror), a calibration evaluation framework for open-ended QA that samples answers from the model, groups them into semantic classes, and uses the resulting frequencies as confidence. We study two estimators within this framework: Sem₁-ECE, the same-sample self-consistency score, and Sem₂-ECE, a held-out variant that separates answer selection from confidence evaluation. We prove both are asymptotically unbiased, and further show that they agree on easy questions but diverge on hard ones with Sem₂ achieving strictly smaller calibration error, so their gap also serves as a diagnostic for question difficulty. Experiments on three open-ended QA benchmarks across five leading commercial LLMs match our theoretical predictions and show that Sem-ECE outperforms verbalized confidence and existing sampling-based methods, while complementing logit-based evaluation when internal probabilities are unavailable.