NeurIPS 2026 Papers

We introduce AIRA-Compose — a framework that enables AI research agents to autonomously explore and discover novel neural architectures. AIRA-Compose leverages the previously-proposed Composer abstraction to define a combinatorial design space over fundamental model primitives (Attention, MLP, and Mamba), then uses a guided search policy to efficiently navigate this space for novel neural architecture design. Our approach operates in two stages: (1) agents iteratively designs and evaluates candidate architectures at the million-parameter scale, then (2) top-performing candidates are scaled to billion parameters for pre-training. Overall, AIRA-Compose discovers 14 novel architectures spanning two families — AIRAformers (Transformer-based) and AIRAhybrids (Transformer-Mamba Hybrid-based). When pre-trained at 1B scale under a fixed token budget, agent-discovered top-performing architectures consistently outperform both Llama 3.2 and Composer-found alternatives. On downstream tasks, AIRAformer-D and AIRAhybrid-D improve accuracy by 2.4% and 3.8% over Llama 3.2, respectively. AIRA-Compose also finds novel model architectures that achieve steeper, more efficient compute-optimal scaling frontiers. AIRAformer-C scales 54% and 71% faster than Llama 3.2 and the best Composer-found Transformer, while AIRAhybrid-C scales 23% and 37% faster than the modified Nemotron-2 and the best Composer-found hybrid, respectively. For the first time, AIRA-Compose demonstrates that AI research agents can discover hybrid architectures that surpass hand-designed baselines on the accuracy–compute frontier. It establishes a flexible paradigm for LLMs to discover next generation foundation models, marking a step towards recursive self-improvement.

Existing research has identified three structural performance bottlenecks in AI research agents: (1) synchronous single-GPU execution constrains sample throughput, limiting the benefit of search; (2) a generalization gap where validation-based selection causes overfitting and performance to degrade over extended search horizons; and (3) the limited capability of fixed, single-turn LLM operators imposes a ceiling on search performance. We introduce AIRA$_2$, which addresses these bottlenecks through three architectural choices: an asynchronous multi-GPU worker pool that increases experiment throughput linearly; a Hidden Consistent Evaluation protocol that delivers a reliable evaluation signal; and ReAct agents that dynamically scope their actions and debug interactively. On MLE-bench-30, AIRA$_2$ achieves a mean Percentile Rank of 81.5% at 24 hours and 83.1% at 72 hours, outperforming the strongest baseline, which achieves 72.7%. On AIRS-Bench, AIRA$_2$ exceeds human state-of-the-art on 6 out of 20 diverse research tasks. Ablations confirm that each architectural component is necessary, that performance follows a predictable scaling law that transfers across LLM backbones, and that the "overfitting" reported in prior work was driven by evaluation noise rather than true data memorization.

Image-text offset alignment, which refers to the consistency between visual transformation directions and their corresponding textual transformation directions, has become an important relational property in multimodal contrastive representation space (e.g., CLIP or SigLIP). This property is widely utilized in text-guided generative modeling tasks, such as generative model domain adaptation and text-guided Image-text offset alignment, which refers to the consistency between visual transformation directions and their corresponding textual transformation directions, has become an important relational property in multimodal contrastive representation spaces such as CLIP and SigLIP. This property plays a central role in text-guided generative modeling tasks, including generative model domain adaptation and text-guided image editing, where textual offsets are used to guide visual transformations. These methods fundamentally rely on the assumption that image and text offsets are well aligned, such that textual transformation directions can provide reliable guidance for corresponding visual transformations. However, the validity of this assumption has not been systematically examined. **In this work**, we question this foundational assumption by conducting a comprehensive empirical analysis of image-text offset alignment in multimodal contrastive representation space. Our findings reveal not only noticeable offset misalignment but also a meaningful positive correlation between image-text offset misalignment and semantic concept distance across six large datasets and eight contrastive vision-language models. Based on this discovery, we propose Adaptation with Iterative Refinement (AIR), a method that iteratively refines text offsets through anchor sampling and our proposed concept description learning to reduce image-text offset misalignment and improve guidance accuracy. Comprehensive experiments on zero-shot generative model domain adaptation and text-guided image editing, including qualitative, quantitative, and user studies, consistently show that AIR enables state-of-the-art performance in these tasks. **Code and additional experiments are available in the supplementary material.**

Multi-Agent Debate (MAD) improves reasoning on multimodal tasks, but its accuracy and token efficiency depend on how agents interact during debate. A largely overlooked design choice is which interaction modalities agents use to exchange evidence. Existing methods rely on fixed text- or graph-based interaction, which fails to convey fine-grained multimodal evidence, degrading accuracy while inflating token cost due to verbose descriptions. To address this, we propose Adaptive Interaction in Multi-Agent Debate (AIM), an adaptive framework that dynamically selects the best combination of interaction modalities for each instance. Beyond text and graph, AIM introduces task-dependent modality interaction, where agents exchange targeted regions of interest from the task-specific multimodal input, such as image regions, audio segments, and spatio-temporal clips. To determine which combination is the best for each instance, AIM first generates a structured baseline response, extracts interpretable routing features, and finally uses a lightweight router to select a modality combination that avoids both under-fusion (i.e., agents miss decisive evidence) and over-fusion (i.e., redundant modalities inflate token cost) risks. Across eight multimodal question answering (QA) benchmarks of Visual-QA, Audio-QA, and Video-QA, AIM achieves the highest accuracy on all datasets, improving upon the best single-modality baseline by up to 8.9% while reducing token cost by up to 53.4% relative to the best fixed-modality baseline.

Generative artificial intelligence (AI) is increasingly integrated into the online platforms where humans exchange opinions; large language models (LLMs) now polish users' posts on LinkedIn and provide context for content shared on X. While prior work has shown that AI can express biased opinions and shape individuals' opinions during human-AI interactions, less attention has been paid to its influence on collective opinion formation when mediating human-to-human communication. We address this gap via a combination of empirical and theoretical analyses. We show empirically that LLMs from multiple popular families introduce directional biases when instructed to edit human-written texts on contested topics, for example, nudging texts in favor of gun control and against atheism. Building on this observation, we introduce a mathematical model of opinion dynamics in which an AI system sits between users on a social network, transforming the opinions they express and perceive. By analytically characterizing the equilibrium of this model and performing simulations on real social network data, we show that biases introduced by AI in human-to-human communication can be amplified through the network and shift collective opinion in their direction. In light of these findings, we investigate whether such biases are controllable by online platforms. We audit the "Explain this post" feature on X and find evidence of pro-life bias in Grok's outputs on abortion-related content, which we trace back to specific design choices.

When does an AI assistant improve human decision-making, and when does it hurt? We consider the setting in which a human worker makes a binary decision by observing her own signal and receiving a binary or probabilistic recommendation from an AI. We develop a behavioral model which considers both her belief regarding the overlap of her signal with the AI as well as the trust she has in the AI when considering its recommendation. The model yields a phase diagram in AI capability and signal overlap, partitioning the parameter space into complementarity, impairment, and automation regions. A worker who neglects signal overlap (correlation neglect) can experience performance loss when incorporating AI into her decision and is eventually dominated by AI alone as AI capability grows, with the complementarity range narrowing as overlap rises. Trust calibration shifts region boundaries: undertrust shrinks the impairment region but expands automation, while overtrust expands impairment but extends complementarity to be preferred at higher AI capability levels. Applying the framework to ChatBench (643 workers, GPT-4o and Llama-3.1-8b), AI assistance raises accuracy by 21 percentage points, but workers undertrust the AI by roughly a factor of four. The model yields design insights linking AI deployment choices, worker behavioral parameters, and outcome quality.

We argue that in multiple areas of AI for drug discovery (AIDD), deep learning models are not learning the meaningful biological or chemical features they were hypothesised to capture, but instead learn non-generalisable features, for example, from dataset biases. To address this, we propose the systematic use of misaligned baselines. We define misaligned baselines as models that rely on signals purposely misaligned with the intended learning objective. Rather than modelling the underlying biology or chemistry directly, these baselines draw on reductionist sources of information, heavily perturbed input features, or heuristic models. Competitive performance of such baselines reveals when models are not learning the biologically meaningful representations they were designed to learn. By examining case studies across multiple branches of AIDD, we demonstrate that misaligned baselines have consistently exposed such failure modes and crucially informed the evaluation and improvement of these models. We argue that adopting them as standard practice will help to ensure that progress in AIDD reflects genuine advances rather than artefacts of data-generating processes or evaluation design.

Capability evaluations aim to surface dangerous model behaviors before deployment, but their reliability depends on the assumption that hidden capabilities can be elicited. We show this assumption does not hold under adversarial conditions by constructing backdoor attacks that embed encrypted neural networks within host model weights, decrypting and executing them only upon receiving a secret trigger. Using digital lockers, these hidden circuits remain provably hard to elicit and interpret under standard cryptographic hardness assumptions, even given full access to model weights, extending prior work from hiding fixed strings to arbitrary computations. We implement our constructions in PyTorch and empirically validate resistance to supervised fine-tuning on a small chemical reaction prediction task - a proxy for CBRN-relevant capabilities. Our results reveal a limit of capability audits. We release our models to support the development of stronger defenses.

AI models deployed in critical domains, such as AI safety research, may subtly sabotage our efforts due to misalignment. _Sandbagging_ is a form of sabotage in which an AI intentionally underperforms on the task given to it. It is particularly pernicious on _fuzzy_ tasks, i.e. tasks which are hard to grade or require intuition. To understand sandbagging on fuzzy tasks, we introduce a novel AI control framework that considers sandbagging as an adversarial game between a blue team and a red team. The blue team uses a weak trusted model to construct a weak score against which they would train a strong, potentially sandbagging model to remove the sandbagging if it were present. The red team then tries to find model behaviors that are rated highly by the weak score, and thus might not be trained out, but actually correspond to poor performance. We test our framework on the task of writing experimental proposals for research questions from recent ML papers. We use a language model with access to the original paper as a proxy "ground-truth" scorer. Our red team discovers sandbagging behaviors using multi-objective evolutionary prompt optimization. We show that Opus 4.6 can write proposals that are worse according to the ground truth proxy than those of GPT-OSS-20B, while the weak scorer rates them as highly as the best proposals from Opus 4.6. To mitigate the threat of sandbagging, we propose an adversarial optimization algorithm for the blue team that discovers more robust prompts for the weak model. This algorithm produces a blue team prompt that our red team optimization fails to exploit.

There are now thousands of measurement instruments used to evaluate language models, and the resulting scores are routinely used to support consequential claims about capabilities and risks. Yet, the evidence linking benchmark scores to constructs (i.e., abstract concepts such as reasoning ability or safety) is often implicit, incomplete, or absent. We introduce AI Construct Lexis, a versioned ontology and interactive artifact that extracts and organizes relationships among constructs, behaviors, and measurement instruments as claimed in the literature for LLM evaluation. Drawing on over 8,337 published papers from selected top AI venues, we use language models to extract constructs, definitions, behavioral indicators, and measurement instruments, yielding 977 distinct constructs, 1,694 measurement instruments, 830 behavioral indicators, and 5,900 relationships. The Lexis captures relationships claimed in the published literature, not necessarily validated relationships; valid score interpretation still requires empirical evidence that the instruments capture the constructs they are used to support. By exposing the field’s implicit measurement assumptions, the Lexis enables their inspection, testing, falsification, and refinement into evidence-based relationships. On expert-annotated samples, the extraction pipeline achieves high agreement on extracted concepts while revealing substantial ambiguity in the boundaries of constructs and behaviors. By making these relationships explicit, AI Construct Lexis provides infrastructure for a more rigorous and cumulative science of AI evaluation.

Representation alignment has recently emerged as an effective paradigm for accelerating Diffusion Transformer training. Despite their success, existing alignment methods typically impose a fixed supervision target or a fixed alignment granularity throughout the entire denoising trajectory, whether the guidance is provided by external vision encoders, internal self-representations, or VAE-derived features. We argue that such timestep-agnostic alignment is suboptimal because the useful granularity of representation supervision changes systematically with the signal-to-noise ratio. In high-noise regimes, diffusion models benefit more from coarse semantic and layout-level anchoring, whereas in low-noise regimes, the training signal should emphasize spatially detailed and structurally faithful refinement. This non-stationary alignment behavior creates a representational mismatch for static single-level supervisors. To address this issue, we propose Adaptive Hierarchical Prior Alignment (AHPA), a lightweight alignment framework that exploits the hierarchical representations naturally embedded in the frozen VAE encoder. Instead of using only a single compressed latent as the alignment target, AHPA extracts multi-level VAE features that provide complementary priors ranging from local geometry and spatial topology to coarse semantic layout. A timestep-conditioned Dynamic Router adaptively selects and weights these hierarchical priors along the denoising trajectory, thereby synchronizing the alignment granularity with the model's evolving training needs. Extensive experiments show that AHPA improves convergence and generation quality over baselines and incurs no additional inference cost while avoiding external encoder supervision during training.

In this paper, we consider mixed-motive games (MMGs) which model various real-world multi-agent scenarios where self-interested agents must both cooperate and compete. We identify an issue with existing gifting methods, where agents become overly dependent on incoming gifts and fail to learn reward-yielding behaviors, a phenomenon we call *gift over-reliance*. To address this, we propose a refusal-augmented gifting framework that balances cooperation and competition in MMGs. Our approach introduces an adaptive refusal mechanism that allows agents to selectively reject a portion of received gifts, thereby mitigating gift over-reliance and preventing the agents from being exploited by other agents. This mechanism is gifting-method-agnostic, serving as a plug-in for existing schemes while enabling decentralized learning. We demonstrate that across diverse MMG environments and gifting schemes, our mechanism consistently reduces gift over-reliance, prevents lock-in to sacrificing policies, and improves $\alpha$-fairness without necessitating major changes to existing training pipelines.

Optimizing nonlinear preferences in multi-objective reinforcement learning (MORL) is essential for capturing complex trade-offs like risk aversion or fairness. However, such non-linearity has historically bifurcated nonlinear MORL objectives into two distinct paradigms: Scalarized Expected Return (SER) and Expected Scalarized Return (ESR). While SER requires global-level optimization and ESR requires non-Markovian policies, leading to fragmented optimization strategies, we bridge this divide through the Aggregation–Expectation–Transformation (AET) framework. By unifying both criteria through a tripartite decomposition of scalarization, AET provides a principled foundation for general nonlinear MORL. Building on this framework, we propose AETDICE, a tractable offline RL algorithm for AET objectives. By utilizing DICE-style density-ratio estimation in an augmented state space, AETDICE enables sample-based optimization from static datasets. Our framework resolves long-standing barriers and captures respective trade-offs induced by AET framework, which existing methods fail to address.

Atomic electron tomography (AET) aims to recover the three-dimensional arrangement, chemical identity, and temporal evolution of individual atoms. Yet most reconstruction methods are still evaluated primarily by voxel-level image similarity, such as PSNR, SSIM, or Fourier shell correlation. In this work, we show that this evaluation practice can select scientifically incorrect reconstructions: a volume that looks accurate can contain atoms that are shifted, merged, missed, hallucinated, chemically mislabeled, or assigned inconsistent identities through time. We introduce AET-Bench, a dataset and benchmark for atom-first evaluation of atomic electron tomography. AET-Bench contains 755 samples across a six-tier realism ladder, spanning clean analytical simulations, binary alloys, 4D atomic dynamics, multislice scattering, real experimental tilt series, and Brownian Pt nanocrystal trajectories. It defines four complementary tasks and evaluates 14 core method families with 21 logged method entries under a unified protocol. AET-Bench reveals a central failure mode hidden by voxel-only evaluation. On the AET-Sim-Mono tier, high-SSIM neural and Gaussian reconstructions can recover substantially fewer atoms than classical or sparsity-regularized methods: for example, 4D-GS achieves strong voxel similarity but only 43.9\% atom recall, while FISTA-TV reaches 95.1\% atom recall despite much lower SSIM. This inversion persists across atom-matching thresholds, showing that it is not an artifact of a single cutoff. On more realistic multislice and real tiers, atom recovery further degrades, exposing a physics gap between visually plausible volumes and reference atomic models. For 4D reconstruction, we find that smooth temporal density fields do not guarantee persistent atom identities. AET-Bench establishes atom-level fidelity as a necessary evaluation axis for AET. Our results suggest that future AET methods should not be judged only by how reconstructions look, but by whether they recover the atoms, species, and trajectories that scientists actually use.

Current world models for explicit world simulation typically follow two distinct pathways: (1) video generative models (e.g., Sora), which simulate realistic appearances but fail to preserve 3D consistency; and (2) 3D generative models (e.g., Marble), which generate geometrically consistent scenes but often yield static, less realistic observations. We present AEON, a unified framework that bridges video-based and 3D-based world models and combines the benefits of both. The key insight of AEON lies in a generative model that operates within the video latent space while producing renderable 4D graphics primitives—specifically, 4D Gaussian Splatting (4DGS)—as the world modeler. We train a 4D reconstruction model to predict both 4DGS and camera poses from monocular videos, serving as our initialized 4D decoder. Furthermore, we design a domain adapter that translates generative video latents into reconstructive 4D latents through a distillation process on the video VAE. AEON naturally supports video-based world modeling by rendering 4DGS at predicted camera trajectories, and constructs a persistent 4D environment with scene dynamics for realistic world simulation. Consequently, AEON empowers a diverse range of world modeling applications and achieves state-of-the-art performance on standard benchmarks.

Reinforcement learning (RL) has substantially improved the ability of large language model (LLM) agents to interact with environments and solve multi-turn tasks. However, effective agentic RL remains challenging: sparse outcome-only rewards provide limited guidance for assigning credit to individual steps within long interaction trajectories. Existing approaches often introduce dense intermediate supervision, such as process reward models or auxiliary self-supervised signals, which increases supervision and tuning complexity and may limit generalization across tasks and domains. We present AEM, a supervision-free credit assignment method that adaptively modulates entropy dynamics during RL training to improve the exploration-exploitation trade-off. Since in agentic RL the environment is typically affected by a complete response, rather than an individual token, our analysis lifts entropy dynamics from the token level to the response level, aligning uncertainty estimation with the effective action granularity of LLM agents and reducing sensitivity to token-level sampling noise. We further show that entropy drift under natural-gradient updates is governed by the interaction between the sampled-response advantage and its relative surprisal. Motivated by this result, AEM derives a practical response-level uncertainty proxy and uses it to rescale advantages, leveraging the evolving balance between positive and negative samples to naturally transition from exploration to exploitation. Extensive experiments on ALFWorld, WebShop, and SWE-bench-Verified with models ranging from 1.5B to 32B demonstrate that AEM consistently improves strong RL baselines, including a +1.4\% gain when integrated into a state-of-the-art software-engineering RL training framework.

Document QA agents do not fail only at the final answer. They can also write unsupported claims into public working memory, and those writes can affect later retrieval and synthesis. We introduce ADMIT (Auditable Decoupled Memory-write admission for Intermediate Traces), a low-budget admission controller for public OCR/text evidence packs. ADMIT applies a deterministic substring-support check $S_{\text{str}}(\hat{y}, B) > 0$ to each public commit and allows one bounded RepairRead. If repair still fails, ADMIT uses a memory-fail-closed / output-fail-open rule: it writes a $\bot$ (NO_COMMIT) sentinel to memory while the final-answer surface may still fall back to the baseline answer. On held-out M3DocVQA, a retrained final-row router closes the answer-surface gap; ADMIT's distinguishing evidence is therefore memory-surface contamination, not final-answer accuracy. The lexical invariant guarantees zero-substring accepted intermediate contamination by construction; the contribution is therefore the public memory-write boundary as a low-cost control surface with decoupled fail actions, not the invariant itself. In a 7-arm dynamic stress diagnostic on three agents, ADMIT enforces the invariant and preserves raw EM at this operating point (zero contaminated trajectories on every agent and no detectable raw-EM loss relative to the fail-open ablation), whereas final-only baselines still leave $12$--$32%$ contamination. ADMIT enforces lexical auditability, not semantic truth: relation-level entailment, citation-aware faithfulness, and visual-region grounding are not claimed, and lower-density VisDoMSlide and text-only FRAMES are reported as boundary cases.

The KV cache is a major memory bottleneck in LLM inference, especially under large batch sizes and long contexts. While quantization is a promising solution, existing static group-wise quantization methods are constrained by a trade-off between metadata overhead and accuracy: increasing the group size reduces per-group metadata (e.g., zero-points, scaling factors, and outlier indices), but exacerbates quantization errors under the sinusoidal-like temporal drift of post-RoPE outlier channels; conversely, using smaller groups improves accuracy but incurs substantial metadata overhead. In this paper, we propose ADKV, a low-overhead adaptive delta quantization framework for KV cache in LLM inference, which leverages the temporal structure of the KV cache. ADKV tracks per-channel zero-points and scaling factors via online EMA, adapting to the sinusoidal-like temporal drift of post-RoPE outlier channels, thereby eliminating the rigidity of static group-wise quantization. This adaptive mechanism further enables delta coding to embed metadata directly into the quantized sequence, decoupling metadata overhead from context length. To minimize the MSE of attention outputs, we design an SGD-based calibration algorithm that models ADKV as an RNN. Experiments on generation and long-context benchmarks show that, while matching the accuracy of static group-wise quantization, ADKV reduces the metadata memory footprint by $\sim 12.8\times$. A custom fused ADKV-GEMV CUDA kernel optimized for memory-bound autoregressive generation achieves $\sim 1.96\times$ throughput over the FP16 baseline.

Continual pre-training (CPT) enables injecting domain-specific knowledge into large language models during the learning rate annealing phase. However, this compute-efficient paradigm forces practitioners to navigate a challenging optimization space: simultaneously choosing the injection fraction $\lambda$, the data mixture ratio $r$, and the re-warmup peak learning rate $\eta$. Because existing scaling laws have yet to comprehensively integrate and couple these interacting variables, they cannot reliably forecast the resulting trade-offs and final convergence outcomes across unseen injection configurations. To bridge this gap, we introduce the Annealing-phase Domain Injection Scaling Law (ADIS-Law), a parametric framework that explicitly models late-stage distribution shifts as physical perturbations applied to the base pre-training trajectory. ADIS-Law formalizes two opposing macroscopic behaviors: general-domain performance follows a linear forgetting-recovery process, whereas target-domain adaptation adheres to a power-law adaptation trend. Across data-budget and model-scale extrapolation, ADIS-Law predicts target-domain trajectories with an average Spearman correlation of $0.947$ and forecasts tail losses with relative error below $5\\%$. We further show that ADIS-Law can predict optimal injection strategies at extrapolated scales: it ranks target-scale candidate policies with a mean Spearman correlation of $0.936$. By replacing blind grid search over $(\lambda, r, \eta)$ with a single fitted law, ADIS-Law offers a data-driven approach for strategy selection in annealing-phase domain injection.