NeurIPS 2026 Papers

Vision-Language-Action (VLA) models promise a single language-conditioned policy adaptable to many manipulation behaviors, yet deployment additionally requires skill retention and long-horizon composition: a policy must retain previously acquired skills and reuse them inside composite tasks whose behavior changes across phases. The standard recipe absorbs every new task into one shared update, which provides neither a parameter-level locus for reusable skills nor a mechanism for invoking each skill along the trajectory. We observe that this missing structure is latent rather than absent: per-skill low-rank updates of a pretrained VLA occupy a robustly distinguishable, low-dimensional subspace, and a single composite-task LoRA concentrates roughly half of its Frobenius energy in this subspace, against a near-zero matched random-rotation control. Building on this geometry, we introduce PRISM, which represents reusable skills as parameter-level primitives: each skill family is stored as a dedicated low-rank action-expert adapter in a primitive bank, primitives are composed at inference by a per-chunk VLM-conditioned router that reads the prefix representation to track task progress, and the router is supervised by per-chunk labels from a deterministic, action-derived oracle. During composite training, only the vision encoder, the language LoRA, and the router are updated; every primitive is held frozen, so the per-skill action-expert parameters are preserved by construction. On RoboCasa365, PRISM preserves atomic-skill success through composite training and reaches 20% on long-horizon composite tasks, while a standard $\pi_{0.5}$-LoRA two-stage fine-tune instead loses roughly 25 atomic points and reaches only 11% composite. We additionally validate PRISM on an AgileX PIPER real-robot setup.

Large language model (LLM) agents increasingly operate over long and recurring external contexts, like document corpora and code repositories. Across invocations, existing approaches preserve either the agent's trajectory, passive access to raw material, or task-level strategies. None of them preserves what we argue is most needed for repeated same-context workloads: reusable orientation knowledge (e.g., what the context contains, how it is organized, and which entities, constants, and schemas have historically been useful) about the recurring context itself. We introduce **PEEK**, a system that caches and maintains this orientation knowledge as a context map: a small, constant-sized artifact in the agent's prompt that gives it a persistent peek into the external context. The map is maintained by a programmable cache policy with three modules: a Distiller that extracts transferable knowledge from inference-time signals, a Cartographer that translates it into structured edits, and a priority-based Evictor that enforces a fixed token budget. On long-context reasoning and information aggregation, PEEK improves over strong baselines by 6.3–34.0\% while using 93–145 fewer iterations and incurring 1.7–5.8$\times$ lower cost than the state-of-the-art prompt-learning framework, ACE. On context learning, PEEK improves solving rate and rubric accuracy by 6.0–14.0\% and 7.8–12.1\%, respectively, at 1.4$\times$ lower cost than ACE. These gains generalize across LMs and agent architectures, including OpenAI Codex, a production-grade coding agent. Together, these results show that a context map helps long-context LLM agents interact with recurring external contexts more accurately and efficiently.

We study contextual dynamic pricing with linear valuations and bounded-support agnostic noise, whose induced demand curve may be non-Lipschitz with arbitrary jumps and atoms. Such discontinuities break the cross-context interpolation arguments used by smooth-demand pricing algorithms, while the best previous method achieved only $\widetilde O(T^{3/4})$ regret. We propose Conservative-Markdown Redirect-UCB Pricing, a polynomial-time algorithm that combines randomized parameter estimation, conservative residual-grid probing, and confidence-based one-step redirection. Our algorithm achieves $\widetilde O(T^{2/3})$ optimal regret, matching the known lower bounds of Kleinberg and Leighton (2003) up to logarithmic factors and improving over the previous upper bound of Xu and Wang (2022). Under stochastic well-conditioned contexts, this closes the long-existing open regret gap in linear-valuation contextual pricing under agnostic non-Lipschitz noise distribution.

State Space Models (SSMs) have emerged as a compelling alternative to Transformers, enabling sequence modeling with constant memory and linear compute. While SSMs excel in certain settings like length generalization, they continue to lag behind Transformers on tasks that require in-context learning and precise retrieval, slowing their adoption for large-scale language modeling. In this work, we demonstrate that both the success and failure of SSMs in these domains can be explained by studying the role of the gating mechanism, a prevalent component in modern recurrent networks. Specifically, we show through theory and experiments that this gating mechanism causes SSMs to first learn an in-weights ``memorization'' solution, while delaying, or even preventing, convergence to a correct in-context learning solution. Importantly, this happens even in cases where there are no fundamental limitations due to the architecture or its memory capacity. On the other hand, we find that gating is often beneficial for improving generalization to long sequence lengths. Our results illuminate the crucial role of the gating mechanism in shaping both the training dynamics and generalization of SSMs, and provide a basis for understanding and improving linear-time models.

Multimodal agents that interact with users over days or months must remember not only what was said, but what was shown, heard, and played. Current memory benchmarks are predominantly text-only or cover at most two modalities. Those that do incorporate multimodal content still evaluate with aggregate scores that conflate retrieval and reasoning errors, and provide no mechanism to test whether performance degrades as context scales, leaving our understanding of agent memory fundamentally incomplete. We introduce OmniMemBench, to our knowledge the first long-term memory benchmark that spans image, audio, and video in multi-session dialogue. Its design is guided by three requirements for reliable evaluation. First, an anti-leakage mechanism filters questions answerable from text alone, reducing text-only solvability to a negligible level. Second, two LLM-judged retrieval metrics—Clue Coverage and Entry Relevance—operate on content rather than entry IDs, enabling unified retrieval diagnosis across RAG and memory paradigms. The gap between retrieval coverage and downstream QA scores further localizes failures to the memorization stage. Third, content-isolated distractor injection scales context from 128K to 1M tokens while holding evidence constant, so scalability can be measured without confounding difficulty. The benchmark covers 103 characters and 6,655 QAs across 8 task types and 4 context tiers. Evaluating representative long-context, RAG, and structured-memory methods, we find that no single paradigm dominates: long-context models lead overall, yet RAG surpasses them on targeted fact extraction. Further analysis shows that memory methods retrieve clues at coverage rates comparable to RAG but score substantially lower on QA, revealing that the bottleneck is not what is retrieved but what is lost during memorization. Caption-based processing irreversibly discards perceptual details; raw embeddings degrade retrieval accuracy; and scaling the memorization model yields negligible gains.

Multimodal Reinforcement learning with verifiable rewards (RLVR) algorithms assign the same advantage to every token in a generated sequence, providing no differential signal for tokens with varying degrees of dependence on visual evidence. A simple experiment confirms this issue: \textbf{scaling token advantages by uninformative random noise already outperforms uniform credit}, suggesting that uniform credit is empirically suboptimal. Beyond that, we observe RLVR tuning under uniform credit weakens visual grounding along three complementary axes: attention, gradient attribution, and functional dependence on image embeddings, \textbf{collectively indicating insufficient utilization of visual evidence}. To understand this degradation, we provide a theoretical analysis showing that, under language-prior dominance and regularity conditions, uniform credit can amplify a pretrained text-favored gradient imbalance, contributing to weaker visual attention. In response, we propose Hierarchical Context Credit Reassignment (\textbf{HiCCR}), which reassigns credit at two levels: a token-level weight amplifies the advantage for visually grounded tokens, and a trajectory-level weight upweights rollouts with stronger visual engagement. The entire mechanism \textbf{adds less than 3\% wall-clock overhead} and applies to mainstream RLVR algorithms without auxiliary models or additional data. On ten benchmarks spanning mathematical and general multimodal reasoning, HiCCR consistently improves upon its corresponding base algorithm and achieves state-of-the-art results among open-source models of comparable scale, while alleviating the weakening of visual grounding observed under RLVR training.

Many physical systems exhibit a low-dimensional structure that varies with external parameters: link lengths in a robot, forcing constants in a fluid, or Reynolds numbers in a flow shift the underlying manifold while preserving its intrinsic dimension. Constrained AutoEncoders (cAEs) learn such manifolds through an *idempotent* encoder-decoder projection, a property that unconstrained autoencoders cannot match and that is essential whenever the model is applied iteratively. However, the standard strategies for making a cAE context-dependent, namely concatenating the context to the input or affinely modulating hidden activations, break the encoder-decoder idempotency, sacrificing the projection guarantee precisely in the setting where it would be most valuable. To restore this guarantee under context variation, we developed the Neuromodulated Constrained Autoencoder (NcAE), which modulates the activation slope and bias of a cAE through a context-driven hyper-network. This paper presents the NcAE, its theoretical foundation, and its empirical validation. We prove that for every context, including contexts unseen at training time, the reconstruction map remains an idempotent projection, the topology of the learned manifold is invariant, and context perturbations induce smooth changes in the manifold. We evaluated our approach on a 16-DoF pendulum with context dependent coupling and the Lorenz96 system across a bifurcation. The NcAE matched or exceeded the best of six baselines on reconstruction, idempotency, and latent-geometry metrics, while being the only architecture that preserves geometric consistency by construction. The NcAE thereby provides a stable, geometry-preserving coordinate system across families of physical regimes.

Learning effective representations from battery operational history is essential for predictive health management and control-oriented applications. Existing approaches rely on response-dependent engineered features or short early-life windows, limiting generalisability and precluding use in prospective settings where future responses are unavailable. We propose $\texttt{NOCE-Net}$, a deep nested sequence model that learns control-enabling representations directly from raw cyclic operational data. Our data pipeline, $\texttt{re-BatteryData}$, standardises heterogeneous aging datasets into semantically equivalent Full Equivalent Cycles (FECs) defined by absolute charge throughput, enabling end-to-end learning across diverse operational regimes. $\texttt{NOCE-Net}$ combines multi-patch transformer encoders for intra-cycle dynamics with a GRU backbone for inter-cycle state evolution, operating on approximately $700$ batteries across ten public datasets spanning over $300$ operational profiles. In zero-shot evaluation on unseen battery chemistries and usage patterns, $\texttt{NOCE-Net}$ achieves $13.7$% MAPE in voltage response prediction, outperforming strong long-sequence baselines by over $43$% while using $40 \times$ fewer parameters. Shape and component diagnostics of the learned hidden states reveal that the GRU's principal direction of evolution achieves Spearman $|\rho| \geq 0.95$ with the capacity fade trajectory on average across all datasets with monotone degradation profiles, including zero-shot test batteries, providing evidence that the representations encode degradation dynamics transferable to downstream tasks beyond battery response modelling.

Recent advances in agent-based methods have demonstrated strong promise for issue localization, a critical prerequisite for software issue resolution. However, most existing agent-based methods rely on a single agent with a growing context, where long-context accumulation compresses the effective reasoning space. Meanwhile, the flat exploration structure hinders the balance between file-level breadth and function-level depth. To address these limitations, we propose NINJA, a hierarchical Navigator-INspector Joint Architecture for context-efficient issue localization. NINJA decomposes repository exploration into global navigation and local inspection: the navigator maintains the global search state, performs file-level search, and dispatches selected entry files, while inspectors independently conduct function-level exploration around the assigned files in separate contexts. Through multi-round interactions, inspectors return suspicious locations as feedback, and the navigator updates the global state to decide whether to continue exploration or finalize localization. This hierarchical design balances file-level breadth with function-level depth, while independent inspector contexts prevent local exploration traces from accumulating in a single growing context. To further strengthen both global coordination and local exploration, we introduce a two-stage agentic fine-tuning strategy. Extensive experiments across multiple benchmarks and LLM backbones show that NINJA consistently outperforms competitive baselines. Notably, after fine-tuning, Qwen3-Coder-30B-A3B-Instruct surpasses the strong closed-source Claude-Haiku-4.5 model. Our code is available at https://anonymous.4open.science/r/NINJA-67FB/.

End-to-end full-duplex speech models have brought open-source machine conversation close to human fluency, yet existing systems fall short of real-world deployment in two entangled respects: long-context conversational robustness and multi-party interaction capability. Realistic settings, including meetings, group lessons, family dinners, and social-robot reception, are inherently long-horizon and multi-party at the same time, requiring a single model to perceive, attribute, contextualize, and respond across multiple speakers over extended durations. Progress along these axes is bottlenecked by both data and evaluation. On the data side, open multi-party conversational speech corpora total only a few hundred hours and are not designed for codec-frame-level full-duplex modeling. On the evaluation side, existing long-audio benchmarks focus on passive listening, while existing speech-to-speech benchmarks remain dyadic and short. In this work, we extend the Moshi paradigm along long-horizon and multi-party axes simultaneously, in both English and Chinese, with three contributions. First, we release an open data engine and a 57.6 k-hour corpus for long, multi-party, bilingual full-duplex dialogue. The engine produces parallel-stream audio with controllable length, participant count, conversational dynamics (turn-taking, overlap, backchannels, interruption, addressee shifts, long-range co-reference), and language (English, Chinese, and intra-sentential code-switching), exceeding all prior open multi-party conversational speech corpora by more than an order of magnitude. Second, we introduce MultiTalkBench, the first benchmark to jointly evaluate long-form full-duplex dialogue with an average duration of 32.6 minutes, multi-party, and bilingual full-duplex dialogue, with explicit probes for long-range entity tracking, topic coherence, and addressee selection. Third, we train a bilingual Moshi-style full-duplex model on the released corpus that sustains coherent multi-party English-Chinese conversation over extended durations, substantially outperforming open-source baselines including Moshi, MiniCPM-o-4.5, and Qwen3-Omni-30B-A3B-Instruct.

Efficient long-sequence modeling remains a central challenge for large language models, as self-attention scales quadratically with sequence length. Mamba offers a linear-time alternative through selective state space recurrence, but its predominantly diagonal state transitions restrict explicit interactions among state dimensions. We propose Motif-Mamba, a structured state space model that augments Mamba with a motif-constrained low-rank recurrent pathway. Inspired by the dynamics of three-node network motifs, the proposed pathway projects hidden states into a compact dynamical subspace, imposes motif-guided interactions, and maps the resulting dynamics back to the original state space. This design enhances cross-dimensional communication while preserving the linear-time recurrent structure of Mamba. Experiments on long-sequence extrapolation, language modeling benchmarks, and brain--computer interface decoding show consistent improvements over Mamba backbones, suggesting that motif-guided low-rank dynamics provide an effective structural prior for long-range sequence modeling.

Medical multimodal large language models (MLLMs) have advanced image understanding and short-video analysis, but real clinical review often requires full-procedure video understanding. Unlike general long videos, medical procedures contain highly redundant anatomical views, while decisive evidence is temporally sparse, spatially subtle, and context dependent. Existing benchmarks often assume this evidence has already been localized through images, short clips, or pre-segmented videos, leaving the retrieval-before-reasoning problem under-tested. We introduce MedHorizon, an in-the-wild benchmark for long-context medical video understanding. MedHorizon preserves 759 hours of full-length clinical procedures and provides 1,253 evidence-grounded multiple-choice questions that jointly evaluate sparse evidence understanding and multi-hop clinical reasoning. Its evidence is extremely sparse, with only 0.166% evidence frames on average, requiring models to search noisy procedural streams before interpreting and aggregating findings. We evaluate representative general-domain, medical-domain, and long-video MLLMs. The best model reaches only 41.1% accuracy, showing that current systems remain far from robust full-procedure understanding. Further analysis yields four key findings: performance does not scale reliably with more frames, evidence retrieval and clinical interpretation remain primary bottlenecks; these bottlenecks are rooted in weak procedural reasoning and attention drift under redundancy, and generic sampling methods only partially balances local detail with global coverage. MedHorizon provides a rigorous testbed for MLLMs that retrieve sparse evidence and reason over complete clinical workflows.

The key--value (KV) cache is a major bottleneck in long-context inference, where memory and computation grow with sequence length. Existing KV eviction methods reduce this cost but typically degrade performance relative to full-cache inference. Our key insight is that full-cache attention is not always optimal: in long contexts, irrelevant tokens can dilute attention away from useful evidence, so selective, learnable eviction can improve generation rather than merely approximate the full cache. We introduce a global retention-based KV eviction method that learns each token's future utility under a unified memory budget. Lightweight retention gates assign utility scores to cached KV entries, and a shared final scoring projection calibrates these scores across all layers and heads. This enables a single global eviction policy in which tokens from different layers, heads, and modalities compete directly for cache capacity. We further provide theoretical analysis showing that preferentially retaining useful tokens reduces attention dilution, and we justify geometric retention as a query-agnostic proxy for future utility. Across long-context language and vision--language benchmarks, our method substantially reduces KV memory while matching or surpassing full-cache inference. These results suggest that learned, globally calibrated KV eviction is not only a compression technique, but also a mechanism for improving long-context reasoning.

Video-based world models hold significant potential for generating high-quality embodied manipulation data. However, current video generation methods struggle to achieve stable long-horizon generation: classical diffusion-based approaches often suffer from temporal inconsistency and visual drift over multiple rollouts, while autoregressive methods tend to compromise on visual detail. To solve this, we introduce LongScape, a hybrid framework that adaptively combines intra-chunk diffusion denoising with inter-chunk autoregressive causal generation. Our core innovation is an action-guided, variable-length chunking mechanism that partitions video based on the semantic context of robotic actions. This ensures each chunk represents a complete, coherent action, enabling the model to flexibly generate diverse dynamics. We further introduce a Context-aware Mixture-of-Experts (CMoE) framework that adaptively activates specialized experts for each chunk during generation, guaranteeing high visual quality and seamless chunk transitions. Extensive experimental results demonstrate that our method achieves stable and consistent long-horizon generation over extended rollouts. Our code is available at: https://anonymous.4open.science/r/AMSVVD-fdg245.

Algorithmic risk predictors are increasingly being used to inform resource allocation in healthcare and other high-stakes domains. Common in many such settings, the public decision maker delegates service provision to private organizations and compensates them according to each individual's predicted cost of service. We show that this common allocation mechanism can fail when organizations engage in \emph{favorable selection}, i.e., strategically choosing individuals whose actual costs are below the payment model predictions. Motivated by health insurance payments in the Medicare Advantage (MA) program, we provide a theoretical framework to study the interaction between a decision maker and a strategically selecting service provider. Our theoretical framework predicts two empirical patterns observed in Medicare Advantage that have become central concerns: enrollment in the private program expands, while government payments rise above the counterfactual cost of direct public provision. Guided by our theoretical framework, we then propose a reformed payment policy that is implementable under the government's data-access constraints. We prove that its equilibria eliminate overpayment while preserving profits for efficient organizations. We validate the framework in synthetic and semi-synthetic experiments that illustrate the advantages of our reformed policy, over the status quo.

Off-policy, value-based reinforcement learning methods such as Q-learning are appealing because they can learn from arbitrary experience, including data collected by older policies or other agents. In practice, however, bootstrapping makes long-horizon learning brittle: estimation errors at later states propagate backward through temporal-difference (TD) updates and can compound over time. We propose *long-horizon Q-learning* (LQL), which introduces a principled backstop against compounding error when learning the optimal action-value function. LQL builds on a prior *optimality tightening* observation: any realized action sequence lower-bounds what the optimal policy can achieve in expectation, so acting optimally earlier should not be worse than following the observed actions for several steps before switching to optimal behavior. Our contribution is to turn this inequality into a practical stabilization mechanism for Q-learning by using a hinge loss to penalize violations of these bounds. Importantly, LQL computes these penalties using network outputs already produced for the TD error, requiring no auxiliary networks and no additional forward passes relative to Q-learning. When combined with multiple state-of-the-art methods on a range of online and offline-to-online benchmarks, LQL consistently outperforms both 1-step TD and n-step TD learning at similar runtime.

Long-horizon agency requires an AI system to preserve task state across hours or days of work, many tool calls, and repeated context resets. A common response is to scale a single resource: the model’s context window, together with inference-time reasoning. We argue that this response conflates long horizon with long context. Durable long-horizon state should not live only in the prompt; it should be externalized into the harness, the non-model infrastructure that manages tools, files, plans, checkpoints, sub-agents, permissions, and recovery. Publicly documented long-running agent systems already rely on such structures, including progress files, plan files, git history, rollback checkpoints, scratchpads, sub-agent decomposition, and browser-state checkpoints. We support this position with three lines of argument. First, memory-hierarchy history shows that pressure to scale one working resource repeatedly gave way to externally managed hierarchical state. Second, we identify five structural mismatches between long-context scaling and long-horizon agency: degraded effective context, lossy compaction, a single trust region, no transactional semantics, and no working-set decomposition. Third, we map long-horizon requirements to what context scaling provides and what harness-managed state must supply. We conclude by proposing eight harness state-management patterns and arguing that harness state should become a first-class research object for long-horizon agency.

Sparsity has long been a central theme in LLM efficiency but its role in context processing remains unresolved. As LLM workloads shift toward longer contexts and agentic interactions, the compute and memory bottlenecks of attention become increasingly critical, raising the question of whether these constraints are fundamental. Our position: this constraint is artificial, unnecessary and the future of LLM inference lies in extreme but principled sparsity along the context dimension. Our position comes out of various bits of empirical and theoretical evidence. Firstly, we find insistance on dense attention unreasonable since in long context a query essentially projects the O(N) attention information into hidden space of diemsion $d << N$. This, as we show is extremely lossy. Instead, we propose a completely context-sparse context processing combining top-k sparsity with linear attention / SSMs as the way forward. To support our proposal we show that this combination can approximately cover the functional space of dense softmax attention. Independently but importantly, in a first elaborate study of its kind, we empirically show a strong trend towards current LLM models, which are not trained for context sparsity, being extremely robust to inference time decode-sparsity across tasks of varying complexities such as retrieval, multi-hop QA, and mathematical reasoning and agentic coding. For instance, Qwen3.5-27B can tolerate upto 100x sparsity on benchmarks of RULER-HARD, LOFT and AIME2025 without loss of quality, and upto $50\times$ on SWE with a small drop in quality. These results emphasize the possibility that we can transition to complete sparsity without any loss of capability. Additionally, we also discuss what this shift in paradigm of context processing means for hardware. Importantly, we show that even current hardware is equipped enough to realise gains from this sparsity. For instance, our sparse decode kernels can accelerate large context processing by a factor 10x over FlashInfer at 50x sparsity levels on current hardware such as H100. Overall, these results position extreme context sparsity not as a heuristic, but as a principled foundation for LLM inference, training, and architecture design, both feasible and beneficial, and a compelling direction for future systems.

Long-form generation in language models often collapses into repetitive loops well before the nominal context limit. This is commonly attributed to model capacity or training data, and addressed through costly retraining or post-training pipelines. We show that the sampling algorithm is a primary determinant of long-context output quality, improving outputs without further training costs. Across ten open-weight models from 1B to 120B parameters, output lengths from 8K to 64K tokens, and more than 13,500 generations, full-distribution-aware samplers (P-less, Top-H, top-$n\sigma$) dominate automated metrics and human evaluation, while standard truncation (top-$p$, top-$k$) collapses into late-context repetition; min-$p$ falls in a middle tier. A 1,500-annotation Amazon Mechanical Turk study finds P-less winning more than nine in ten head-to-head pairings against a top-$p$ 0.9 baseline on a 120B model, with 10-gram repetition approaching zero at 64K tokens. A per-step selected-token rank diagnostic isolates the mechanism: standard truncation drives the rank to the argmax floor by the final quartile, while full-distribution-aware methods stay in a healthy band across the full window. This gap that widens with model scale, establishing sampler choice as an under-appreciated bottleneck for long-context generation in open-weight models. We frame long-context generation as sampling from outside the training distribution, and encourage further study for usecases like long-context agentic coding.