NeurIPS 2026 论文列表

We introduce **PD-RAG**, a technique for retrieval-augmented generation (RAG) that leverages a language model's own randomness to safeguard privacy. The algorithm partitions documents into groups, generates a candidate answer from a randomly chosen group, and then uses a privacy test to enforce that the released answer could have been produced by multiple disjoint document groups, thereby ensuring a controllable degree of *plausible deniability*. Compared to existing methods that operate at the token level and therefore incur both substantial utility loss and growing privacy budget per generated token, PD-RAG operates at the answer level, so the privacy it offers does not loosen with output length. We prove that PD-RAG satisfies $(\varepsilon,\delta)$-differential privacy. We experimentally evaluate PD-RAG on three QA benchmarks using three language models and find that it reduces membership inference attack advantage to near random while consistently outperforming alternative methods on all four utility metrics we used and running between $17$ and $33{\times}$ faster.

Long-horizon tool-augmented agents suffer sharp degradation as trajectories grow: small tool errors are stored, repeatedly reused, and amplified into cascading failures. Existing approaches rely on post-hoc verification, item-level memory scoring, or executive context management, but do not directly control how unreliable evidence propagates through reuse. We propose \textbf{EPOCH} (Edge-Pathway Outcome-supervised Cascade-Halting memory), which mitigates cascades by learning an outcome-correlated reliability proxy on reuse pathways connecting observations over time, instantiated as a sparse memory graph with quality-weighted edges, a Temporal Graph Transformer over edge reliabilities, and a learned mid-context correction. \textbf{We prove that quality-weighted and uniform retrieval are asymptotically separated in horizon length:} the ratio of cumulative cascade error diverges unboundedly as $T \to \infty$ whenever the calibration parameter is positive, a result that does not require distributional regularity. We validate this prediction empirically at trajectory lengths up to $T=500$ and additionally establish a quantitative edge-vs-node separation in terms of an empirically measurable cross-context error variance. Because outcome supervision is correlational by construction, we do not claim the learned scores recover causal pathway reliability; we validate them on two complementary axes (counterfactual edge ablation: $\rho=0.71$, $n=21{,}743$; human annotation: ROC-AUC~$=0.94$, ECE~$=0.041$, $n=6{,}000$). Across 11 multi-modal benchmarks spanning native long-horizon search, wrapped knowledge-seeking tasks, and clean perception controls, EPOCH consistently outperforms the strongest reliability-aware and memory-augmented baselines in noisy long-horizon regimes while remaining on par on clean perception controls---quality-aware filtering does not collapse on clean data without claiming it helps there.

Dense associative memory underlies modern attention mechanisms; however, conventional Modern Hopfield Network (MHN) with linear kernels yields overlapping attractor basins and spurious states under high memory load, limiting their theoretically exponential storage capacity in practice. We construct a class of nonlinear phase kernels that reshape the energy dynamics into deep, well-separated basins, and prove that any kernel in this class preserves exponential storage capacity while guaranteeing convergence to fixed points. As a representative instantiation, we introduce the Sigmoidal Phase Hopfield Network (S-Hop), which guarantees monotone energy descent and mitigates the practical loss of memory capacity. Experiments on MNIST and CIFAR-10 show that S-Hop achieves up to a $50\times$ increase in critical capacity over baseline models. We also provide a systematic capacity measurement of Hopfield networks on TinyImageNet, where S-Hop exhibits improved retrieval robustness and noise resilience. The phase-kernel design provides a principled route toward more robust dense associative memory.

Retrieval-augmented generation (RAG) is bottlenecked by the token budget of large language models: feeding long documents into the context window is expensive, while aggressive textual compression discards fine-grained information that is irreversibly lost in a one-dimensional token sequence. We argue that this bottleneck is not fundamental to information density but to the choice of compression \emph{modality}. Modern visual encoders can pack the contents of a rendered text page into roughly a hundred visual tokens with little semantic loss, opening a new compression-fidelity tradeoff that purely textual methods cannot reach. We present \textbf{OpticalRAG}, a framework that renders documents as page images, encodes them with a frozen vision encoder, and distills the retrieved visual tokens into as few as $16$ injected tokens before passing them to the LLM. Two ingredients make this work without expensive cross-modal training: (i) \emph{Encoder-Bridged Retrieval}, which reuses the shared representation space of a pretrained vision-language model to retrieve visual chunks from a text query, with no contrastive training; and (ii) \emph{Query-Driven Distillation}, a lightweight cross-attention compressor that condenses thousands of visual tokens into a small set of query-conditioned prefix tokens. On LongBench-v2, BAMBOO, and LooGLE-v2, OpticalRAG achieves the best compression-aware accuracy (CAP) on all three benchmarks while injecting only $16$ tokens, and improves the closed-book Qwen2.5-7B-Instruct backbone by $5.17$ points on LongBench-v2.

Autoregressive (AR) video generation extends videos by producing latent chunks sequentially, but scaling to long videos requires repeated access to a growing historical KV cache. Existing methods reduce this cost by truncating the KV cache or compressing it into implicit memory, but both lose explicit access to query-relevant historical details. We propose OmniMem, an explicit full-range memory retrieval framework that performs sparse KV retrieval over the historical cache. To make this practical for chunk-based AR video generation, OmniMem addresses two issues: (i) local bias in sparse KV selection and (ii) Union Explosion in memory access. Adaptive Window Exclusion removes local-window blocks from the selection candidates when sufficient long-range history is available, preserving the sparse budget for informative long-range retrieval. Query-shared KV Selection reduces cross-query diversity, while Per-Head Scattered KV Access avoids expanding head-specific selections into a large selected KV buffer. This allows each attention head to retrieve non-contiguous KV blocks according to its own selection pattern. Experiments on long-video generation show that OmniMem improves Dynamic Degree by 52.3\% and preserves strong consistency over strong baselines, while maintaining comparable memory usage.

Memory-augmented LLM agents enable interactions that extend beyond finite context windows by storing, updating, and reusing information across sessions. However, training such agents with reinforcement learning in multi-session environments is challenging because memory turns the agent's past actions into part of its future environment. Once different rollouts write, update, or delete different memories, they no longer share the same intermediate memory state, making trajectory-level comparisons fundamentally unfair. This violates a key assumption behind group-relative methods such as GRPO, where rollouts are compared as if they were sampled from the same effective environment. Consequently, trajectory-level rewards provide noisy or biased credit signals for long-horizon memory operations. To address this challenge, we introduce Memory-R2, a training framework for long-horizon memory-augmented LLM agents. Its core algorithm, LoGo-GRPO, combines local and global group-relative optimization. The global objective preserves end-to-end learning from long-horizon trajectory-level rewards, while local rerollouts compare alternative continuations from identical intermediate memory states, yielding fairer group comparisons and more precise supervision for memory writing, updating, and retrieval. Beyond credit assignment, Memory-R2 jointly optimizes memory formation and memory evolution with a shared-parameter co-learning design, where a fact extractor and a memory manager are instantiated from the same LLM backbone through role-specific prompts. To make memory construction more controllable, we further formulate each session as a multi-step decision process by chunking the interaction and alternating the two memory roles. A progressive curriculum from 8 to 16 to 32 sessions stabilizes training as the memory horizon grows. Together, these components provide an effective training paradigm for memory-intensive LLM agents in long-horizon multi-session settings.

Associative memory models classically describe retrieval as convergence to stored patterns, but the theory of sequential retrieval remains comparatively limited. We develop a two-timescale input-driven Hopfield network in which fast associative retrieval is modulated by a slow feedback variable evolving over a prescribed memory-transition graph. This yields autonomous, graph-constrained transitions while preserving analytical tractability. Using slow-fast and geometric singular perturbation theory, we characterize memory fixation via memory manifolds and identify the geometric regions that govern stability loss and escape from retrieved memories. We further derive reduced slow dynamics and explicit criteria for transition onset, self-sustained retrieval, and collapse, expressed through escape times, gain thresholds, and fixed points. Together, these results provide a tractable framework for analyzing feedback-driven sequential retrieval in Hopfield networks.

The memory capabilities of Large Language Models (LLMs) have garnered increasing attention recently. Despite great success achieved, existing retrieval-based memory approaches typically overlook the differences between memories and employ a unified strategy to process all memories, leading to suboptimal performance. Thus, an intuitive question arises: can we categorize memory into different types and select appropriate strategies? However, given the topic-rich, scenario-complex, and boundary-blurred nature of memory scenarios, achieving precise classification of memories is not easy. To address this challenge, we propose a memory multi-class dataset in this paper, termed TriMEM, which provides precise annotations for memory types across diverse scenarios. Building upon this foundation, we propose a novel memory framework, named MemoType, which can adaptively recognize each memory and query type with the learned router model. With the memory and query routing, MemoType can retrieve the memory with corresponding query types and design tailored retrieval strategies, thereby enhancing the retrieval performance. Moreover, we theoretically prove that any single retrieval strategy is subject to a fundamental upper bound on its expected retrieval precision in multi-class corpora, leading to systematic precision degradation. Extensive experiments on three datasets demonstrate that MemoType consistently outperforms existing methods, achieving up to 16.18% improvement in Recall@1.

Long-context dialogue systems must decide both when to access memory and which parts of the interaction history are relevant. Existing approaches typically rely on heuristic retrieval signals or always-on memory usage, failing to account for the changing and potentially inconsistent nature of user preferences. In this work, we propose a unified framework for memory access and selection based on changing preferences. We formulate personalized memory retrieval as identifying which historical turns provide evidence about a user’s latent preference state, rather than relying on surface-level semantic similarity. To this end, we quantify the utility of each memory turn using a Bayes factor, defined as the improvement in the model’s likelihood of the reference response when the turn is included in context. This provides a principled measure of evidence strength and a unified signal for both memory access and selection. By framing memory retrieval as utility estimation, the model learns to identify salient turns and regulate memory usage based on expected utility. Experiments on four heterogeneous memory benchmarks show that our approach outperforms existing embedding-based retrieval on long-context, preference-intensive tasks where modeling changing preferences is essential, while remaining competitive in low-density regimes where semantic similarity suffices.

Large language models acquire rich internal knowledge during pre-training, yet compact models are still typically forced to relearn such knowledge from raw data or imitate it indirectly through distillation. We investigate a different route: can pretrained knowledge itself be directly reused during pre-training? To this end, we propose Memory Grafting, a representation-level transfer paradigm that extracts structured memories from a large frozen donor model and injects them into a compact student throughout training. Unlike knowledge distillation, which transfers behavior through output alignment, or retrieval-based methods, which expose external knowledge only at inference time, Memory Grafting makes pretrained knowledge part of the student’s optimization process itself. This suggests that knowledge in large language models can be modular, transplantable, and reusable across scales, pointing to a new path toward more efficient language model pre-training.

Long-tailed visual recognition is limited not only by biased optimization, but also by the under-retention of rare-class evidence in parameters learned from imbalanced data. MemTailor addresses this limitation by augmenting a frequency-aware multi-expert backbone with a hierarchical class-aware memory that preserves visual evidence outside the backbone. The memory stores per-class prototypes and bounded instance slots for each expert, allocates proportionally richer capacity to rare classes, and records both shallow and deep features to retain complementary discriminative cues. During training, same-class memory retrieval constructs class-adaptive feature augmentation and provides an auxiliary supervised objective on the augmented features. At inference, class-level memory quality and prediction uncertainty determine when stored evidence is allowed to correct expert logits, and the final prediction fuses memory-corrected logits across multiple input scales. Experiments on long-tailed visual recognition benchmarks show that MemTailor improves balanced recognition, especially for under-represented classes.

Integrating external memory into Large Language Models (LLMs) typically faces a trade-off between flexibility and depth. Explicit text-based retrieval keeps memory editable, but often acts as a static prompt prefix, increasing context length and interacting with the reasoning process only indirectly. Conversely, parametric memory can influence internal computation more deeply, but updating memory content usually requires additional optimization and may couple domain-specific information with the model's general reasoning behavior. In this work, we introduce MemPilot, a decoupled latent memory framework that dynamically guides the hidden reasoning states of frozen LLMs. MemPilot constructs an external bank of compact latent representations from offline reasoning trajectories, retrieves candidate memory entries, and integrates their latent memory tokens into hidden states via cross-attention and gated residual fusion. By separating domain-specific memory content from the learned memory-use mechanism, MemPilot enables target-domain adaptation through offline memory-bank substitution without updating the base model. Experiments across QA, coding, and mathematical reasoning benchmarks show that MemPilot achieves strong in-domain performance and more robust cross-domain transfer than other memory-augmented baselines. These results suggest that LLMs can benefit from reusable latent memory-use mechanisms while keeping memory content external and replaceable.

Diffusion Language Models (DLMs) offer attractive advantages over Auto-Regressive (AR) models, such as full-attention parallel decoding and flexible generation. However, standard DLM training uses a static, single-step masked prediction objective that never exposes the model to the progressive denoising dynamics of inference, and forces all contextual information to be maintained purely through token-space attention, which becomes increasingly diluted as context length grows. We propose MemDLM (Memory-Enhanced DLM), which introduces a second memory channel by embedding a simulated denoising trajectory into training via Bi-level Optimization. An inner loop updates a set of fast weights, forming a Parametric Memory that captures the local trajectory experience, while an outer loop updates the base model conditioned on this memory. By offloading part of the memorization burden from token-space attention to parameter space, MemDLM yields faster convergence, stronger long-context representations, and lower training loss, even when the fast weights are discarded at inference time. Re-enabling the inner loop at inference provides an additional prompt-specific adaptation effect, where the Parametric Memory acts as an emergent in-weight retrieval mechanism on challenging Needle-in-a-Haystack tasks.

Modeling fine-grained human manipulation of object functional parts is essential for achieving human-like dexterous manipulation in virtual reality and robotics. Despite recent advances, data-driven generation methods remain largely constrained by patterns learned from limited training data, rendering them prone to hallucinating task-inconsistent manipulations when encountering samples beyond the training distribution. To address this challenge, we propose ManipulationRAG, a framework that augments an internal generative model with externally retrieved, task-relevant manipulation knowledge. Specifically, we design a hand-centric manipulation taxonomy and construct a structured knowledge base that organizes task descriptions, objects, manipulation types, and corresponding hand pose sequences, supported by a retrieval mechanism for accurate knowledge retrieval. The retrieved knowledge serves as an external manipulation-specific prior, guiding the diffusion model toward task-consistent and physically plausible manipulation synthesis. Extensive experiments demonstrate that our method outperforms existing methods and generalizes robustly to object manipulation beyond the original diffusion model's training domain.

Long-term LLM agents must decide what to write into persistent memory before future queries are known, yet existing evaluations measure only final QA accuracy, entangling memory writing with retrieval and reasoning. We introduce MEMAUDIT, a package-oracle evaluation protocol that turns memory writing into a finite, auditable optimization problem with a certified ground-truth optimum under a fixed storage budget. Each evaluation package specifies an experience stream, candidate memory representations, and future-query requirements, enabling exact measurement of how well a memory writer preserves task-relevant information. We instantiate MEMAUDIT with a semantic coverage objective under storage and exclusivity constraints and compute exact optima via certified optimization. Across controlled and naturalistic settings, MEMAUDIT disentangles representation quality, validity preservation, and budget-aware selection effects that end-to-end QA cannot isolate. This enables, for the first time, principled and reproducible evaluation of memory-writing policies independent of downstream retrieval and reasoning.

Long-form technical text generation underpins knowledge-intensive workflows, yet remains challenging for large language models (LLMs) due to the need for globally consistent logical structuring and faithful technical reasoning beyond local coherence. Patent drafting is a canonical instance of this challenge, demanding holistic generation of a legally compliant and technically exhaustive document through sustained multi-expert collaboration. Existing approaches often focus on partial section generation or rely on manually crafted outlines, limiting scalable automation in realistic settings. In this work, we propose LogicTree-RAG, a logic tree-guided retrieval-augmented generation framework that induces a hierarchical logic tree as a global organizational backbone to organize and ground technical disclosures, without relying on expert-defined drafting priors. Each node in the logic tree represents a technical element and is constructed through evidence-guided recursive generation. A hybrid traversal mechanism then maps the logic tree into patent sections, enabling controllable and section-balanced generation. Extensive experiments show that LogicTree-RAG consistently improves content quality and language conformity over strong LLM-based baselines and achieves longer structured generation with high token efficiency, demonstrating the effectiveness of logic-centric generation for complex technical document drafting.

Embodied agents performing long-horizon tasks require a memory representation in which the state transitions of dynamic objects remain queryable in natural language across hours-to-days observation horizons. Existing systems either drop fine-grained motion (clip-level video-language embeddings), keep it only as raw coordinates (geometric SLAM), or organise it around immediate task context (agent working memories). None of them gives the agent a per-object timeline whose state transitions are themselves queryable in language. Our key contribution is \textbf{Linguistic Trajectory Encoding} (LTE), which compresses dynamic object motion histories via a hybrid representation combining natural language descriptions, sparse spatial anchors, and visual anchors. LTE adapts compression to motion complexity by anchoring periods without reliable observations to the last seen location, while representing motion with geometric waypoints and linguistic descriptions to preserve accuracy. To evaluate these capabilities across extended time horizons, we construct the \textbf{Spatial Memory Benchmark} (SMB) from EgoLife multi-day recordings, targeting capabilities absent in existing benchmarks: semantic trajectory retrieval and long-horizon object retrieval. On SMB, the LTE-based system achieves $45.3\%$ success in semantic trajectory retrieval and $48.7\%$ in long-horizon object retrieval, outperforming structured-memory and VLM baselines (best prior: $31.9\%$ and $34.4\%$). LTE achieves trajectory compression by factors of $8.7\times$ to $26.1\times$ with sub-second query latency on $24$\,h video. On Ego4D natural-language queries, the system reaches $28.75\%$ / $55.10\%$ R@1/R@5, $+15.80$ / $+31.30$ pts over EgoVLPv2.

Clinical case search aims to retrieve disease-consistent prior cases from electronic health record databases for a natural-language patient query under a retrospective retrieval objective. The task is challenging because patient queries contain heterogeneous evidence, including symptoms, diagnoses, laboratory results, treatments, and timelines. These fields are often only partially specified, and fixed retrieval pipelines cannot adapt from prior search trajectories. Existing clinical retrieval methods typically flatten the query into a single representation, use static retrieval pipelines, or lack an explicit mechanism for cross-case adaptation. \textbf{PatientSearch-SE} addresses retrospective disease-centric clinical case search with three components: a \emph{sufficiency-aware planner} that separates dimensions suitable for direct matching from those requiring hypothesis completion, a \emph{hierarchical tool orchestration} module that maps these states to retrieval and re-ranking actions, and a \emph{trajectory-guided memory consolidation} mechanism that writes reusable disease cards and strategy templates without parameter updates. We also propose an evaluation framework that combines disease-centric retrieval outcomes with trajectory-level process probes for retrieval behavior analysis. Experiments on PMC-Patients and MIMIC-IV show higher disease-centric retrieval metrics for PatientSearch-SE than for the compared same-interface sparse, dense, and reasoning-based baselines, with larger gains on PMC-Patients and more modest gains on MIMIC-IV. Its memory-accumulation results are consistent with controlled external-memory adaptation, and its retrieved cases provide additional evidence for diagnosis-masked downstream few-shot RAG prediction under this retrospective protocol.

In-context learning has recently been linked to implicit gradient descent in linear self-attention models, suggesting that context can induce a forward-pass update. Retrieval-augmented generation (RAG) also relies on context, but retrieved documents are usually treated as static evidence rather than signals for adaptation. We study RAG as an in-context optimization process. First, we show that one linear self-attention layer can implement one gradient-descent step on a unified linearized RAG objective covering both projection-based and dot-product retrieval interfaces. This gives an exact regime where retrieval-augmented prediction and in-context optimization coincide. We use this result not as a literal model of LLM computation, but as a guide for adapting the interaction between queries and retrieved evidence. We then test the boundary of this correspondence: it remains stable under controlled linear extensions, but becomes feature-distribution dependent under nonlinear architectures. Finally, we turn this view into a lightweight method for frozen RAG LLMs. The method keeps the retriever and backbone fixed, and predicts a context-conditioned update to a generator-side evidence-use interface. Across seven QA benchmarks, two retrievers, and two frozen LLM backbones, this forward-only update improves a shared-interface baseline, transfers to held-out tasks, and approaches test-time gradient adaptation at much lower per-query cost