NeurIPS 2026 Papers

Convolutional architectures have emerged as powerful alternatives to Transformers for sequence modeling. The primary advantage is that they offer improved theoretical sequence length complexity by leveraging the Fast Fourier Transform (FFT). However, this theoretical improvement does not always meaningfully land in practice. One critical obstacle is that applying standard FFTs is not amenable to the large-scale training pipeline wherein data is packed from different sources into a single sequence for hardware efficiency. Indeed, standard FFT algorithms are not easily amenable to document packing. Existing workarounds suffer from severe inefficiencies, crippling the practical performance of convolutional architectures. We close this gap with RubiConv, a novel algorithm for performing hardware-efficient, boundary-respecting convolutions on packed sequences. Extensive experiments show that RubiConv achieves significant speedups over both attention and standard FFT-based baselines. This work makes the theoretical efficiency of long convolutional models a practical reality for large-scale, real-world data packing.

Agents that perform state estimation using pretrained perception models, e.g., Vision-Language Models (VLMs), face the fundamental challenge that predictions are uncertain. Still, most approaches determinize these predictions into a single best-guess state and make decisions as if it were ground truth. This is brittle because when the guess is wrong, the agent's decisions may be unsafe or infeasible. At the other extreme, maintaining a full distribution over symbolic states is often intractable. We propose a middle path, drawing on the underlying probabilities the perception model already produces (e.g., a VLM's output-layer token distributions) to construct a θ-confidence Minimal Belief Set (MBSθ), a minimal subset of states whose cumulative probability meets a user-defined threshold. The minimality of an MBSθ ensures that the agent considers only the most likely states, enhancing robustness in downstream decision-making while maintaining computational tractability. Using a novel three-layered belief system, we efficiently approximate individual state probabilities from per-feature predictions and task-level constraints, on which we base our MBSθ construction, while maintaining only a lean factored representation of the belief. Our algorithm, PRaVaSS, takes advantage of the unique structure of this belief system to construct an MBSθ with correctness guarantees and polynomial complexity. Using a VLM as the backbone of our state estimator, we validate our method's theoretical properties, scalability with problem size, and improvement of downstream decision-making when used in a task planning pipeline.

Moving an object in a single image requires geometry-consistent spatial rearrangement, including handling occlusions, revealing previously unseen regions, and maintaining coherent shadows and reflections. Existing approaches are not well suited to this setting and often fail to preserve such scene-level consistency. We address this problem by introducing a geometry-aware object motion method that operates directly on the internal representations of diffusion transformers. Our key insight is that rotary positional embeddings (RoPE) define a structured spatial field that can be explicitly manipulated to induce controlled motion. We extend 2D RoPE into a depth-aware formulation that encodes 3D spatial structure, enabling consistent object displacement and scene-aware updates. Our model is trained using synthetic data combined with a small set of real images via parameter-efficient fine-tuning. Despite minimal real supervision, it preserves object identity under large spatial displacements, generates plausible content in newly revealed regions, and consistently updates scene-dependent effects such as shadows and illumination. Experimental results demonstrate the superiority of our method.

Synthetic data generation is becoming increasingly prevalent as advances in generative modeling accelerate. However, theoretical understanding of when and how synthetic data can improve downstream estimation and prediction remains underdeveloped. This question is especially relevant when synthetic samples are generated from proposal mechanisms informed by trained generators or domain knowledge, whose distributions may differ from the target population. To address this question, we propose REAL-Syn: Risk-guided Estimation-aware Acceptance for Learning with Synthetic Data, a framework for statistical learning with synthetic data selection. REAL-Syn uses selected subsets of candidate synthetic data to reduce estimator variability and often improve downstream prediction under target-proposal mismatch. The proposed procedure uses a covariance-based risk surrogate to select from the set of candidate synthetic samples; the accepted samples are then reweighted and pooled with the real observations for inference. We present theoretical results on the surrogate-optimal acceptance rule and the proposed selection procedure. We empirically show that REAL-Syn achieves competitive performance in simulations and an application to solar flare intensity forecasting.

Open-loop action repetition—hereafter also referred to as a skip—enables reinforcement learning agents to reduce decision frequency by executing selected actions for adaptive durations. A key challenge is how to evaluate the value of such a skip. Existing Skip-MDP methods typically bootstrap from the greedy value of the terminal state, implicitly assuming that the agent can immediately resume near-optimal control after the skip. This assumption is unreliable during learning, when the underlying action policy is stochastic, inaccurate, and continually changing. It can also amplify optimistic estimation errors, distorting the learned preference over skip lengths. We propose Expected Skip Evaluation, a reformulation of skip-value learning that evaluates skip terminal states through expected future control rather than idealized greedy control. This reflects imperfect control during training while preserving the optimal skip-value fixed-point as control becomes reliable. We instantiate this principle in RARe (Risk-Aware Repetition), a practical framework for discrete and continuous action spaces. Experiments across grid-world, continuous-control, and safety-critical benchmarks show improved sample efficiency and better-calibrated skip decisions.

Reconstructing multi-camera videos is a fundamental task in autonomous driving. However, recent learned methods either rely on costly attention-based networks for global geometry modeling or require complex post-hoc alignment for chunked predictions, limiting their scalability to long multi-view rig sequences. We propose RigRecon, a generalized rig-aware reconstruction framework that enables efficient cross-view interaction in a low-resolution space while asynchronously decoding high-resolution depth maps in a frame-wise manner. Input images are first down-sampled and processed by a transformer backbone to capture coarse global structure, which is then provided to a decoder to reconstruct frame-wise high-resolution depth maps. By fully leveraging calibration and low-resolution feature interaction, RigRecon achieves consistent-scale geometry and accurate pose estimation with significantly reduced memory cost. It achieves state-of-the-art performance on 3D reconstruction, depth estimation, and camera pose estimation, and can efficiently process videos with over 1000 images in a single feedforward pass.

Recent advancements in multimodal diffusion language models (MDLMs) have exhibited strong global modeling capabilities through iterative refinement. We observe that mask uncertainty follows an overall downward trend during iterative denoising, while a few denoising steps exhibit localized entropy spikes closely associated with hallucinations. We argue that anomalous steps can be identified by measuring the entropy deviation of each denoising step from its neighboring temporal window. We hypothesize that discrete token commitment discards the distributional information at masked positions, limiting the continued interaction between candidate semantics and the context during entropy spikes. With this goal, we present StepRefine, a plug-and-play decoding strategy that leverages semantic context to achieve reliable denoising. The core of our method lies in a step-aware switch between discrete denoising and latent refinement. The model employs probability-weighted embeddings at entropy spike steps to preserve diverse reasoning hypotheses and performs multiple rounds of continuous refinement. Meanwhile, a refinement controller monitors distributional stability to reduce unnecessary loops and switch the model back to standard discrete denoising. Through extensive experiments, StepRefine demonstrates significant hallucination mitigation across different MDLMs on multimodal benchmarks.

Standard decoding in diffusion language models (DLMs) is controlled through parallel, token-wise unmasking decisions. However, diffusion decoding is inherently a structured, sequence-level process, in which token states evolve under dynamic cross-token dependencies and are therefore poorly captured by purely token-wise unmasking control. Motivated by this mismatch, we view diffusion decoding from a structural perspective, in which iterative denoising progressively gives rise to token subsets that are internally reliable and weakly dependent on the remaining masked positions. We therefore introduce structural commitment, a sequence-level unmasking principle that treats approximately closed token subsets as the basic unmasking units in diffusion decoding. We instantiate this principle with Structural Commitment via Closure Expansion (SCCE), a structure-aware inference-time algorithm that discovers approximate closures and commits high-certainty subsets by unmasking them, with certainty measured by balancing token reliability against external dependency leakage. Empirically, SCCE improves the accuracy–efficiency frontier over local confidence-based unmasking baselines while adding negligible measured per-step overhead in our implementation.

Onboard satellite models often require frequent updates, but the weights adapted to earlier data distributions can quickly become outdated. However, updating large-scale model parameters in orbit presents significant challenges due to the limited uplink bandwidth of Low Earth Orbit (LEO) satellite systems, particularly for hyperspectral satellite imagery, where high-dimensional spectral–spatial inputs lead to increased model size and update costs. Existing full fine-tuning methods are thus expensive to retrain and difficult to deploy under strict communication constraints. To address this challenge, we propose NE-LoRA, a parameter-efficient adaptation framework for bandwidth-constrained onboard hyperspectral model updates. NE-LoRA combines a primary low-rank branch with a nonlinear auxiliary branch to capture both global update trends and complex spectral–spatial variations. Additionally, we introduce a differentiated training strategy for multi-matrix adapters, motivated by the asymmetric initialization and gradient dynamics of different adapter matrices. Experiments on four hyperspectral datasets and three representative backbone models demonstrate that NE-LoRA consistently outperforms LoRA-based baselines and remains competitive with, and in several cases superior to, full fine-tuning. Across the evaluated settings, NE-LoRA updates only a small fraction of the total parameters on average while preserving low deployment overhead, offering a favorable accuracy–communication trade-off for onboard hyperspectral adaptation.

Density Peak Clustering (DPC) focuses on identifying centers and propagates labels at the sample level, which makes density estimation cutoff-sensitive and assignment chain-vulnerable, yielding suboptimal performance under heterogeneous densities and complex cluster structures. This paper argues that density peaks inference should instead be performed on resolution-controlled structural units: density becomes a stable count statistic, relative distance inherits the separation of the enclosing hierarchy, and assignment runs over a compact graph of structural units rather than long sample-to-sample chains. This view is realized in Resolution-Aware Structural Density Peak Clustering (RSDP), which builds a constrained hierarchy, selects an admissible level via a single resolution parameter, and applies a one-pass DPC rule on the selected structural units. Under a faithful-realization condition, it is proved that RSDP's empirical $\gamma$-ranking recovers the dominant population units and that its one-pass assignment recovers the sampled ground-truth partition. On synthetic and real-world datasets, RSDP achieves the highest ACC and NMI against state-of-the-art DPC variants, exceeding the strongest by 19.91% and 9.09% in average ACC and NMI, respectively.

Standard training of diffusion models typically relies on a forward diffusion process anchored by ground-truth target images. However, in image fusion tasks, the scarcity of real fused images makes it difficult to formulate a forward process, thereby precluding the standard training. Fortunately, our visual analysis reveals that image fusion can be effectively modeled without standard training or reliance on the real images. Nevertheless, the misalignment of input image pairs remains a significant bottleneck for fusion quality. Recognizing that one-off registration is ill-suited to the progressive generation of diffusion models, we propose a unified registration-fusion framework driven by a time-aware joint optimization mechanism. Specifically, we design a time-aware registration network to progressively optimize registration to guide fusion, while the fusion results provide feedback constraints to the registration network at each time step. This mechanism facilitates joint optimization of registration and fusion, significantly improving the quality of fused results. Experimental results show that the proposed method performs excellently on multiple datasets, validating its effectiveness and superiority.

Federated Multi-View Learning (FedMVL) enables multiple clients with heterogeneous data views to collaboratively train global models without sharing private data. Existing FedMVL algorithms often overlook the reliability of local predictions, as both uncertainty quantification and calibration remain challenging in privacy-preserving federated settings. Heterogeneous sensing devices yield varying view qualities, but view-isolated training further exacerbates inherent bias of local model, inducing unreliable outputs. Without proper calibration, these artifacts cause spurious cross-view conflicts, ultimately undermining the accuracy and reliability of the aggregated global model. To address these issues, we propose a Reliable Federated Multi-View Learning with evidence calibration (FedRMVL-CAL), a FedMVL framework with adaptive evidence fusion. The local calibration scheme harmonizes uncertainty across views, while the fusion mechanism adaptively reconciles conflicting opinions, enhancing robustness in global inference. Theoretical analysis and extensive experiments demonstrate that FedRMVL-CAL achieves superior accuracy and reliability compared to existing approaches, ensuring trustworthy global predictions across heterogeneous views.

In real-world multimodal time-series inference, a common practice is completion–inference decoupling: missing modalities are heuristically imputed and then passed to deterministic fusion and decision making. We show that this chained design yields unreliable predictions under missingness, noise, and temporally varying modality reliability, because imputation errors propagate without uncertainty feedback. Through theoretical and empirical analysis, we identify latent-consistent completion and evidential reliability modeling as two key ingredients for robust multimodal inference. Building on these insights, we propose MD2E-MI, a manifold-aware diffusion-based imputation and evidence-driven multimodal inference framework that couples diffusion-based completion with uncertainty-aware inference in a single pathway. Experiments demonstrate stable performance across diverse missingness and noise regimes, while providing interpretable, temporally varying reliability estimates.

Adapting language models on long-form corpora can improve domain behavior while also making protected passages easier to reproduce verbatim from short prefixes. We study this tension through \emph{relative energy gain}, the token-level log-density ratio between an adapted model and its pre-adaptation anchor on an exact protected continuation. This quantity separates ordinary predictability from update-induced copying pressure and accumulates over suffixes as a likelihood-ratio advantage for extractable memorization. Building on this view, we introduce Energy-Gated Copyright Regularization (EGCR), a training objective that augments maximum likelihood with a soft relative-energy budget on protected tokens. EGCR uses a differentiable gate combining local persistence of relative gain with anchor surprisal, so regularization is concentrated on distinctive spans whose probabilities rise in a memorization-like way while ordinary next-token supervision is retained elsewhere. Across BookMIA adaptation stress tests, training-prefix extraction, CopyBench transfer, and gate ablations, EGCR reduces exact-match and overlap-based reproduction while preserving held-out language-modeling and book-related utility. Token-level visualizations further show that the gate focuses on a small set of expressive positions associated with verbatim recovery. These results suggest that relative-energy barriers offer an effective and inspectable mechanism for copyright-aware language model adaptation.

In recent years, large language models (LLMs) have demonstrated remarkable capability to generalize across diverse natural language processing tasks, inspiring the development of graph foundation models (GFMs) for large-scale pre-training. However, unlike language models with explicit token units, graphs lack a well-defined unit for generalization, making it challenging to design effective pre-training strategies. In this work, we propose REEF, a novel GFM framework that leverages relation tokens as the fundamental units. We construct a vocabulary of relation tokens to encode relational information within graphs. To accommodate diverse relations, we introduce two hypernetworks that adaptively generate the parameters of aggregators and classifiers in graph neural networks based on relation tokens. In addition, we design another hypernetwork to construct dataset-specific projectors and incorporate a dataset-level feature bias into the initial node representations, enhancing flexibility across different datasets with the same relation. Extensive experiments demonstrate that REEF consistently outperforms existing methods in both pre-training and transfer learning, highlighting its potential as a general-purpose graph foundation model. Our code is publicly available at https://anonymous.4open.science/r/REEF-16CF/.

The Panoptic Quality (PQ) metric is the standard for jointly evaluating instance and semantic segmentation. However, its original definition relies on a One-to-One matching between predicted and ground truth segments, which is only straightforward when the IoU threshold exceeds 0.5. Below 0.5, multiple matching strategies emerge in a poorly explored problem space. We systematically elucidate this space by recasting segment matching as a constrained bipartite assignment problem. Independently bounding the prediction- and ground-truth-side degrees yields four matching strategies: One-to-One, Many-to-One, One-to-Many, and Many-to-Many. We show that the first three are well-defined within the PQ framework, while Many-to-Many falls outside it. These strategies become relevant when instances are fragmented, adjacent objects are difficult to delineate, or annotations are noisy. Central to our framework is a vertex-based accounting of TP, FN, and FP, anchored to ground truth and predicted segments rather than to matching edges. We further show that the framework extends naturally to part-aware panoptic segmentation, and we explore part-aware evaluation on biomedical data. Across configurable case studies we report how different combinations of thresholds and matching strategies behave in practice. We release a unified open-source package built on Panoptica. It exposes Voronoi-based region-wise analysis, part-aware evaluation, and Area Under Threshold Curve computations as configurable options.

Personalization systems encode long-horizon, evolving user trajectories into compressed preference states that downstream task heads consume for prediction or generation. This compression is difficult because preference trajectories contain stable long-term interests, transient short-term shifts, and episodic bursty interests. We argue that a practical route is not to design another task-specific encoder or incur costly host-encoder finetuning, but to correct the under-expressed compressed state before the downstream task head consumes it. We propose $\texttt{REPAIR}$ as a selective *pre-head state-repair method* for frozen personalization encoders. It plugs into frozen hosts, uses event representations from the forward pass, and avoids a second pass over the raw history. $\texttt{REPAIR}$ compares the compressed state with per-timestep event representations, estimates state-relative corrective evidence, resolves it through long-term, short-term, and episodic temporal regimes, and retains only the most corrective signals to form a compact correction. We evaluate $\texttt{REPAIR}$ across recommendation and generation personalization settings using MovieLens, MIND, PENS, and Amazon Reviews 2023. Across recommendation tasks, $\texttt{REPAIR}$ improves the evaluated frozen hosts. On MovieLens, it improves the highest-scoring frozen host in our instantiated baseline suite by +1.27/+1.54 MRR/nDCG@5; on MIND, it improves the highest-scoring frozen host by +4.41 MRR; and on Amazon product recommendation, it improves two SOTA baselines by +15.33 and +17.91 MRR. It also outperforms budget-matched post-head correctors, and the full long-, short-, and episodic repair remains the strongest temporal variant. On PENS personalized headline generation, $\texttt{REPAIR}$ improves subjectivity-sensitive generation most when the decoder is preference-coupled, with average PerSEval gains of +14.62% and +22.19% for two representative preference-coupled decoders. Relative to frozen predictive encoders, deployment overhead is 31--34% memory and 38--44% latency.

Reasoning models increasingly depend on test-time computation, but raw accuracy conflates intrinsic ability, problem difficulty, guessing effects, and reasoning budget. Item Response Theory (IRT) adjusts for problem difficulty, yet collapses test-time compute into a single latent score. We propose reasoning-aware IRT, an extension in which each model's effective ability depends on baseline ability, reasoning-budget responsiveness, and overthinking curvature. Parameter estimation is performed via multi-pass generalized EM with post-hoc initialization refinement. On LiveBench with 42 language models, our method improves rank stability, gives stronger held-out correctness discrimination, and reveals interpretable structure unavailable to classical IRT: a unified benchmark--ability scale with quantified reasoning-token benefits, budget-efficient and overthinking regimes, and rank reversals obscured by raw scores. Our framework offers a compact, explainable evaluation tool for reasoning models under test-time scaling.

Noisy labels can cause deep neural networks to overfit incorrect annotations, leading to biased predictions and poor generalization. A common remedy is to select clean samples after a short warm-up phase, under the assumption that the model begins to learn class-specific patterns from the early training stage. However, this assumption often fails under class imbalance, as the majority classes can hinder the model from learning distinctive features of minority classes. In this paper, we propose a readiness-aware sample selection strategy that identifies clean samples only from classes whose distinctive features have been sufficiently learned and are thus considered ready for selection. We further introduce a novel negative learning scheme to enhance class separability by discouraging confusion with the most similar incorrect classes. The proposed method is supported by theoretical analysis and demonstrates outstanding performance on both benchmark and real-world datasets, showing improved robustness and generalization under noisy and class-imbalanced conditions.