NeurIPS 2026 论文列表

Recent work on the sequence universality of State Space Models (SSMs) has introduced efficient, maximally expressive continuous-time approaches for time-series modelling. While these works focus on discriminative settings, we extend this perspective to generative time-series modelling by proving that maximally expressive Structured Linear Controlled Differential Equations (SLiCEs) are universal time-series generators, in the sense that they can approximate the induced path laws of continuous causal pushforwards on compact latent sets in $W_\infty$. Building on these theoretical results, we propose Generative SLiCEs (G-SLiCEs), a maximally expressive continuous-time model for flow matching on path-space. Empirically, we show that expressivity improves performance in probabilistic forecasting and downstream tasks, while retaining the advantages of continuous-time models such as generalising to arbitrary observation grids. This is particularly beneficial for irregular grids, where fixed-grid models often struggle.

The Segment Anything Model (SAM) achieves remarkable performance in visual segmentation. The latest SAM3 extends promptable segmentation to concept-level prediction, broadening the scope of segmentation foundation models. While recent works reveal that SAM and SAM2 are vulnerable to adversarial examples, the robustness of SAM3 under the concept segmentation paradigm remains unexplored. In addition, existing adversarial attacks on SAM-series models exhibit limited cross-prompt transferability. To this end, we propose AdvPCS, a universal cross-prompt adversarial attack for Promptable Concept Segmentation (PCS), including a min-max prompt optimization strategy, a global–local perception deception attack, and a temporal transition deviation attack. Specifically, we first identify the hardest-to-attack prompts via min-max bilevel optimization. In the inner maximization, we enhance diversity over candidate point, box, and text prompts. In the outer minimization, we select prompts with the highest responses based on the confidence scores output by the detector. Given the selected prompts, we apply the perception deception attack to minimize both global and local existence probabilities under joint prompting, and employ the temporal memory misalignment attack to maximize inter-frame semantic inconsistency and corrupt memory pointers. Extensive experiments on four benchmark datasets show that a single universal adversarial perturbation (UAP) generated by our method generalizes across frames from different videos and achieves strong attack performance under point, box, and text prompts. In particular, it reduces the average mIoU of various PCS models on the SA-CO dataset to below 5%, demonstrating strong attack ability.

Byte-level byte-pair encoding (BBPE) tokenizers are attractive for large language models (LLMs) in multilingual settings because they cover all Unicode text. Under UTF-8, however, many scripts start from a higher byte-level fallback cost than English: when no learned merge covers a character $c$, the tokenizer must emit multiple byte-derived symbols. We call this worst-case pre-merge cost the encoding floor: $$ f(c) = |\\mathrm{utf8}(c)| \\in \\{1,2,3,4\\}. $$ A higher floor inflates token budgets, shrinks usable context, and increases per-request cost. Changing the underlying text encoding can reduce this gap, but a single global encoding can also make already-efficient English spans more expensive in mixed-script text. We propose Universal Byte-Level Encoding (UBE), a dual-alphabet tokenizer with deterministic per-character routing: $$ r(c) = \\begin{cases} \\mathrm{UTF\\text{-}8}, & |\\mathrm{utf8}(c)| \\leq 2,\\\\ \\mathrm{UTF\\text{-}16}, & |\\mathrm{utf8}(c)| > 2. \\end{cases} $$ Characters satisfying $|\\mathrm{utf8}(c)| \\leq 2$ stay on the UTF-8 path, while characters satisfying $|\\mathrm{utf8}(c)| > 2$ are routed through UTF-16. This reduces token costs for Basic Multilingual Plane (BMP) scripts whose characters typically satisfy $|\\mathrm{utf8}(c)| = 3$ and have the highest token premiums, without raising costs for already-efficient spans. UBE changes only the byte representation presented to byte-pair encoding (BPE): the BPE merge rule remains standard, and exact decoding $\\mathrm{decode}(\\mathrm{encode}(s)) = s$ is preserved. Across intrinsic tokenization evaluations, UBE reduces cross-lingual dispersion in English-normalized token-count ratios $\\pi(\\ell) = m(\\ell)/m(\\mathrm{en})$, lowering cross-script token-budget disparity. In multilingual language model experiments, UBE preserves LM quality within seed variability. In the main multilingual settings, UBE reduces token counts most for high-premium scripts and also slightly lowers English token counts; these reductions translate into more usable context under fixed token budgets and faster content-matched prompt processing.

Universal approximation theorems establish the expressive capacity of neural network architectures. For dynamical systems, existing results are limited to finite time horizons or systems with a globally stable equilibrium, leaving multistability and limit cycles unaddressed. We prove that Neural ODEs achieve $\varepsilon$-$\delta$ closeness, i.e., trajectories within error $\varepsilon$ except for initial conditions of measure $< \delta$, over the \emph{infinite} time horizon $[0,\infty)$ for three target classes: (1) Morse-Smale systems (a structurally stable class) with hyperbolic fixed points, (2) Morse-Smale systems with hyperbolic limit cycles via exact period matching, and (3) systems with normally hyperbolic continuous attractors via discretization. We further establish a temporal generalization bound: $\varepsilon$-$\delta$ closeness implies $L^p$ error $\leq \varepsilon^p + \delta \cdot D^p$ for all $t \geq 0$, bridging topological guarantees to training metrics. These results provide the first universal approximation framework for multistable infinite-horizon dynamics.

We propose an adaptive proximal gradient method for minimizing the sum of two functions, where one is a simple convex function, and the other belongs to one of the three classes: nonconvex smooth, convex nonsmooth, or convex smooth. The key feature of the method is an adaptive step size that accumulates historical gradient mapping norms in the denominator. Without any modification or knowledge of problem parameters, the method converges across all three problem classes under mild bounded-iterates and bounded-variance assumptions, with rates matching those of the proximal gradient method up to logarithmic factors, in both deterministic and stochastic settings. For the convex setting, we further propose an accelerated variant. It retains a similar near-optimal convergence rate for the nonsmooth case and achieves an improved rate of order $\widetilde{O}\big(1/t^2 + \sigma/\sqrt{t}\big)$ for the smooth case, which is optimal up to logarithmic factors. Notably, we develop new techniques for controlling the effect of stochastic noise, which are applicable across all three problem classes in the stochastic setting and enable simplified analysis.

Vision Transformers (ViT) dominate computer vision. However, their reliance on rigid patch projectors hinders transfer to Earth Observation (EO), where inputs vary widely in modality, scale, and resolution. We introduce UniverSat, a ViT-style backbone built around a Universal Patch Encoder that maps patches from arbitrary spatial, spectral, and temporal resolutions, and from both optical and non-optical sensors, into a shared embedding space with a single set of weights. This enables training one model on heterogeneous multimodal corpora in self-supervision, yielding robust sensor-agnostic spatial features. We validate this approach with strong results across classification and segmentation on standard EO benchmarks from GeoBench, PANGEABench, and SpectralEarth. The code and model will be released in open-source.

Pruning and quantization are two dominant techniques that effectively address the computational and storage burdens of large language model (LLM) inference on edge devices. Recently, one-shot pruning has gained particular attention for its ability to identify weight supports via optimization without retraining. However, integrating such methods seamlessly with quantization for further model lightweighting is non-trivial. The discreteness of quantization shatters their required variable continuity, inevitably collapsing this integration into a decoupled pruning-quantization pipeline with inherently suboptimal outcomes. To tackle this problem, we propose Quantization-aware One-shot Pruning (QOP), which directly optimizes the quantized weights under sparsity and discreteness constraints, thereby explicitly capturing the impact of quantization on pruning within its objective. Specifically, QOP generalizes the alternating direction method of multipliers to sparse-constrained discrete optimization, enabling the identification of the high-quality support and the update of quantized weights. Theoretically, via the well-conditioned approximation obtained by a slight perturbation, we establish convergence to a stationary feasible point and provide the convergence rate. Extensive experiments across various LLMs, sparsity levels, and quantization settings demonstrate that QOP consistently outperforms existing baselines in terms of both average accuracy and perplexity under weight quantization.

Generalization to harder reasoning and planning problems remains a central challenge for AI systems. Although both domains can be viewed as search problems where valid solutions are easier to verify than generate, existing methods often rely on domain-specific solvers or diffusion-based energy models tailored to individual settings. We introduce Energy Minimization Search} (EMS), a unified framework for reasoning and planning that learns a single time-invariant energy landscape. Unlike diffusion-based energy-based models (EBMs), which perform annealed inference over time-dependent landscapes, EMS performs direct gradient-based optimization on one static landscape using the same training objective and inference procedure across domains. Across reasoning tasks (graph coloring, N-Queens, 3-SAT) and planning tasks (path finding and maze solving), EMS matches or outperforms diffusion-based EBMs, domain-specific solvers, and diffusion-based planners while using substantially less inference compute.

Real-world applications of modern AI systems require generalization to both unseen environments and novel human users. The field of zero-shot coordination (ZSC) focused primarily on deploying AI to novel partners in otherwise familiar environments seen in the training. The popular population-based methods enable partner generalization by training the agent on a diversity of simulated human users (serving as training-time partners) in a fixed environment, aiming to cover the diverse behaviors of novel human users. Such a paradigm does not consider the more challenging realistic scenarios where both the users and the environment are unseen during training. Recently, training across environments has been explored in training a single self-play agent, which shows better generalization on both novel human users and unseen environments. In this work, we ask whether jointly modeling partner diversity and environment diversity yields stronger coordination with novel partners in novel environments. We introduce Cross Environment Co-Play (CECP), a two-stage framework that first trains a population of self-play agents across a distribution of procedurally generated environments, and then trains a cooperator policy against this partner population. To further couple social and environmental reasoning, we add a next-state prediction module that encourages representations to encode both partner behavior and task structure, and validate the design choice with ablation studies. We evaluate CECP on both a neuroscience-inspired cooperative foraging task, and the population ZSC benchmark Overcooked, under held-out partner and held-out environment conditions. CECP achieves the strongest performance among the compared baselines and attains state-of-the-art results under our evaluation protocol for human-AI collaboration. We provide further analysis using the model's latent representation encoded by the next-state prediction module, showing it can not only improve the performance of the cooperative AI, but also provide more interpretable latents, which demonstrate the inference of partner skill levels.

Unified multimodal models provide a natural and promising architecture for understanding diverse and complex real-world knowledge while generating high-quality images. However, they still rely primarily on frozen parametric knowledge, which makes them struggle with real-world image generation involving long-tail and knowledge-intensive concepts. Inspired by the broad success of agents on real-world tasks, we explore agentic modeling to address this limitation. Specifically, we present Unify-Agent, a unified multimodal agent for world-grounded image synthesis, which reframes image generation as an agentic pipeline consisting of prompt understanding, multimodal evidence searching, grounded recaptioning, and final synthesis. To train our model, we construct a tailored multimodal data pipeline and curate 143K high-quality agent trajectories for world-grounded image synthesis, enabling effective supervision over the full agentic generation process. We further introduce FactIP, a benchmark covering 12 categories of culturally significant and long-tail factual concepts that explicitly requires external knowledge grounding. Extensive experiments show that our proposed Unify-Agent substantially improves over its base unified model across diverse benchmarks and real world generation tasks, while approaching the world knowledge capabilities of the strongest closed-source models. As an early exploration of agent-based modeling for world-grounded image synthesis, our work highlights the value of tightly coupling reasoning, searching, and generation for reliable open-world agentic image synthesis.

We analyze generalization error, uniform stability, and uniform argument stability of gradient descent (GD) and stochastic gradient descent (SGD) over discrete parameter spaces, where each update involves deterministic or stochastic rounding. We show that deterministic rounding degrades the generalization error of GD on convex, Lipschitz, and smooth loss functions, increasing the rate from $O(T/n)$ to $O(T/\sqrt{n})$, and establish matching lower bounds. We further prove that uniform stability of GD becomes $\Omega(T)$, showing that stability-based generalization bounds are vacuous in this setting. In contrast, for the same losses, stochastic gradient descent with deterministic rounding admits nontrivial uniform stability guarantees, which differ qualitatively from the real-valued case and exhibit distinct dependencies on the number of iterations and the dimension: we prove tight bounds $O(T/n)$ for one dimension and $O(T^2/n)$ for higher dimensions. We also show that stochastic rounding can introduce generalization error that increases with the dimension; such a phenomenon is absent in standard real-valued optimization and in the deterministic rounding case. Finally, we provide upper bounds on uniform argument stability for stochastic rounding schemes and show that these bounds are tight when the loss can be represented as a sum of coordinate-wise functions.

Spectral gradient descent (SpecGD) orthogonalizes matrix parameter updates and has inspired practical optimizers such as Muon. They often perform well in large language model training, but their dynamics remain poorly understood, especially in factorized parameterizations where the product matrix does not receive orthogonalized updates. We study such dynamics through matrix factorization (MF), where the orthogonalization is applied separately to the factor updates. We analyze spectral gradient flow (SpecGF)—a continuous-time analog of SpecGD—in the low-rank MF setting and prove "equal-rate" dynamics: all singular values grow at equal rates up to small deviations. Consequently, smaller singular values attain their target values earlier than larger ones, contrasting with the largest-first stepwise learning observed in standard gradient flow. Moreover, we prove that SpecGF in our setting converges to global minima from almost all initializations, provided the factor norms remain bounded; with $\ell_2$ regularization, we obtain global convergence. Empirically, we observe that LoRA fine-tuning with orthogonalization-based optimizers including Muon exhibit near-uniform growth in the product of LoRA adapters, consistent with the mechanism predicted by our MF analysis.

Discrete diffusion models are typically trained by predicting the clean data given a noisy observation, which is then used with a certain choice of reverse transition parameterization. Two choices coexist in the literature, which we refer to as marginalization and plug-in. Although they coincide in the case of masked diffusion models (MDMs), we revisit this question for uniform diffusion models (UDMs), where the distinction matters. We show that the plug-in parameterization is not optimized by the denoising posterior but by a \emph{leave-one-out} posterior that removes the influence of the local observation. We give a thorough characterization of the resulting training target, together with exact conversion formulas between the denoiser, the \emph{leave-one-out} posterior, and the score. As a by-product, the leave-one-out characterization yields a Gibbs-based predictor–corrector scheme at no additional training cost. We further propose an absorbing-state construction of uniform diffusion, whose denoiser decomposes into simpler absorbing denoising posteriors, and which recovers the full UDM joint law following a simple resampling step. Empirically, we show that these design choices interact strongly with optimization and generative quality; on language modeling, the \emph{leave-one-out} denoiser improves over other parameterizations.

Multi-agent language-model systems typically rely on a human designer or external orchestrator to fix the agent population, roles, and workflow—a single point of failure that places coordination decisions outside the team executing them. Existing approaches address fragments of this problem: communication standards lack coordination logic, automated workflow-search systems concentrate authority in a single planner, and decentralized peers lack explicit governance, handoff validation, and bounded repair. To close this gap, we introduce the Unified Resource-Grounded Coordination Protocol (URCP), an orchestrator-free protocol in which a fixed pool of black-box models and executable resources collectively decides task framing, role formation, workflow topology, assignment, checkpoint validation, and final completion through supermajority voting under a single decision rule Φq. A five-level repair hierarchy reopens only the affected part of the workflow on failure; under standard timing and communication assumptions, every run terminates with an explicit verdict. Empirically, with twelve heterogeneous models against eight baselines on MultiAgentBench, τ-bench, MemoryAgentBench, GAIA, and REALM-Bench, URCP exceeds the strongest baseline by +9.7 / +7.9 / +10.6 points on the three primary benchmarks, terminates deadlock-free in 100% of runs, and remains stable up to 20% agent failure. Trace analyses show that the gains come from voted role concentration on frontier models (73.9% of critical-path roles), frequent critic/verifier insertion (63.5% of accepted schemas with at least three roles), and bounded repair rather than from model strength alone.

Geometry estimation from perspective images has greatly advanced, maturing to the point where off-the-shelf foundation models are able to reconstruct 3D scene structure not only from multi-view imagery, but even from a single view. A natural extension is 3D reconstruction from panoramas, with the exciting prospect of recovering a full $360^\circ$ scene from a single panoramic image. In this work, we introduce PaGeR (Panoramic Geometry Reconstruction), a framework to lift powerful 3D foundation models, designed for perspective imagery, to the panorama domain. Our strategy is to start from a pretrained transformer for 3D reconstruction and turn it into a unified high-performance model that predicts scale-invariant depth, metric depth, sky masks and surface normals from both perspective and omnidirectional images, a single forward pass. By keeping architectural changes to a minimum and mixing perspective and panoramic images during training, Pager retains the rich 3D prior of the underlying foundation model while learning to also estimate geometrically consistent $360^\circ$ scenes from single panoramas. We extensively test our method in both indoor and outddor environments and find that it delivers state-of-the-art performance and excellent zero-shot performance across a wide range of scenes.

The efficiency and fidelity of 3D Gaussian Splatting critically depend on adaptive density control, which grows and suppresses Gaussian primitives throughout optimization. However, the standard pipeline still relies on heuristic view-space gradients, scale thresholds, and global opacity resets, rather than explicitly assessing the loss-reducing effect of each split, clone, or opacity update. This mismatch can allocate primitives to already-sufficient regions, miss under-reconstructed structures, and destabilize training through blanket opacity changes. In this work, we recast adaptive density control as a collection of local loss-reduction decisions. We derive efficient Taylor-based per-Gaussian surrogates that predict the objective change caused by three primitive-level operations: splitting, cloning, and opacity decay. This yields unified loss-aware framework with three components. First, we extend second-order splitting to a normalized joint position-scale space, enabling children to adapt not only their locations but also their anisotropic extents. Second, we formulate duplicate cloning as a surrogate minimization problem, selecting only Gaussians whose duplicated contribution is predicted to improve reconstruction and assigning each child a principled opacity. Third, we replace global opacity reset with selective soft opacity decay that weakens harmful primitives while preserving useful ones. Across standard 3DGS benchmarks, our method produces more targeted primitive allocation and achieves improved rendering quality with comparable or smaller Gaussian budgets compared to baselines.

Stochastic estimators are fundamental to large-scale optimization, where population quantities must be inferred from noisy oracle observations. Although influential methods such as momentum, SPIDER, STORM, and PAGE have been highly successful, their analyses are largely estimator-specific and expectation-based, obscuring the structural tradeoffs that determine reliability. In this paper, we develop a unified framework for stochastic variance-reduced estimation based on a recursion with three components: memory retention, reset probability, and a correction term for iterate movement. This framework recovers several classical estimators, motivates new second-order variants, and yields a bias-variance decomposition of estimation error. Our main result is a unified high-probability bound proved using a new dimension-free vector-valued Freedman inequality, valid for smooth normed spaces involving random sums of vector martingales. The result applies in both Euclidean and non-Euclidean settings, including the analysis of mirror-descent-based methods in Banach spaces. As applications, we obtain high-probability oracle complexities for unconstrained optimization with mirror descent, establishing the logarithmic dependence on the confidence level. We also derive the first $\tilde{\mathcal{O}}(\varepsilon^{-3})$ oracle-complexity bounds for stochastic optimization with expectation constraints, improving upon the existing $\tilde{\mathcal{O}}(\varepsilon^{-4})$ complexity by leveraging variance-reduced estimation for the first time in this setting.

Understanding 4D scenes is a core challenge in visual intelligence, requiring models to jointly capture scene geometry, appearance, and dynamics from partial observations. Existing approaches, however, are subdivided into feedforward models that are used for direct geometric prediction and generative models for visual synthesis. We propose a unified generative-predictive framework for 4D scene understanding that jointly models RGB video and geometric representations such as depth and camera pose with a single joint generative model. Our generative formulation enables us to learn from diverse, heterogeneous datasets with different subsets of geometric and visual annotations, and enables us to condition on arbitrary subsets of inputs, allowing a single model to perform cross-modal generation, geometric prediction, and future scene inference from partial observations. In addition, our generative approach enables test-time search over multiple candidate scene completions, improving geometric prediction from sparse observations. Overall, we illustrate the efficacy of our generative-predictive framework on a suite of generation and prediction tasks.

Speculative decoding accelerates Large Language Models via draft-then-verify, where verification can be framed as an Optimal Transport (OT) problem. Existing approaches typically handle multi-draft and multi-step aspects in isolation, applying either flat OT to single-step drafts or per-token rejection sampling to tree-structured candidates. This separation leaves the joint regime (where multi-step dependencies meet multi-draft branching) poorly optimized, as local verification rules fail to exploit the coupling between horizontal and vertical dimensions of candidate trees. In this paper, we propose a unified perspective that casts tree-based verification as a conditional OT problem. Our key insight is that vertical dependencies can be abstracted through prefix acceptance probabilities, which act as dynamic scaling factors to actively guide horizontal draft selection. Based on this principle, we introduce UniVer, a verification algorithm that jointly optimizes across tree levels by composing local optimal transport plans under prefix constraints. We prove that UniVer remains lossless and achieves the optimal acceptance rate under the proposed conditional framework. Extensive experiments across different tasks and models demonstrate that UniVer improves acceptance length by 4.2\% to 8.5\% over standard recursive rejection sampling without replacement, while maintaining exact distributional alignment with the target model.