NeurIPS 2026 论文列表

The rapid proliferation of personalization techniques in Stable Diffusion has driven the widespread deployment of adversarial perturbations as a protective measure against unauthorized data mimicry. Existing diffusion-based purification methods primarily rely on the noising and denoising paradigm and have proven somewhat effective in neutralizing these protections. However, they suffer from a severe trade-off between purification efficacy and perceptual fidelity, especially under large perturbation budgets. To mitigate this problem, in this paper, we propose Dynamic Sparse Semantic Anchoring (DSSA), a novel purification framework. By starting the reverse denoising process from pure Gaussian noise, DSSA removes the initial adversarial perturbations and circumvents the dilemma of selecting an optimal noising level. To prevent the loss of original semantics caused by pure noise generation, we propose a dynamic sparse anchoring strategy that directly integrates sparse pixels from the adversarial sample to guide the reconstruction. Additionally, we design a perception-guided early-stopping mechanism to prevent the perturbation resurgence caused by this direct pixel integration. Extensive experiments across diverse datasets and protection schemes demonstrate that our method consistently outperforms existing baselines, achieving state-of-the-art performance in efficacy and fidelity, successfully neutralizing even advanced adaptive protections.

Decentralized training of deep learning models is widely used to enable data privacy and on-device learning over networks. In realistic scenarios, heterogeneity across clients' local data poses an optimization challenge and can severely degrade test accuracy, especially on sparse communication graphs. Building on recent advances in centralized primal averaging, we propose Decentralized Primal Averaging (DPA), a decentralized learning algorithm that combines primal averaging with a local quasi-global momentum estimate. DPA maintains three coupled sequences for optimization, gradient evaluation, and output averaging, while using consecutive mixed iterates to construct a network-aware momentum direction without transmitting an auxiliary tracking variable. We prove a nonconvex stationarity guarantee for DPA's averaged output model and establish a linear-speedup result up to network- and heterogeneity-dependent residual terms. We also introduce DPA-1G, a one-gossip-per-iteration variant that matches the communication budget of DSGD-style baselines. Experiments following the GUT decentralized image-classification benchmark show that DPA outperforms DSGD, gradient tracking, QG-DSGDm, momentum tracking, and QG-GUTm across datasets, architectures, topologies, and heterogeneity levels, with the largest gains under high heterogeneity. DPA-1G matches or exceeds the one-gossip baselines at the same per-iteration communication budget.

Asynchronous reinforcement learning (RL) has become a critical paradigm for accelerating large-scale LLM post-training in industrial settings, yet it faces a structural \emph{long-tail dilemma}: rollout efficiency is bottlenecked by a small number of the longest trajectories, which are precisely the most valuable ones for RL training. Existing approaches alleviate this dilemma at the cost of either system overhead (e.g., re-prefill in partial-rollout methods) or algorithmic compromises (e.g., discarded long trajectories in replication-based methods). We trace these tradeoffs to a common assumption: the rollout cluster serves a single policy version at any moment. We propose \textbf{DORA} (\textbf{D}ynamic \textbf{OR}chestration for \textbf{A}synchronous Rollout), which breaks this assumption by maintaining multiple policy versions concurrently within the rollout cluster. DORA combines three mechanisms: \emph{multi-version streaming training} that decouples trajectory completion from batch boundaries, a centralized \emph{load-balancing orchestrator} that re-partitions resources across versions, and \emph{zero-re-prefill migration} that transfers KV-Cache directly across same-version instances. Experiments on open-source benchmarks show that DORA achieves up to $2.12\times$ end-to-end throughput improvement and $8.2\times$ rollout-stage acceleration over synchronous training while preserving convergence parity. \textbf{In real-world production deployment} with thousands of accelerators, DORA achieves up to $6.2\times$ rollout speedup and produces competitive open-source LLMs.

We present DMax, a new paradigm for efficient Diffusion Language Models (dLLMs). It mitigates error accumulation in parallel decoding, enabling aggressive decoding parallelism while preserving generation quality. Unlike conventional masked dLLMs that decode through a binary mask-to-token transition, DMax reformulates decoding as a progressive self-refinement from mask embeddings to token embeddings. At the core of our approach is On-Policy Uniform Training, a novel training strategy that efficiently unifies masked and uniform dLLMs, equipping the model to recover clean tokens from both masked inputs and its own erroneous predictions. Building on this foundation, we further propose Soft Parallel Decoding. We represent each intermediate decoding state as an interpolation between the predicted token embedding and the mask embedding, enabling iterative self-revising in embedding space. Extensive experiments across a variety of benchmarks demonstrate the effectiveness of DMax. Compared with the original LLaDA-2.0-mini, our method improves TPF on GSM8K from 2.04 to 5.47 while preserving accuracy. On MBPP, it increases TPF from 2.71 to 5.86 while maintaining comparable performance. On one single H200 GPU using the SGLang framework, our 16B model achieves an average of 955 TPS at a batch size of 1.

Multimodal Large Language Models (MLLMs) have shown strong multimodal instruction-following ability, but adapting them to diverse visual-language domains typically assumes centralized data access and costly joint training. This is restrictive when data is distributed across private, domain-specific, or permission-limited clients. To this end, we propose DISTMOE, a mixture-of-experts (MoE) approach for distributed visual instruction tuning. In each layer of the language decoder it augments the public feedforward network (FFN) with a client-specific private FFN expert, with the goal to acquire domain-specific knowledge. However, independent expert training causes the private FFNs to learn representation of different scale and magnitudes, making merging the experts difficult. To reduce client-specific drift, we introduce a public-anchored expert composition stage that updates only routers and lightweight private projection adapters on a mix of local client data and public data, via an isotropic regularization loss, therefore making it rehearsal-free. During inference, DISTMOE performs modular routing over public and private experts, enabling token-wise domain composition without explicit domain labels. Experiments across diverse visual-language benchmarks show that DISTMOE enables flexible expert reuse, effective domain adaptation, and competitive performance while preserving modular control over client-specific knowledge.

We introduce DIAL (Directional Intervention, Asymptotically Limited), a parameter-efficient adapter that adds a bounded, monotone, scalar-controlled residual $\delta(s,c) = \sigma(g) \odot \tanh(c/T) \odot \tanh(Dz)$ with $z = D^\top \phi(s)$ to a base language model's hidden state. The architectural form yields a closed-form upper bound $\lVert \delta \rVert_\infty \le \sigma(g_{\max})$ on the residual $L_\infty$ norm, computable from model weights alone. Across 5 base models (3.8B to 70B parameters, 4 architecture families; one checkpoint per family, with 5-seed csweep variance at Llama-3.1-8B), the residual saturates at the predicted bound with mean ratio $1.000 \pm 0.001$ (worst-case family deviation 0.2 percentage points). On Llama-3.1-8B under GPT-4o judge ($n=100$ per bench, zero content-filter blocks), DIAL exceeds a FiLM baseline trained with the same multi-$c$ objective on every benchmark and judge. Semantic vs strict judge: HarmBench DIAL +0.52 / +0.55 vs FiLM +0.44 / +0.39; AdvBench DIAL +0.63 / +0.66 vs FiLM +0.46 / +0.50; XSTest DIAL +0.33 / +0.25 vs FiLM +0.26 / +0.22; both judges agree on the ranking. Outside training range, DIAL's residual saturates at $\sigma(g_{\max})$ while FiLM's grows unboundedly: at $c = 10^4$, FiLM's residual $L_\infty$ norm is 534 times larger than DIAL's. The bound holds across all control values, not only trained ones; concrete deployment scenarios (configuration error, multi-tenant policy ceilings, adversarial composition) are where this distinction matters in practice. The bounded-monotone form produces analogous saturation behavior on two continuous-control RL environments (HC-Vel $n=35$, Ant-Goal $n=30$) under matched oracle-$c$; the prior fails when $c$ must be inferred online (meta-RL). Ablations confirm both the bound and the saturating temperature are necessary; a frozen-gate variant retains the bound at lower controllability magnitude. We release adapter train reports and canonical bench generations, algebraic derivations (with a SymPy reproducibility script), the full GPT-4o judge cache (semantic and strict), and a human-labeled validation set ($n=50$, Cohen's kappa = 0.895). Full adapter checkpoints are deferred to camera-ready release.

High-compression tokenizers are essential for scaling latent image generative models. However, aggressive compression creates a fundamental tradeoff between reconstruction fidelity and generation efficiency: high compression image encoder always increase the learning diffucity of diffusion training, resulting slow model convergence. Recent representation autoencoders speed-up the diffusion training by improving the latent feature's expressive capability by replacing VAE encoders with pretrained semantic encoders, yet they are typically limited to moderate compression and lose pixel-level details necessary for faithful reconstruction. To achieve both high compression and fast diffusion training, we propose DC-SAE, a Decoupled Compact Semantic Autoencoder designed for high-compression image generation with accelerated diffusion model convergence. DC-SAE consists of two key components: (1) a macro-level architecture design that leverages semantic encoders to enable higher compression ratios, and (2) a pixel-level encoder that preserves low-level details, ensuring high-fidelity image reconstruction. We empirically demonstrate that DC-SAE performs strongly on image generation tasks, achieving both compact latent representations and efficient training dynamics. % Specifically, on ImageNet dataset with $512 \times 512$ resolution, DC-SAE achieves $32\times$ spatial compression, with \textbf{29.79} PSNR and \textbf{3.05} gFID, substantially outperforming the previous state-of-the-art high-compression tokenizer baselines DC-AE by 13.5\% and 59.7\% on PSNR and gFID respectively, maintaining comparable throughput and faster diffusion model training convergence.

Modeling neurodegenerative pathology from observational cohorts is challenging because subjects are typically observed as unpaired endpoint snapshots rather than aligned longitudinal trajectories. Existing approaches to pathology transport are either endpoint-conditioned or population-level, yet neither extreme alone yields the best terminal-distribution fit. We introduce the Doob-Bridge Posterior Sampler (DBPS), a controlled-SDE posterior sampler with balanced endpoint-population control that blends individual endpoint guidance with population-level guidance. This ρ-weighted controller, obtained as the Cole–Hopf log-gradient of a geometric interpolation between endpoint and population desirability functions, is the exact optimal feedback policy of a linearly-solvable stochastic optimal control problem with closed-form terminal and running costs. Endpoint-conditioned bridge sampling and population Doob sampling are the ρ = 0 and ρ = 1 boundary cases; ρ ∈ (0, 1) realizes the genuine balanced regime. We evaluate DBPS on tau PET pathology transport in Alzheimer’s disease across ADNI and NACC cohorts (84 brain regions). Compared to representative baselines from all major unpaired-transport families, DBPS achieves state-of-the-art performance across multi-cohort settings (union training: SWD 0.760 vs. 0.821; held-out ADNI: 0.822 vs. 0.859), and remains competitive in single-cohort training (ADNI: 0.868 vs. 0.876). Beyond distribution matching, the learned controls recover canonical Braak-stage pathology organization without anatomical supervision, suggesting that balanced SOC objectives capture biologically meaningful structure in latent neurodegenerative progression.

Symbolic regression (SR) aims to discover interpretable mathematical expressions from data and plays a key role in scientific discovery. However, existing methods face a common bottleneck: the enormous search space and severe combinatorial explosion make it difficult to achieve a favorable trade-off between accuracy and algorithmic efficiency. We argue that, in addition to designing increasingly complex solver-specific SR strategies, an equally promising direction is to equip them with a shareable auxiliary strategy that can provide reliably search guidance. To this end, We propose DASS, a solver-agnostic Dynamic Auxiliary Search Strategy that improves SR by constructing and refining a high-potential auxiliary search subspace. DASS represents the auxiliary search space as a functional subspace spanned by a set of high-potential basis function terms. It estimates term utility through multi-environment evaluation, filters unstable guidance, and updates terms' potential score with a Gibbs posterior, thereby calibrating the subspace to provide more reliable guidance for downstream solvers. The High-quality solutions produced by the solver are used to update the set of basis terms. This forms a virtuous closed-loop optimization process: the subspace guides the downstream solver, and the solver feedback reshapes the subspace for subsequent exploration. DASS can be seamlessly integrated with various types of SR solvers. Extensive experiments on LLM-SRBench show that DASS significantly improves the accuracy of original solvers with a manageable increase in runtime.

Physical control systems, such as building HVAC systems, thermal processes, and power-grid assets, often involve delayed dynamics, safety constraints, and costly online interaction. Because the effect of a control action may appear after several time steps, effective control often requires reasoning over histories of states, actions, and outcomes. Pretrained sequence controllers, such as Decision Transformers, provide a natural way to model this history-dependent control problem by learning policies from offline trajectories, reducing the need for expensive online retraining. However, their performance can degrade when deployment conditions differ from the training data. In building control, for example, actuator wear, sensor drift, occupancy changes, and weather variation can all affect the observed response of the system to the same control action. Online adaptation is therefore needed, but raw prediction error is not always a reliable signal for updating the controller. The key challenge is to identify which part of the observed mismatch is action-sensitive and should guide policy adaptation. To address this challenge, we propose DARPAN, a Drift-Aware Residual Policy Adapter Network for pretrained physical control. DARPAN keeps the pretrained Decision Transformer frozen and performs online adaptation only through lightweight residual adapters, preserving the offline policy and limiting the number of trainable parameters. Instead of updating from raw prediction error, DARPAN uses a controllability filter trained in simulation to extract the action-sensitive part of the residual and filter out mismatch that is weakly related to the controller's actions. During deployment, a frozen predictor estimates the next state, the controllability filter produces a filtered residual, and a short history of filtered residuals forms a drift signature. This signature is used to select an existing adapter or create a new one when a persistent drift pattern appears. We benchmark DARPAN on SustainGym under actuator faults, sensor drift, delays, and external disturbances. DARPAN reduces the drift-induced performance loss by 31.7% and uses 63% fewer online parameters than Online DT. We release anonymized code and experiment configurations at https://huggingface.co/Darpan2026.

Reinforcement learning with verifiable rewards (RLVR) has become a dominant recipe for improving mathematical and code reasoning in open-weight language models, but existing methods rely on fixed hyperparameters despite highly non-stationary training dynamics. We present DACE-RL (Diagnosis-Aware Compute-Efficient RL), a closed-loop framework that promotes training diagnostics to first-class signals. DACE-RL tracks four lightweight statistics—token entropy, sample diversity, advantage saturation, and answer consistency—and feeds their EMA-smoothed state to a controller that adaptively sets per-prompt rollout budget, sampling temperature, asymmetric clipping bounds, and entropy regularization strength. We formalize adaptive rollout allocation as a difficulty-conditioned multi-armed bandit and provide a regret bound that improves over uniform-budget GRPO under heterogeneous prompt difficulty; we also give a stability result for the dual-loop entropy-diversity controller. Across three small base models (Qwen2.5-Math-1.5B, Qwen2.5-7B, and Llama-3.1-8B-Instruct) and six math reasoning benchmarks, DACE-RL matches or improves Pass@1 while reducing rollout-GPU-hours by 38–52%, and improves Pass@32 by 4.7 absolute points on average. Pilot analysis further shows that diagnostic inflection signals appear 100–300 update steps before reward plateau, indicating that diagnosis-aware control is a key lever for compute-efficient RLVR.

Speculative decoding accelerates LLM inference by having a small drafter propose tokens that a larger target model verifies in parallel. Current SOTA diffusion-based parallel drafters such as dFlash predict the full $B$-token block in one forward pass, allowing deeper drafters and higher accepted length. However, across the field, from autoregressive tree drafters to parallel block drafters, training losses use fixed per-position weights that do not adapt to the drafter's current bottleneck position. We derive per-position training weights directly from the expected accepted block length, so that each position's weight reflects its actual contribution to acceptance. The resulting loss, D-PACE (Dynamic Position-Aware Cross-Entropy), automatically identifies where the block's acceptance bottleneck lies and shifts training signal there as the drafter improves. D-PACE consistently improves both wall-clock speedup and accepted length across diverse benchmarks and positions as a drop-in replacement with negligible training overhead and requires no changes to the drafter architecture or inference pipeline, making it broadly applicable to general model architectures.

Paired translation between quasiperiodic physiological waveforms (i.e., recovering a target oscillatory signal from the source) is central to the interpretation of cardiovascular signals derived from wearables placed at different body locations. This source-to-target mapping in these problems carries inherent geometric structure: the phase wraps around the cycle and must be treated as a circular variable, the amplitude remains strictly positive, and the beat-to-beat alignment can drift unpredictably across cycles and subjects. While deep neural networks have been used for phase estimation and complex-valued signal modeling, prior work does not explicitly learn phase transport between paired signals. Consequently, neither endpoint-supervised regression nor the standard affine path used in flow matching accounts for this phase--amplitude structure. We introduce \emph{cylindrical geodesic flow matching} for paired cardiovascular waveform translation. We show that the standard affine path used in flow matching distorts intermediate amplitude and instantaneous frequency when interpolating between quasiperiodic signals; replacing it with a closed-form geodesic on the phase--amplitude cylinder eliminates these artifacts and converts each training pair into dense, geometry-consistent velocity supervision. On zero-shot photoplethysmography and limited-support seismocardiography adaptation benchmarks, our method consistently outperforms interpolation baselines and matches or exceeds direct supervised prediction, reducing Hilbert Transform, $L_2$, and Dynamic Time Warping distance by up to ${\sim}15\%$ over the strongest competing baseline. These results suggest that bridge geometry is a critical inductive bias for flow matching on oscillatory signal translation.

Reconstructing patient-specific cyclic cardiac motion at arbitrary phases is central to dynamic cardiac assessment, underpinning measurements such as ejection fraction, stroke volume, and regional myocardial deformation. Existing methods---pair-wise registration, time-conditioned decoders, and coordinate-based implicit fields---share a common implicit assumption: the cardiac cycle is treated as a sequence of discrete phases, with temporal structure stitched together through learned mappings supervised at the observed frames. Yet the cycle is, by physical fact, a closed periodic process. We propose \textbf{CycleSpectra}, a spectral motion representation that takes the cyclic structure as a starting point of the representation rather than as a property to be learned. Given a pair of sparsely sampled phase observations, the model learns the spectral structure of the underlying cyclic cardiac motion in a low-dimensional latent space, and recovers the displacement at any phase through a fixed analytic synthesis layer---time never enters the network as a learnable signal, only as a coordinate the readout takes by construction. To support evaluation, we collect a thin-slice phase-resolved 4D cardiac CT cohort and benchmark under both canonical and non-canonical input protocols, reporting whole-cycle motion-fidelity metrics alongside standard image and anatomical metrics. Under arbitrary two-frame input, CycleSpectra achieves the lowest ejection-fraction error and highest ventricular volume--time curve correlation among recent baselines, while remaining competitive on image fidelity. Beyond two-frame inference, the proposed framework supports test-time multi-frame latent fusion for higher accuracy, validated on both the in-house cohort and a cross-modality 4D cardiac MR cohort. Code will be available.

Submodular functions---functions exhibiting diminishing returns---are central to machine learning. When the objective is monotone and non-negative, the greedy algorithm achieves a tight $63\%$ approximation. But many practical objectives incorporate costs that make them negative on some inputs, and all existing multiplicative guarantees require non-negativity. Prior work handles negativity through additive bounds for the special class of decomposable functions and non-monotonicity through partial-monotonicity parameters, but these address each difficulty in isolation and neither extends the classical structural theory. We extend curvature---a parameter measuring how far a function deviates from linearity---to all submodular functions, handling both non-monotonicity and negativity through a single classical concept. A greedy algorithm with pruning achieves a curvature-controlled multiplicative ratio for any submodular function, including those taking negative values---the first such guarantee beyond monotonicity and non-negativity. In the non-monotone regime $1 \le c_g < 2.2$, the bound strictly beats the best known uniform ratio of $0.401$ (for non-negative $f$), and it recovers the classical $(1-e^{-c_g})/c_g$ guarantee for monotone functions. A multilinear-extension variant extends the framework to general combinatorial constraints via multilinear relaxation. Experiments on cost-penalized experimental design, coverage, feature selection, and a curvature sweep on Multi-News passage selection support the theory.

Molecular conjugates, including PROTACs and peptide-drug conjugates (PDCs), derive their function from the joint behavior of multiple coupled components, yet most generative approaches design these components independently and combine them only after generation. Such staged pipelines ignore cross-component dependencies and often produce conjugates that are chemically invalid, fall outside empirical conjugate distributions, or lose function upon assembly. We introduce $\textbf{C}$o$\textbf{up}$led $\textbf{O}$bjective-Guided Discrete $\textbf{F}$lows for $\textbf{Mo}$lecular $\textbf{C}$onjugate $\textbf{A}$ssembly $(\textbf{CupOFMoCA})$, a discrete generative framework that formulates conjugate design as a constrained, coupled generation problem. CupOFMoCA restricts generative trajectories to a chemically feasible conjugate manifold and biases local transitions using target-specific activity predictors, ensuring all components remain mutually compatible throughout generation. We show that coupling constraints and objective guidance enable anticipatory design that preserves post-assembly predicted activity and produces structurally realistic conjugates across both PDC and PROTAC settings, outperforming staged baselines, including LinkerNet, DiffLinker, and DiffPROTACs for PROTACs, across assembly validity, predicted activity, and physicochemical property ranges. These results demonstrate that explicit coupling and constraint enforcement are sufficient to recover functional conjugates across conjugate classes, and provide a principled foundation for generative modeling where function emerges only at the level of the assembled system. Our anonymous code repository can be found at https://anonymous.4open.science/r/Cupofmoca-Neurips.

Merging separately trained LoRA adapters is a practical alternative to joint multi-task training, but it often hurts performance. Existing methods usually treat the LoRA update $\Delta W = BA$ as a single object and do not distinguish the two LoRA matrices. We show that the main source of LoRA merge interference comes from the output-side matrix $B$. Across tasks, $B$ repeatedly uses a small set of shared directions, while $A$ remains much more task-specific. As a result, the merged adapter overemphasizes these shared directions, and task-specific information is lost. We propose Pico (Pre-merge interference calibration in output-space), a data-free method that calibrates $B$ before merge by downscaling over-shared directions and then rescaling the merged update. Pico plugs directly into existing merging methods such as Task Arithmetic, TIES, and TSV-M. Across eight different benchmarks from math, coding, finance, and medical domains, Pico improves average accuracy by 3.4-8.3 points over the corresponding base method and achieves the best overall average performance. Pico also enables merged adapters to outperform the LoRA trained with all task data. These results show that LoRA merging works better when the two LoRA matrices are treated separately.

Production deployments often swap between different-sized models in a family for cost-quality cascading, mid-conversation switching, and routing, and each swap forces the receiver to repay the prefill from scratch. We propose cross-model KV cache transfer, where the receiver reuses the source’s KV cache, skipping prefill. We find that cross-model KV has substantial linear structure across matched-KV pairs, where source and target share KV head count and per-head dimension. On Qwen3 14B→32B, one source layer explains 56% of variance in the target’s keys and 32% in values, rising to 79% and 65% with multiple source layers. Building on this, we design a closed-form ridge mapper that operates per head and proceeds in three steps. First, for each target layer we select the top-kmost predictive source layers and concatenate their KV as input. Second, we strip RoPE from the keys before mapping, so the fit is position-free and reusable across context lengths. Third, we fit ridge regression on a small calibration set of 500 FineWeb-Edu sequences of 1,024 tokens each. Surprisingly, across six pairs in three families, this linear mapper retains 71–98% of the receiver’s standalone-prefill accuracy on four pairs, while two degrade sharply. A nonlinear MLP recovers up to +22 pp HellaSwag retention on the failures. The mapper runs 3–31×faster than re-prefill and remains stable across multi-turn handoff, making cross-model KV cache transfer practical.

Neural Posterior Estimation (NPE) trains an amortized conditional density estimator to approximate posterior distributions in simulation-based inference. Each training pair is a single draw from the joint distribution of latent and observation, so posterior variance at an observation must be inferred from variation across pairs rather than within them. However, with a sufficiently flexible variational distribution, the NPE training objective is unbounded, and standard optimizers yield arbitrarily small variance. Regularizers like early stopping, dropout, and weight decay constrain network capacity without targeting the correct posterior variance. We propose Cross-Fitting NPE (CF-NPE), which separates mean estimation from higher-moment estimation via sample splitting. Concretely, CF-NPE estimates a conditional-mean ensemble by $K$-fold cross-fitting, then fits a conditional density to the resulting out-of-fold residuals. The conditional variance of an out-of-fold residual is bounded below by the true posterior variance, giving CF-NPE a well-defined population-level target that standard NPE lacks. Across eight synthetic benchmarks and a cosmological application, CF-NPE achieves higher held-out log-likelihood than standard NPE, with gains that grow with the latent dimension.