NeurIPS 2026 论文列表

Born-Oppenheimer molecular dynamics delivers quantum accurate trajectories by evolving nuclei on the electronic ground-state surface while retaining electronic observables such as dipoles and polarizabilities. In Kohn-Sham density functional theory, this typically relies on an iterative self consistent field solver at each time step. Machine learning can predict electronic quantities or improve SCF initialization, yet dynamics remain sensitive to residual self consistency error, which can yield non conservative forces and energy drift, and repeated diagonalization limits differentiable and accelerator efficient implementations. We propose a gauge-symmetric dual Lagrangian framework that avoids SCF loops by propagating the physically meaningful electronic state as the occupied subspace and its density projector on a Grassmann manifold. A residual driven update stabilized by Rayleigh type dissipation with time decaying inertia yields a closed evolution of the subspace, its conjugate momentum, and the projector. Coupled with a neural gauge-symmetric Hamiltonian with analytic coordinate derivatives, the method provides closed form forces with Pulay corrections and enables stable quantum BOMD on molecular simulation benchmarks.

State Space Models (SSMs) compress sequence history into a bounded recurrent state, making the resulting memory law a central architectural choice for long-context performance. Most modern SSMs rely on ODE-based dynamics that lead to exponential forgetting, limiting their ability to retain information over broad temporal ranges. We introduce FRAC, a selective SSM architecture derived from fractional dynamics that replaces this exponential decay with power-law long memory. To make fractional dynamics practical, FRAC approximates the heavy-tailed target kernel with a finite-state, log-spaced sum of exponential modes. This construction turns fractional memory into an efficient recurrent module with parallel training and prefill, while retaining bounded-state autoregressive decoding. Extensive experiments, including 1.3B-parameter language modeling, demonstrate that FRAC consistently improves long-context performance over state-of-the-art SSM baselines while staying competitive on short-context. These results show that fractional dynamics provide a practical and effective prior for long-context SSMs.

Transformer models rely on attention mechanism to capture long-range dependencies but suffer from quadratic complexity, limiting their scalability to long sequences. Kernel-based linear attention reduces this complexity but typically relies on fixed or weakly learnable kernels, restricting expressiveness and performance. In this work, we propose Flexformer, a flexible linear Transformer that learns attention kernels in a fully data-driven manner. Flexformer builds on random Fourier feature-based linear attention and treats spectral frequencies as trainable parameters, enabling the model to learn a broad family of attention kernels. We develop both stationary and nonstationary variants, with the latter offering strictly greater expressiveness. Extensive experiments on language modeling and sequence classification demonstrate that Flexformer consistently outperforms baselines. Moreover, Flexformer can be effectively distilled from pretrained Transformers to recover softmax attention and exhibits strong kernel transferability across domains, achieving both high efficiency and competitive performance on long-sequence tasks.

Masked Generative Models (MGMs) enable parallel decoding and achieve strong performance across modalities, but require full-sequence bidirectional transformers at every step, making training costly and degrading quality under low sampling budgets. Existing work improves efficiency via better samplers or fixed-depth backbones, but does not vary the depth of the denoiser across sampling steps. We introduce Fixed-Point Masked Generative Models (FP-MGMs), which replace part of the denoiser with a fixed-point solver over shared attention layers to enable adaptive depth with fewer parameters. To make it more effective for masked generation, we first introduce a cross-step consistency loss, which aligns hidden representations at neighboring denoising steps and, second, three-state reuse (3SR) which warm-starts the solver using the previous solution by treating unchanged, still-masked, and newly revealed tokens differently. Together, these components define our complete training-to-inference framework for fixed-point masked generation, \emph{CoFRe}. We also show that pre-trained MGMs can be converted into FP-MGMs with short fine-tuning, avoiding full retraining. Across modalities, CoFRe improves the quality and cost trade-off. On OpenWebText, CoFRe reduces parameters by 38.8\%, training time by 11.5\%, and VRAM by 16.9\%, while improving generative perplexity from 830.8 to 101.8 at a budget of $96$ transformer-block forward passes, compared to MDLM. In ImageNette, CoFRe reduces training time by 48.6\% and VRAM by 50.7\%, while improving FID in all sample budgets tested. Overall, CoFRe offers a practical framework for cheaper training and stronger low-budget masked generation.

We introduce a new tool, Express, for converting a non-causal attention approximation into a causal approximation with matching approximation guarantees. When combined with the state-of-the-art Thinformer approximation, Express improves upon the best known causal attention guarantees, delivering $\log^{3/2}(n)/s$ approximation error with only $O(s)$ memory and $O(s^2 \log^2(n))$ compression overhead for a sequence of length $n$. We pair these developments with an efficient I/O-aware Triton implementation, demonstrate substantial speedups over FlashAttention 2, and use Express to accelerate three distinct components of the language modeling pipeline: long-context prefill, KV cache compression, and long-form decoding.

Foundation models trained on large unlabeled corpora develop emergent capabilities — tasks that go beyond the training objective. We extend this paradigm to multimodal biology with Bonbon, a foundation model for molecular interactions trained on approximately 3.2 trillion tokens of paired protein-ligand sequences. From a single self-supervised objective, Bonbon exhibits four zero-shot emergent capabilities at hierarchical levels of resolution — functional (mechanism of action), residue (binding site localization, including orthosteric/allosteric discrimination), bond (covalent versus non-covalent engagement), and atom (active moiety identification at single-bond resolution). All four capabilities generalize to out-of-distribution proteins. Bonbon was trained on protein-ligand sequences alone, without any task-specific supervision and without structural coordinates. We conducted controlled scaling experiments across four model scales from 42M to 1.6B parameters; every capability evaluated shows a sharp jump at the largest scale on continuous metrics, even as pretraining loss decreases smoothly. This demonstrates that loss-based scaling laws alone are insufficient to predict when zero-shot emergent capabilities become available. The capabilities reported here emerge after training at a 1:2,000 parameter-to-token ratio to accommodate the sparsity of paired biological interaction data. The same frozen representations achieve state-of-the-art binding and affinity predictions at five orders of magnitude greater speed than structure-based methods. In prospective wet-lab validation, Bonbon achieves a 70% hit rate on de novo compounds, discovering novel chemotypes not previously documented as inhibitors for those targets. All of the tested compounds had less than 60% structural similarity to molecules in the training corpus.

Mobile and wearable sensing offer a scalable pathway to longitudinal mental health monitoring, yet accurate prediction remains limited by two fundamental forms of distribution shift in human behavior. The first arises across users, where similar sensor patterns can carry different clinical meanings, a phenomenon we call inter-subject heterogeneity. The second arises within a single user, where behavioral baselines themselves evolve over time, producing intra-subject behavioral drift. In this domain, ground-truth labels come from Ecological Momentary Assessment (EMA), brief self-reports collected in everyday life that are intrusive and inherently sparse. As a result, approaches that address these distribution shifts by updating model parameters are difficult to apply. To address this gap, we propose DyCon (Dynamic Context Modeling), a dual-loop framework that personalizes by building and evolving a separate natural-language personal context for each user, without updating any model parameters. Within each session, a momentary refinement loop restructures this personal context to resolve inter-subject heterogeneity. Across sessions, a longitudinal update loop integrates only validated insights to track intra-subject behavioral drift. Extensive experiments on four public benchmarks (GLOBEM, PMData, StudentLife, LifeSnaps) show that DyCon consistently outperforms baseline methods, particularly under sparse supervision and behavioral drift. Our results suggest that evolving an explicit personal context offers a practical pathway for longitudinal personalization in real-world health monitoring.

Drifting Models have recently achieved state-of-the-art performance in one-step image generation by training generators to follow corrective vector fields. However, their standard reverse-style drift improves samples from the model side, which can severely underutilize training samples and bottleneck early convergence. To address this, we introduce DualDrift, a unified framework that augments the original reverse drift with a novel forward drift. The forward drift lets each data sample assign correction to generated rollouts, providing denser data-side signals and accelerating early training. At the same time, this dense data-side signal can overemphasize dominant regions when used alone. DualDrift therefore uses a parameter-free scheduler driven by empirical batch statistics to balance the fast early correction of forward drift with the stable refinement of reverse drift. Evaluated on ImageNet $256\times256$, DualDrift achieves faster convergence and significantly improves over the baseline Drifting Models in both FID and Inception Score.

EHR foundation models create new opportunities for scalable clinical prediction across health systems, yet their deployment remains constrained by a basic tokenization bottleneck. The same clinical concept can appear as different ontology codes, local identifiers, synonyms, or free-text descriptions, while standard EHR tokenizers treat these variants as unrelated symbols. We introduce MEDIATOR, a discrete ontology-agnostic medical concept tokenizer that turns EHR harmonization into an open-form tokenization problem. Instead of relying on manual mappings, a fixed ontology vocabulary, or continuous event embeddings, MEDIATOR maps any clinical concept expressed as text into a shared vector-quantized codebook. MEDIATOR is trained with synonym supervision from paired concept descriptions, and further guided by enriched concept descriptions, empirical code co-occurrence, and curated relation graphs so that equivalent expressions collapse while non-equivalent concepts remain organized in a medically meaningful discrete space. The resulting vocabulary is compact, inspectable, reusable by standard EHR Transformers, and open to previously unseen code descriptions or text strings. Used as the token interface of an EHR Transformer, MEDIATOR improves mean AUPRC on 20 MIMIC-III tasks by 4.0% and improves zero-shot transfer to 14 eICU tasks by 8.8%, while supporting transferable attribution across hospitals.

Diffusion language models intrinsically fail to capture correlations between decoded tokens, which leads to a harsh trade-off between sampling quality and throughput. To solve this issue, we propose DiLaDiff, a variant of masked diffusion language models with three components: (1) a continuous latent space with semantic capabilities, learned by an auto-encoder fine-tuned from an existing masked diffusion language model; (2) a latent diffusion model learning the prior over the encoder distribution; (3) a consistency model distilling the learned prior into a few-step latent generative model. We show that, even without distillation, our latent-guided diffusion model outperforms the masked diffusion baseline while significantly accelerating inference. Consistency distillation further lowers the computational overhead of continuous diffusion, such that the latent is generated in negligible time compared to discrete decoding.

Accurate forecasting of Earth's dynamic systems is vital in geophysics, where neural operators offer a powerful data-driven paradigm. Standard operators often treat the Earth as a flat Euclidean plane, introducing geometric artifacts. In contrast, Spherical Neural Operators (SNOs) resolve this mismatch by operating within the spherical spectral space, naturally adapting to spherical geometries. However, for computational efficiency, existing SNOs typically rely on the rotation equivariance assumption. This assumption simplifies the transitions of physical states into isolated spectral modal evolutions, thereby over-idealizing the Earth as a homogeneous system. To solve this problem, we no longer rely on the rotation equivariance assumption that isolates spectral modal evolution, but directly derive an explicit modal interaction kernel and propose Diagonal Spherical Neural Operators named DIAGNO. DIAGNO explicitly parameterizes cross-modal interactions through spectral modal decomposition, thereby preserving the inherent heterogeneity of Earth dynamics. Furthermore, leveraging the unique diagonal structure of the spherical spectrum, we design inter- and intra-diagonal interaction mechanisms to natively capture essential zonal and meridional dynamics, respecting intrinsic geophysical regularities. Extensive experiments across simulated fluid dynamics and real-world Earth system reanalysis (including atmosphere and ocean) datasets demonstrate DIAGNO’s superior capability, consistently achieving state-of-the-art performance in multi-step forecasting. Beyond this, we validate DIAGNO’s practical analytical value in professional geophysical domains by assessing its fidelity in capturing climatological anomalies and maintaining physical consistency.

Molecular conjugates, including PROTACs and peptide-drug conjugates (PDCs), derive their function from the joint behavior of multiple coupled components, yet most generative approaches design these components independently and combine them only after generation. Such staged pipelines ignore cross-component dependencies and often produce conjugates that are chemically invalid, fall outside empirical conjugate distributions, or lose function upon assembly. We introduce $\textbf{C}$o$\textbf{up}$led $\textbf{O}$bjective-Guided Discrete $\textbf{F}$lows for $\textbf{Mo}$lecular $\textbf{C}$onjugate $\textbf{A}$ssembly $(\textbf{CupOFMoCA})$, a discrete generative framework that formulates conjugate design as a constrained, coupled generation problem. CupOFMoCA restricts generative trajectories to a chemically feasible conjugate manifold and biases local transitions using target-specific activity predictors, ensuring all components remain mutually compatible throughout generation. We show that coupling constraints and objective guidance enable anticipatory design that preserves post-assembly predicted activity and produces structurally realistic conjugates across both PDC and PROTAC settings, outperforming staged baselines, including LinkerNet, DiffLinker, and DiffPROTACs for PROTACs, across assembly validity, predicted activity, and physicochemical property ranges. These results demonstrate that explicit coupling and constraint enforcement are sufficient to recover functional conjugates across conjugate classes, and provide a principled foundation for generative modeling where function emerges only at the level of the assembled system. Our anonymous code repository can be found at https://anonymous.4open.science/r/Cupofmoca-Neurips.

Despite the success of foundation models in language and vision, molecular graph generation still lacks a unified framework for heterogeneous design tasks with reliable controllability. While reinforcement learning (RL) offers a natural post-training mechanism for task-specific optimization, applying it to graph generative models is hindered by the vast atom-wise action spaces and chemically invalid intermediate states. We propose \textbf{Co}ntrollable \textbf{Mole}cular Generative Foundation Models (CoMole), built with a unified motif-aware graph diffusion pipeline. By learning a motif-aware graph space, CoMole transfers pretrained structural priors into controllable generation, where RL optimizes conditional reverse policies over chemically meaningful decisions. We theoretically characterize the bottleneck of atom-level RL and justify motif-aware policy optimization. Across three heterogeneous benchmarks spanning materials and drug discovery, CoMole ranks first in controllability on all nine targets, reduces MAE by up to 48.2\% relative to the strongest baselines, and maintains validity above 0.94 without rule-based correction or post-hoc filtering. We further show that CoMole transfers controllability to unseen properties by optimizing only task embeddings with the generator frozen, achieving performance competitive with strong task-specific baselines.

The Hessian of the energy with respect to the nuclear positions is indispensable in atomistic modelling. However, constructing this matrix requires $O(N)$ Hessian vector products, traditionally limiting high-accuracy Hessians to small systems. Machine learning interatomic potentials (MLIPs) have accelerated atomistic modelling by providing highly accurate energies and forces at $O(N)$ cost, yet the resulting $O(N^2)$ cost of Hessians remains a practical bottleneck for large systems. Based on the insight that we can derive the sparsity pattern for an MLIP's Hessians in closed form, we show in this paper how to use techniques from sparse automatic differentiation to reduce the cost of a local MLIP's Hessians to a system-size-independent number of Hessian-vector products, yielding overall $O(N)$ total cost without any approximations. We benchmark our approach on a variety of systems ranging from alkane chains to water clusters to A$\beta$40 conformers. Depending on the MLIP configuration, we achieve the linear scaling regime already on relatively small systems, resulting in large runtime reductions between $2\times-15\times$ for these systems. This opens up the possibility of scaling high-accuracy MLIP Hessians to very large systems, such as proteins that were previously inaccessible.

Modeling biomolecular conformation and dynamics is critical for elucidating biological functions, motivating generative surrogates for molecular dynamics (MD) simulations to address two complementary tasks: time-independent conformational sampling and time-dependent trajectory generation. Fundamentally, both tasks require matching the distribution of the generated collections to the MD reference, rather than reproducing individual configurations. Yet existing deep-learning approaches predominantly optimize a per-sample regression loss (score- or flow-matching), whose gradient is computed in isolation per sample and therefore carries no direct signal about the cross-sample distributional properties that define both tasks, leading to biased ensembles and long-horizon temporal drift. We address this with Collective Supervision, a training paradigm that aligns the empirical measures of the generated and reference collections via a maximum mean discrepancy on SE(3)-invariant physical observables; a single loss handles both tasks within a unified formulation. To mitigate the exposure bias inherent in autoregressive trajectory rollout, we further introduce Collective Rolling Forcing, which couples Collective Supervision with autoregressive self-rollout during training. Our framework, CoDyna, generalizes across proteins, protein--ligand and protein--protein complexes; on four all-atom MD benchmarks (ATLAS, MISATO, DynaRepo, and DynaBench), a unified model surpasses task-specialized baselines on most thermodynamic and kinetic fidelity metrics and remains structurally stable over 2k-frame rollouts.

Incremental learning aims to continuously learn from dynamic data streams without suffering from catastrophic forgetting. However, existing methods may fail in challenging scenarios where class and domain spaces expand simultaneously, struggling to learn continually and stably from fragmented data streams. To address this challenge, we propose EMCift (Extensible Multi-Center Modeling with Dual-Path Drift Harmonization), a novel model that treats heterogeneous data across domains and semantic spaces as co-existing multi-view observations, enabling the construction of global knowledge from local observations. Our approach organizes extensible multi-center prototypes as a tree hierarchy, allowing the global class representation across different domains to grow organically from fragmented local data. To sustain this dynamic topology, we introduce a dual-path drift harmonization strategy that balances structural stability with adaptive plasticity. Specifically, one path captures essential updates to enable prototype evolution, while the other path leverages a conditional adversarial mechanism to robustly initialize domains, effectively preventing catastrophic forgetting caused by class confusion. Extensive experiments on three datasets demonstrate the state-of-the-art performance of our proposed model in the class-domain incremental learning scenario. Code will be released upon acceptance.

Human pose estimation has advanced rapidly driven by large-scale adult-centric models. However, children remain significantly underrepresented due to data scarcity, distinct body morphology, and unique motion dynamics. We introduce ChildPose, a modular video foundation model designed for child-centric pose estimation. ChildPose transforms pretrained pose encoders into a streaming pediatric model via memory-augmented temporal attention and age-aware co-training. By maintaining a memory bank of adjacent frame representations, the model achieves robust localization under motion blur, occlusion, and atypical poses, while an auxiliary age-prediction objective enforces representations that capture developmental morphology. We curate a diverse, video-based pediatric dataset spanning various age groups and capture conditions, evaluating out-of-distribution generalizability on entirely unseen subjects. Across multiple 2D baselines, ChildPose consistently outperforms both off-the-shelf models and child-data fine-tuned variants, while ensuring stable sequence-level predictions. Our results underscore that effective pediatric pose estimation necessitates child-specific modeling beyond standard adult-centric estimators.

Compositional prompts in text-to-molecule generation often specify several structural constraints at once, such as scaffolds, functional groups, substituents, and finer modifiers. Existing molecular latent diffusion models typically inject the full prompt once and apply it uniformly throughout denoising, which can cause later constraints to overwrite or distort earlier structural commitments. We propose **Chain-of-Generation (COG)**, a training-free progressive latent diffusion strategy for text-guided molecular design. Technically, CoG realizes progressive generation as multi-stage latent denoising over cumulative molecular sub-prompts: it factorizes a prompt into chemistry-informed coarse-to-fine segments, conditions each stage on the cumulative sub-prompt, and continues denoising from the previous-stage latent using a fixed partial-denoising schedule. This turns one-shot conditional generation into a sequence of stage-wise denoising steps that help retain earlier structure while introducing later constraints. Across benchmark, out-of-distribution, and ablation settings, CoG improves structural prompt fidelity over one-shot conditioning on the tested latent graph-diffusion backbones.

Cellular perturbation experiments are essential for probing biological mechanisms and guiding therapeutic discovery. While existing methods often treat perturbations as monolithic distributional shifts, they leave the underlying structural variations implicit. Rather than attempting to infer independent global networks for every condition, we propose scPertCRL, a causally structured generative framework for single-cell perturbation prediction. scPertCRL decomposes perturbation effects into a shared latent structural backbone, condition-specific structural modulations, and exogenous shifts. By leveraging biologically informed embeddings and mechanism-aware supervision, Differential Network Learner parameterizes these localized structural changes, enabling robust generalization to unseen interventions. Extensive experiments on genetic and pharmacological benchmarks demonstrate that scPertCRL significantly improves predictive accuracy and out-of-distribution generalization, while yielding model-based insights into perturbation mechanisms.