NeurIPS 2026 Papers

Many scientific datasets, such as molecular conformational ensembles or single-cell tissue measurements, are naturally modeled as meta-distributions: distributions over probability measures on non-Euclidean domains. Existing generative methods largely assume Euclidean geometry and fail to capture this structure. We introduce Riemannian Wasserstein Entropic Flow Matching (RWEFM), a generative framework on the Wasserstein space $\mathcal{P}_2(\mathcal{M})$ of a Riemannian manifold $(\mathcal{M},g)$. RWEFM is trained by regressing a neural vector field onto Riemannian optimal transport velocities, using McCann displacement interpolations as conditional paths. We confirm theoretically that this construction leads to a valid flow matching approach on $\mathcal{P}_2(\mathcal{M})$ and introduce the Riemannian Entropic Map, a GPU-efficient approximation of the optimal transport map on manifolds. Our experiments show that by respecting the intrinsic geometry of the data, RWEFM can generate whole single-cell samples in hyperspherical latent spaces and protein conformational ensembles on the torus. Code and tutorials are available at \href{https://github.com/AnonRWEFM/RWEFM}{RWEFM}.

Compositional inference - the decomposition of observations into an unknown number of latent components - is central to perception and scientific data analysis. Attention-based models perform well when components are approximately separable, as in object-centric vision. Under additive superposition, however - where multiple components contribute to every observation - we identify a structural failure mode we term slot collapse: multiple slots converge to the same dominant component while weaker ones remain unrepresented. We trace this to a general limitation: attention is memoryless with respect to explained evidence. All slots repeatedly operate on the same input without accounting for what has already been explained, so gradients are dominated by the strongest component, inducing shared fixed points across slots. As a result, attention fails to enforce non-redundant allocation under additive superposition. We address this by introducing residual evidence modeling, which tracks the remaining explanatory capacity of the input, instantiated via evidence depletion - a minimal modification combining multiplicative depletion with an attention bias. Controlled ablations show that parallel attention, sequential processing alone, and loss-based regularization all fail to resolve collapse; evidence depletion, which adds stateful residual tracking to sequential attention, consistently succeeds. Across synthetic benchmarks and real-world audio mixtures (FUSS), evidence depletion reduces slot collapse by up to an order of magnitude, generalizing beyond synthetic settings. On gravitational-wave source inference for the ESA/NASA LISA mission, under identical architectures, data, and losses, standard attention fails while evidence depletion prevents collapse and enables multi-source posterior estimation. These results show that under additive superposition, residual evidence tracking is the operative ingredient - preventing collapse and enabling compositional inference.

Vision-to-code tasks require models to reconstruct structured visual inputs, such as charts, tables, and SVGs, into executable or structured representations with high visual fidelity. While recent Large Vision Language Models (LVLMs) achieve strong results via supervised fine-tuning, reinforcement learning remains challenging due to misaligned reward signals. Existing rewards either rely on textual rules or coarse visual embedding similarity, both of which fail to capture fine-grained visual discrepancies and are vulnerable to reward hacking. We propose Visual Equivalence Reward Model (Visual-ERM), a multimodal generative reward model that provides fine-grained, interpretable, and task-agnostic feedback to evaluate vision-to-code quality directly in the rendered visual space. Integrated into RL, Visual-ERM improves Qwen3-VL-8B-Instruct by +8.4 on chart-to-code and yields consistent gains on table and SVG parsing (+2.7, +4.1 on average), and further strengthens test-time scaling via reflection and revision. We also introduce VisualCritic-RewardBench (VC-RewardBench), a benchmark for judging fine-grained image-to-image discrepancies on structured visual data, where Visual-ERM at 8B decisively outperforms Qwen3-VL-235B-Instruct and approaches leading closed-source models. Our results suggest that fine-grained visual reward supervision is both necessary and sufficient for vision-to-code RL, regardless of task specificity.

Fluorescent microscopy provides a rich view into how proteins localize within cells, but it remains experimentally infeasible to image human proteins across all of the different factors that can impact localization. We introduce Vermeer, a channel-adaptive autoregressive generative model for in silico generation of microscopy images of protein localization. Vermeer conditions generations on protein sequences and landmark stains showing the morphology of cells, which enables it to generalize to unseen proteins and cell lines. We show that Vermeer, trained on the Human Protein Atlas, can generate images with substantially improved perceptual quality and biological fidelity over previous proposals. Additionally, Vermeer's autoregressive framework enables flexible generation using varying channel subsets and orderings, enabling zero-shot transfer to data collected under different imaging conditions and channel configurations than those used for training. These results position Vermeer to enable scalable modeling of protein localization and is a step towards generative foundation models that can operate over distinct microscopy datasets.

Understanding 4D scenes is a core challenge in visual intelligence, requiring models to jointly capture scene geometry, appearance, and dynamics from partial observations. Existing approaches, however, are subdivided into feedforward models that are used for direct geometric prediction and generative models for visual synthesis. We propose a unified generative-predictive framework for 4D scene understanding that jointly models RGB video and geometric representations such as depth and camera pose with a single joint generative model. Our generative formulation enables us to learn from diverse, heterogeneous datasets with different subsets of geometric and visual annotations, and enables us to condition on arbitrary subsets of inputs, allowing a single model to perform cross-modal generation, geometric prediction, and future scene inference from partial observations. In addition, our generative approach enables test-time search over multiple candidate scene completions, improving geometric prediction from sparse observations. Overall, we illustrate the efficacy of our generative-predictive framework on a suite of generation and prediction tasks.

Current unified multimodal models typically rely on discrete visual tokenizers to bridge the modality gap. However, discretization inevitably discards fine-grained semantic information, leading to suboptimal performance in visual understanding tasks. Conversely, directly modeling continuous semantic representations (e.g., CLIP, SigLIP) poses significant challenges in high-dimensional generative modeling, resulting in slow convergence and training instability. To resolve this dilemma, we introduce UniCom, a unified framework that harmonizes multimodal understanding and generation via compressed continuous semantic representations. We empirically demonstrate that channel-wise compression is significantly more effective than spatial token reduction, retaining competitive VAE-free reconstruction fidelity while substantially improving generative convergence. Accordingly, we design an attention-based semantic compressor to distill dense features into a compact unified representation. A Transfusion-style predictor then generates these compressed latents with dense spatial correspondence. At scale, UniCom achieves competitive text-to-image generation and state-of-the-art performance on complex image editing without relying on VAE latents. The gains are especially clear on knowledge-intensive benchmarks.

Genomic events span multiple spatial scales, from single-nucleotide substitutions to large structural variants, yet existing foundation models process only reference DNA sequences, leaving methylation, short variants, and structural-variant boundaries as downstream labels or auxiliary inputs. We propose the Unified Genomic Model (UGM), an encoder-only genomic foundation model that learns multi-scale genomic events as native tokens within a unified 43-token vocabulary. UGM introduces three algorithmic components: (i) a unified event tokenizer that maps reference bases, SNPs, CpG methylation, short INDELs, and structural-variant boundaries into a single event-state space; (ii) event-balanced pretraining, which promotes adequate gradient coverage for rare but biologically important events; and (iii) Site-Level Joint Prediction (SJP), a masked-language-modeling objective that recovers the complete genomic state at a masked position as a single token rather than predicting base, methylation, and variant labels independently. We evaluate UGM against specialized and generalist baselines across functional variant classification, regulatory benchmarks (NT and GUE), structural-variant detection, and cross-event attention analysis. UGM achieves competitive performance, particularly on tasks where explicit event-state information is central. It obtains the highest score on all seven functional variant tasks, reaches competitive splicing prediction on NT, and notably outperforms DNA-only baselines on SV breakpoint detection. These results suggest that native multi-scale event pretraining is a promising direction for genomic representation learning.

Financial markets are complex, noisy environments which present unique challenges for representation learning and world modeling. In this study, we systematically explore the application of supervised and self-supervised representation learning methods to financial markets data and their ability to learn world models of financial markets. To support this, we assemble and release $\textbf{Market-1T}$ a dataset of more than one trillion observations spanning all US equities from 2008 through 2025. We then develop a domain-specific data augmentation and apply a variety of modern SSL objectives including Joint Embedding Predictive Architecture (JEPA), Masked Autoencoder (MAE), and DINO. Our results highlight that supervised methods are still dominant in this domain when applied directly to core quantitative finance tasks: predicting changes in prices, volatility, and transactions cost. Further, when supervised models are trained across tasks they are able to leverage complementarities which enhance cross-task performance. While performance of SSL-based methods on these core tasks lags behind, we find that they better capture latent structure---such as time and assets effects---which supervised models miss. Our results suggest current SSL objectives capture broad features of markets but further work is needed to close the gap between supervised methods.

Diffusion language models (DLMs) promise parallel, order-agnostic generation, but on standard benchmarks they have historically lagged behind autoregressive models in sample quality and diversity. Recent continuous flow-matching approaches over one-hot and embedding spaces have narrowed this gap, suggesting continuous state spaces are highly effective for language. In this work, we further close the autoregressive gap by modeling text as a continuous diffusion process over fixed-width bitstreams. Our approach represents semantic tokens as analog bit sequences and utilizes a matched-filter residual parameterization to isolate contextual learning from analytic independent-bit posteriors. Crucially, our empirical comparisons reveal that while deterministic continuous DLMs are competitive, they can be overly contractive, undershooting real-data entropy. We demonstrate that the remaining quality-diversity gap is bridged by a stochastic sampler that applies Langevin-type corrections gated by the entropy-rate profile, automatically concentrating stochasticity in high-information regions while remaining nearly deterministic elsewhere. On the One Billion Word Benchmark (LM1B), our 130M-parameter bitstream model reaches a generative perplexity (GenPPL) of 59.76 at matched real-data entropy (4.31) using 256 neural function evaluations (NFEs), decisively outperforming prior DLM baselines and reaching the autoregressive reference. On OpenWebText (OWT), our stochastic sampler establishes a new continuous-DLM Pareto frontier, achieving GenPPL = 27.06 at an entropy of 5.26 using 4x fewer steps than previous 1024-NFE baselines. As an additional architectural benefit, bitstream diffusion removes the O(V) vocabulary scaling bottleneck shared by standard DLMs. By predicting O(log V) bitwise logits via semantic bit-patching, our model yields a reduced memory footprint and higher throughput, demonstrating a scalable paradigm for language generation as vocabulary sizes grow.

Multivariate time series forecasting is important in real-world applications, but practical data often show both temporal and spectral non-stationarity. Existing methods may treat severe spectral drift as informative evolution, allow unreliable spectral components to contaminate temporal representations through credibility-agnostic time--frequency interaction, and infer cross-variable dependencies from noisy local correlations, which may lead to spurious long-term dependencies. To address these issues, we propose TFNS, a dual-stream time--frequency framework for suppressing unreliable spectral evidence under non-stationarity. Specifically, we introduce a Spectral Credibility Estimator to quantify patch-level spectral stability and softly suppress low-credibility frequency components for more reliable frequency modeling; a Credibility-Gated Cross-Interaction module to enable bidirectional time--frequency enhancement while adaptively constraining cross-domain information injection according to spectral reliability; and a Spectral Cointegration Graph to capture stable long-term cross-variable structure in the spectral domain while mitigating spurious dependencies under non-stationary drift. Together, these modules form a progressive pipeline from local spectral purification to reliability-aware cross-domain interaction and robust long-term structure modeling. Experiments on 11 benchmarks show that TFNS achieves state-of-the-art performance, ranking first in 44/55 MSE and 43/55 MAE comparisons.

Most time-series foundation models operate on fixed temporal grids, representing signals as vectors, tokens, or patches and learning maps between discretized observations. This grid-to-grid view is convenient for standard forecasting, but it is not the natural object for reusable time-series modeling, where temporal resolution, forecast horizon, and query locations may change across tasks and deployments. We argue that time-series foundation models should instead learn sample-conditioned function-to-function operators. Given finite observations of an underlying signal, such an operator maps the observed context to an output function that can be evaluated at user-specified times. We propose \textbf{TimeOperator}, a compact branch-trunk backbone that realizes this view with a shared context encoder, spectrally informed query features, context-adaptive trunk parameters, and multi-basis temporal responses. Forecasting and imputation are handled by evaluating the same operator on different query sets, while classification reuses the shared context representation. Across deterministic and probabilistic forecasting, imputation, and classification benchmarks, TimeOperator is competitive with existing time-series foundation models. Further analyses show that cross-frequency and cross-horizon prediction can be handled by changing only the query set, rather than changing the decoder or relying on post-hoc resampling.

Pruning reduces the inference cost of large language models, but existing criteria primarily preserve large activations or reconstruct layer outputs. We argue that this overlooks a key computation performed by particularly sparsity-sensitive neurons in the MLP up and gate projections: separating similar inputs into dissimilar outputs. This suggests that effective pruning should preserve not only activations, but also pairwise output differences. We introduce a family of difference-informed pruning methods built upon this principle. Wisp is a first-order, update-free method that scores weights using input-difference norms, and Wisp+ refines this score neuronwise using the input pairs each neuron separates most strongly. Finally, Whisper is a second-order method that uses a lightly regularized difference Hessian as its reconstruction objective. Across the Llama 2 and 3.1 families, our second-order variant consistently improves language modeling performance over strong reconstruction-based baselines, while our update-free variants improve over activation-aware update-free baselines, with gains stronger in more constrained settings. The improvements over Wanda and SparseGPT extend to structured sparsity, downstream evaluations, and other model families, suggesting that preserving input differentiation is a broadly useful signal for post-training LLM sparsification.

Despite the strong performance of diffusion models in molecular generation, the sampling process itself remains largely understudied, with most existing methods adopt off-the-shelf sampling strategies developed for natural images. Yet unlike natural images, molecular distributions are governed by physical laws and are sharply concentrated, making them particularly challenging for standard samplers, which frequently produce invalid or unphysical structures. In this work, we revisit diffusion sampling from a unified stochastic differential equation (SDE) perspective and introduce a general framework parameterized by two interpretable controls: \emph{stochasticity} and \emph{temperature}. Our theoretical analysis reveals that stochasticity accelerates the decay of sampling error, while temperature directly controls distribution sharpness, enabling concentration on physically plausible configurations. Building on these insights, we develop \textsc{TReaSSure} (Temperature-Regulated Stochastic Sampling), a lightweight, training-free strategy that combines stochasticity scheduling with temperature annealing to better capture the sharply concentrated distributions of molecular data. Extensive experiments on small molecule generation, protein structure prediction, and protein design demonstrate that \textsc{TReaSSure} consistently improves generation quality and produces more physically valid structures. Empirical analyses further corroborate our theoretical findings. The method harnesses pretrained models without retraining, underscoring its generality and practical value across diverse molecular domains.

We increasingly use machine learning to label scientific datasets. The models we develop and deploy are improving all the time, but they are not and will likely never be perfect. Mistakes matter, as errors can propagate into our scientific understanding, particularly when systematically biased. Very reasonably, scientists thus review substantial proportions of ML-generated labels to verify or correct mistakes in pursuit of ensuring their scientific findings are not biased by ML. In this work, we focus on helping scientists optimally allocate this reviewing effort relative to their scientific goals. We focus on a specific class of scientists (ecologists) and a specific, widespread, and impactful modeling target (occupancy modeling, which estimates where species are likely to occur, conditioned on environmental factors). We introduce ACORN (Active Continuous-Score Occupancy Modeling), a method that incorporates ML predictions into occupancy models and strategically selects samples for expert review that are maximally informative for downstream ecological analysis. Across camera-trap and bioacoustic datasets, our method recovers ecological conclusions close to those obtained from fully human-labeled data, while requiring substantially fewer expert reviews than non-targeted review policies. Our results suggest that ML-assisted scientific workflows should optimize expert effort for downstream inference, rather than for classifier accuracy alone, especially when human review budget is limited.

Molecular docking aims to predict the three-dimensional structure of a protein–ligand complex; in the conventional structure-based setting, it is conditioned on an input protein structure and ligand, often starting from an apo receptor in flexible docking or a holo receptor in rigid docking. Recent co-folding models exemplified by AlphaFold 3 achieve strong accuracy in protein–ligand complex prediction, but they typically generate complexes from sequence-derived inputs and therefore do not directly use apo structures as structural priors in docking, leading to a larger conformational search space and weaker robustness in difficult or few-step settings. We present BridgeDock, a framework that adapts pretrained co-folding backbones to flexible docking by modeling the apo-to-holo transition as a diffusion bridge, allowing generation from the initial protein–ligand state rather than from Gaussian noise. To make this bridge formulation compatible with pretrained denoisers, we introduce an alignment mechanism that maps bridge states to the original denoising schedule of the pretrained model. Experiments on standard flexible docking benchmarks show that BridgeDock consistently outperforms strong co-folding baselines. Moreover, BridgeDock is computationally efficient at inference time and remains effective even with very few denoising steps.

Schedule-Free SGD, proposed in *The Road Less Scheduled*, achieves optimal convergence rates without requiring the training horizon in advance, by replacing learning rate schedules with a principled form of iterate averaging. However, the method still requires tuning a base learning rate whose optimal value depends on unknown problem constants. In this work, we continue down this road by deriving Polyak-type step sizes for Schedule-Free SGD and Adam that compute the learning rate at each iteration from the sampled loss, gradient, and current iterates alone. We first propose an oracle variant that uses per-sample optimal function values and prove an $\mathcal{O}(1/\sqrt{t})$ anytime last-iterate rate for convex Lipschitz objectives. We then remove the oracle requirement with a safeguarded variant that replaces the unknown optimal values with any available lower bound, achieving the same rate up to a neighborhood that vanishes under interpolation. Both step sizes reduce to existing Polyak rules for standard SGD when momentum is set to zero, unifying standard and schedule-free Polyak methods. Numerical experiments on language modeling, including pretraining and distillation, show that the proposed methods match or surpass tuned Schedule-Free baselines while offering greater robustness to hyperparameter choices.

Existing peptide models saturate natural-sequence proteomics benchmarks, but whether they capture the underlying physicochemical dynamics or merely pattern-match on sequence statistics remains unclear. Public peptide datasets offer no axis along which to test this: non-canonical residues and stereochemical variation are essentially absent. We introduce StereoPep, a benchmark of 48,788 chemically synthesized peptides spanning 11 libraries, produced in the laboratory via high-throughput flow synthesis and characterized experimentally by reverse-phase liquid chromatography retention time. Unlike model-generated synthetic data, every peptide in StereoPep is a real molecule with a real chromatographic measurement. The dataset is constructed around matched diastereomeric pairs (sequences identical in primary structure but differing at a single stereocenter), providing a direct test of whether models capture three-dimensional stereochemical information. StereoPep enables progress across multiple fields: it serves as a benchmark for stereochemistry-aware molecular representations and geometric deep learning, supports adapting protein language models to non-canonical residues, and underpins emerging proteomics multiplexing strategies that encode samples through retention-time shifts. Across specialized retention-time models, graph-based architectures, and protein language models, all approaches predict overall retention reasonably well (Pearson $r \approx 0.80$) but collapse on diastereomer pair discrimination (AUC $\approx 0.55$), revealing that current representations encode atomic composition far better than 3D structure.

Machine learning interatomic potentials (MLIPs) enable efficient and accurate atomistic simulations but depend critically on the quality and diversity of the training data. We introduce Stein kernelized molecular dynamics (SKMD), an enhanced sampling method that uses interacting particle dynamics to acquire informative training configurations for the active learning and fine-tuning of MLIPs. SKMD corresponds to a stochastic variant of Stein variational gradient descent that is adapted for molecular dynamics by incorporating asynchronous particle updates and a kernel of global atomic descriptors, which provides a symmetry-aware measure of configurational similarity. Unlike other enhanced samplers used in molecular dynamics, SKMD preserves the Boltzmann distribution as the asymptotic distribution of the dynamics. This property enforces a balance between the exploration of diverse configurations and attraction toward high-probability regions of the energy landscape. We further propose an approach to efficient online data acquisition using an adaptive stopping criterion that selects non-redundant training data over the course of simulation. We demonstrate SKMD for the active learning of a neural network model of the Muller-Brown potential and the fine-tuning of a MACE interatomic potential for alanine dipeptide. Compared to active learning baselines, our method achieves higher model accuracy in fewer training iterations, with the same number of acquired training samples.

Predicting transcriptome-wide cellular responses to genetic and biomolecular perturbations is central to functional genomics and therapeutic target discovery. Existing methods assign gene positional encodings derived from arbitrary orderings or static co-expression masks built from unperturbed cells, and cannot capture the functional rewiring induced by perturbations. We present SPEAR, a generative framework built on two complementary positional mechanisms. Spectral positional encoding grounds each gene's position in the geometry of the co-expression network via continuous coordinates derived from the graph Laplacian. Adaptive repositioning then derives each gene's position from its perturbation-conditioned hidden state at every layer, allowing the attention genes exert on one another to reorganize without explicit supervision. Evaluated on genetic, pharmacological, and cytokine perturbation benchmarks, SPEAR demonstrates excellent recovery of differential gene-expression, distributional structure, and perturbation-specific identifiability. SPEAR also captures latent positional relationships that closely map to known transcription factor-target interactions.