NeurIPS 2026 Papers

As frontier AI technologies dramatically improve their abilities to perform cybersecurity and reverse-engineering tasks, it is critical to rigorously and realistically track their capabilities in these domains. Increased agent capabilities in reverse-engineering will scale black-box vulnerability discovery, which will have dramatic implications on software security at large. In this work, we present WiREBench, a unique benchmark derived from popular real-world mobile applications, to measure agents' end-to-end vulnerability discovery and reverse-engineering capabilities. This benchmark, containing 22 protocols and 62 subtasks, evaluates agents abilities to identify and exploit historical vulnerabilities in binaries' proprietary network cryptography, via black-box reverse-engineering. To resolve these tasks, agents must chain together static binary analysis, understanding of applied cryptography, and complex tool use via dynamic instrumentation of mobile applications. Beyond benchmarking, we ran WiREBench against real-world applications, identifying and confirming 65 protocol vulnerabilities in applications used by hundreds of millions of users. These results demonstrate WiREBench's utility not just as a benchmark to assess current agent capabilities, but also as a pipeline for auditing cryptographic protocols and reverse-engineering in the real world.

Cell-free DNA (cfDNA) liquid biopsy has been clinically routine for cancer detection and monitoring for years but has yet to produce applications in chronic diseases of parenchymal organs such as fibrotic liver, kidney, and lung disease, despite diseased organs shedding orders of magnitude more cfDNA into the bloodstream than tumors do. This gap is structural: methods developed for cancer have been adapted for chronic disease without a theoretical framework establishing when this should work or where it should break down. We formalize cfDNA classification as recovering a disease-induced shift on a low-dimensional submanifold of the simplex over molecular configurations, observed through sparse genomic windows. The formalization reveals that cancer and chronic disease occupy structurally distinct regimes parameterized by one dimensionless quantity: the inverse participation ratio $c(v)$ of the disease perturbation across windows. Cancer is small-$c(v)$: signal on a few genomic hotspots. Chronic disease is large-$c(v)$: signal spread across many co-regulated regulatory programs. We prove an information-theoretic lower bound: any classifier observing $k$ windows recovers at most a $\sqrt{k/c(v)}$ fraction of the available information in the local-asymptotic-normality regime, regardless of per-cfDNA model complexity. The bound applies architecture-agnostically to classifiers that focus deep computation on a small number of pre-selected genomic regions, i.e. the architectural shape that has succeeded in cancer detection. Such classifiers degrade sharply rather than gradually as $c(v)$ grows, with the transition provable from the geometry of the problem. Chronic-disease classification requires an architecture matched to its regime: compute allocated across a very large set of windows rather than into deeper per-cfDNA models, with $k \gtrsim c(v)$. We propose a concrete architecture and verify on synthetic experiments that the regime transition predicted by the theorem is observed, and the architecture outperforms cancer-style baselines on chronic-like signal. We then validate on real patient data, where the architecture exceeds both cancer-tuned methods and standard-of-care clinical tests.

Tabular foundation models are used as off-the-shelf predictors for heterogeneous tabular tasks, but it remains unclear how they will perform on real genotype-to-phenotype tabular datasets, which carry unique challenges. One such challenge is ancestry-dependent non-stationarity in allele effect sizes (the effects of genetic variation on disease risk): the predictive contribution of a given genetic variant may change significantly across the ancestry spectrum. This is especially consequential for patients from ancestry groups that are underrepresented in existing datasets, because models that are unaware of this non-stationarity are more likely to perform poorly for them. The battery of synthetic tasks used to pretrain tabular foundation models does not capture this structure, and we show that it leads to systematic performance degradation. Using controlled hierarchical Gaussian-process stress tests, we demonstrate that both off-the-shelf TabICL and TabPFN are robust when ancestry dispersion in the data is low, but as dispersion grows, the latent ancestry-dependent non-stationarity of effect sizes becomes a first-order problem, and the predictive performance of both models deteriorates. We confirm the same pattern on All of Us (AoU) (a large biobank containing whole-genome sequencing and electronic health record (EHR) data spanning the ancestry spectrum) across an extensive panel of cancer phenotypes evaluated with the two leading tabular foundation models: TabICL and TabPFN. Holding the in-context training-set size, phenotype, and feature set fixed, ancestry-specific in-context tables, which are less dispersed in ancestry space, consistently outperform size-matched meta-ancestry tables, isolating ancestry dispersion as the driver of degradation. To address the failure without changing the base architecture, we construct two synthetic task families that explicitly encode ancestry-dependent effect drift and instruction-tune an off-the-shelf TabICL model on tasks sampled across both families. On held-out AoU evaluations covering cancer and respiratory disease phenotypes, the tuned model yields more stable performance across ancestry-distance bins and is especially strong in the bins farthest from the center of the in-context exemplars in ancestry space, that is, on subjects whose ancestry is most underrepresented among the provided exemplars. The paper contributes an evaluation protocol, a failure analysis, and two synthetic task families targeted at ancestry-dependent non-stationarity in precision-medicine tabular modeling.

As AI development increasingly involves models adapted to downstream settings, a new algorithmic challenge for developers has surfaced: deciding when to adopt model updates. Currently, the only reliable approach is to retrain on new upstream models and then evaluate extensively, which can be prohibitively expensive. We introduce a framework for assessing when to propagate newly released model versions to downstream applications that does not require access to upstream data or retraining a priori. Our framework incorporates geometric similarity and information-theoretic sufficiency to determine when an upstream update substantially shifts the representational basis recruited for a downstream task. This enables targeted update adoption, and supports emerging norms for coordinating AI supply chain infrastructure.

Visual token pruning is an effective way to accelerate vision-language models and is especially useful for vision-language-action (VLA) inference, where many visual tokens must be processed before predicting robot actions. Existing pruning methods usually estimate which tokens can be pruned based on attention scores or feature diversity, retaining tokens that are either highly attended or visually different from others. However, most of them use fixed pruning schedules, such as pruning once at a preset layer or pruning at uniformly spaced layers. Such schedules can be risky for VLA models, because the model may not know which visual regions matter for the action in early layers. Tokens that look unimportant at first may become useful after the model combines visual observations with the language instruction. In this work, we propose SAPrune, a training-free visual token pruning framework for efficient VLA inference. Instead of pruning at fixed layers, SAPrune uses a small calibration set to observe how action-to-visual attention changes across layers, and chooses pruning layers only after the attention pattern becomes more reliable. At each selected layer, SAPrune applies a dual-path pruning rule: one path protects strongly attended visual tokens from pruning, while the other prevents useful surrounding context from being discarded. Experiments on LIBERO, SIMPLER, and real-world robotic tasks show that SAPrune prunes 87.5% of visual tokens and achieves up to 1.718× inference speedup while maintaining competitive task success rates.

Recent work has reported two failure modes of transcriptomic foundation models: validation loss plateaus beyond 100M parameters, and consistent underperformance against simple linear baselines on clinically relevant tasks. We show that both findings invert under a specific pretraining recipe. We introduce EVA-RNA, a transformer pretrained on 545k human and mouse samples spanning bulk RNA-seq, microarray, and pseudobulked single-cell data, scoped to immunology and inflammation, one of the largest therapeutic areas in clinical development. EVA-RNA exhibits clean power-law scaling from 7M to 300M parameters, with no plateau emerging within our scale range. On a benchmark co-designed with immunologists and drug development experts, EVA-RNA outperforms existing foundation models on every task category, spanning drug discovery, preclinical-to-clinical translation and patient stratification. Mechanistically, EVA-RNA learns species-invariant representations in which orthologous genes progressively align across layers without supervision. We interpret these results as evidence that scope and data composition are sufficient, with conventional architecture and knowledge-informed gene embeddings, to achieve clinical utility and scaling in I&I. We also release EVA-RNA 60M model weights to support continued investigation.

conditioned on desired semantic properties is an emerging challenge in modern generative modeling. Metropolis-Hastings (MH) provides a principled route to conditional sampling, but requires access to exact pointwise target-density evaluations, which are not available in generative settings. Meanwhile, pairwise comparisons by humans or model ``judges'' are highly accessible and have proved valuable across diverse applications. We introduce Pref-MH, a general exact MH sampler for judge-induced conditional distributions using only stochastic binary pairwise comparisons. Our key observation is that the MH unnormalized density ratio matches the preference odds of BT choice models. The central challenge is that while MH requires precise ratio computation, BT judges provide only sampled binary feedback. To this end, we develop a valid accept/reject rule whose resulting Markov chain provably converges to the target distribution. We further show that, for a fixed proposal kernel and budget, Pref-MH is optimal in the Peskun-Tierney sense among this class of exact reversible acceptance rules. Experiments on text generation with LLM judges and image generation with VLM judges demonstrate that Pref-MH provides a practical and flexible approach to conditional sampling from comparative feedback.

In real-world applications, being able to explain *why* a prediction is made is often as important—if not more so—than achieving high accuracy. However, existing methods for sequential event prediction typically prioritize accuracy at the expense of interpretability. Neuro-symbolic approaches offer a promising direction to improve interpretability by learning symbolic rules, but extending them to next-event prediction is challenging due to the presence of temporal dependencies. We introduce RULES, a neuro-symbolic model that learns sequential rules directly from data, and combine it with a residual network to form RUNES (Rule Network for Event Sequences). RULES accurately recovers ground-truth rules from synthetic data and discovers compact, interpretable rule sets on real data. RUNES combines these rules additively with a residual network, achieving competitive accuracy while providing transparent access to active rules and how residuals adjust predictions.

Clustering is a central tool for discovering latent structure in unlabeled data, yet modern clustering pipelines often end with a hard assignment of each observation to a cluster without rigorous measures of assignment uncertainty. We propose a novel weighted conformal approach for constructing confidence sets for cluster labels. The key difficulty is that the labels available for calibration are not observed ground-truth labels, but synthetic labels produced by a data-dependent clustering algorithm. Our method develops a conformal inference algorithm that corrects the resulting mismatch with the latent target labels through weights by formulating conformal clustering as a conditional label-distribution shift problem. We first derive an oracle procedure that attains finite-sample marginal coverage and then develop a computationally tractable and implementable version using estimated conditional label probabilities and augmented calibration. We show that the coverage of the estimated-weight procedure depends on the estimator, giving an explicit bound on the loss relative to the nominal level. Empirical studies beyond the mixture-model settings, explored by the recently developed stochastic split conformal clustering procedure, demonstrate that the proposed weighted approach offers improvements in non-linear and high-dimensional clustering applications in terms of set size.

Accurate emulation of multi-scale physical systems governed by PDEs demands models that remain stable over long autoregressive rollouts while preserving fine-scale structures. Deterministic emulators produce overly-smoothed predictions, while generative approaches better capture details but are costly. Latent-space generative models have emerged as a compromise but with the additional cost of separately pre-trained autoencoders. We propose **Wavelet Flow Matching** (WFM), a novel generative emulator that overcomes current trade-offs between cost and skill by performing optimal-transport directly in the multi-scale wavelet space. Rather than learning a latent compression, WFM leverages the hierarchical structure of a U-Net to jointly predict transport velocities of a prescribed wavelet representation. On three challenging systems of chaotic fluid dynamics, WFM achieves superior long-horizon stability, accuracy and spectral coherence compared to state-of-the-art models. Our results clearly position the wavelet space as an effective training-free representation for generative emulation of complex physical dynamics.

Computing the partition function $\mathcal{Z} = \mathrm{tr}(e^{-\beta H})$ of a quantum spin Hamiltonian $H$ on $n$ sites is a fundamental problem in statistical physics and quantum chemistry. However exact evaluation requires $O(8^n)$ time and $O(4^n)$ memory making it intractable for all but the smallest systems. We propose an unbiased stochastic estimator of $\mathcal{Z}$, and more generally of $\mathrm{tr}(f(H))$ for any function $f$ admitting a convergent power series over exponentially large, implicitly defined operators using only local random walks, avoiding both matrix materialization and matrix-vector products. Our approach combines Graph Random Features with the Hutchinson's stochastic trace estimator, and exploits the algebraic structure of quantum spin Hamiltonians to reduce the per-step computational cost to $O(n + d)$, where $d$ is the mean degree of the spin coupling graph, independent of the Hilbert space dimension $N = 2^n$. We validate the estimator empirically against exact matrix exponentiation for small system sizes, and demonstrate its applicability to system sizes where exact methods are infeasible.

Multimodal agents struggle with physical-world reasoning, particularly spatial reasoning from partial views, as the physical world is inherently complex. We think multimodal agentic reasoning about the physical world should ground its actions in deterministic feedback, rather than rely solely on internal imagination. Concretely, we propose to equip vision–language models (VLMs) with a physics engine (e.g., MuJoCo, UE5), which simulates the consequences of the agent's actions. However, granting the agent raw access to a physics engine is not sufficient, as the agent must first reconstruct the scene inside the physics engine. We therefore introduce the Visual Harness System, an orchestration system that decouples perception from reasoning with a physics engine as the backend. The perception module composes off-the-shelf perception skills, such as camera-pose estimation, open-vocabulary detection, and metric depth, into a structured scene map. The reasoning module then issues multi-turn tool calls that the engine executes over the map, and composes the final answer from the grounded feedback it receives. Experiments on spatial reasoning benchmarks show that our approach consistently outperforms strong baselines across both open- and closed-weight VLMs, with up to $16.57\%$ improvement over prior methods on the MindCube benchmark.

Radio astronomy is an indispensable discipline for studying distant celestial objects. Measurements of wave signals from radio telescopes, called visibility, need to be transformed into images for astronomical observations. These dirty images blend information from real sources and artifacts. Therefore, astronomers usually perform reconstruction before imaging to obtain cleaner images. Existing methods consider only a single modality of sparse visibility data, resulting in images with remaining artifacts and insufficient modeling of correlation. We propose VVTRec, a multimodal radio interferometric data reconstruction method, to enhance visibility information extraction and improve image-domain output quality. Since the information in sparse visibility is inherently limited, we transform it into image-form and text-form features. Therefore, we can leverage vision-language models to process the two derived modalities, providing knowledge banks as a supplement for reconstruction. The sparse visibility is accordingly utilized as the query to perform the integration and selection of multimodal external knowledge. Consequently, VVTRec improves the structural integrity and accuracy of reconstructed images by enriching spatial and semantic information. Our experiments demonstrate that VVTRec effectively enhances imaging results by exploiting multimodal information without introducing excessive computational overhead.

Raman spectroscopy is often hindered by strong autofluorescence backgrounds, which are typically handled by waiting for fluorescence to bleach before recording the Raman spectrum. Here, we instead show that short, incomplete recordings of the bleaching process already contain enough information to recover the underlying Raman signal. We present the first unsupervised model that operates on short, incomplete, variable-length sequences of time-resolved spectra, decomposing them into Raman spectra and autofluorescence components while simultaneously removing measurement noise. Our VAE-based method simultaneously learns (i) a bank of fluorophore spectra comprising the baseline, (ii) a model of the measurement noise, and (iii) the distribution of Raman spectra and decay behavior. We evaluate our method on spectral reconstruction and downstream peak detection, where it outperforms state-of-the-art approaches on multiple simulated benchmarks as well as on a newly collected real-world dataset of spectral time series. We further show that fluorescence decay dynamics themselves contain discriminative information that may be exploited in future work.

Conditional Flow Matching (CFM) unifies conventional generative paradigms such as diffusion and flow-based models. Interaction Field Matching (IFM) is a recently proposed framework that generalizes Electrostatic Field Matching (EFM), rooted in Poisson Flow Generative Models (PFGM). While both frameworks define generative dynamics, they start from different objects: CFM specifies a conditional probability path in data space, whereas IFM specifies a physics-inspired interaction field in an augmented data space. This raises a basic question: **are CFM and IFM genuinely different, or are they two descriptions of the same underlying dynamics?** We show that they coincide for a natural subclass of IFM, which we call *forward-only* IFM. Specifically, we construct mappings between CFM and forward-only IFM and show that they induce the same generative dynamics. We further show that general IFM is strictly more expressive: it includes EFM and other interaction fields that cannot be realized within the standard CFM formulation. Our findings suggest developing generative models using both interpretations rather than treating them separately. Moreover, they highlight a novel direction for generative modeling based on backward-oriented field lines, which lies outside the conventional CFM formalism and may lead to new generative properties.

Transformers are powerful models but are inefficient with large context due to the quadratic complexity of the attention mechanism. Limiting their context window addresses the efficiency concern but at a cost: they cannot remember everything from the past, which severely limits their computational power. We show that we can simultaneously achieve efficiency and universal computation if we restrict attention heads to be 2-dimensional. First, we give the first algorithm for standard 2D attention that serves every insertion and every query in amortized polylogarithmic time per token. Prior work either restricted key/query magnitudes or paid polynomial overhead per step. Second, we show that the class of 2D-attention models is universal and efficient: a vanilla decoder transformer efficiently simulates an arbitrary RAM machine via chain-of-thought decoding so that, combined with our first result, the simulation runs in $\mathrm{polylog}(N)$ time per simulated RAM step end-to-end. In contrast, earlier universality results addressed only computability, showing Turing completeness while incurring large polynomial factors in their execution time. Third, as an application, we turn a transformer into a computer. We provide a compiler that supports the full $\texttt{i32}$ WebAssembly subset with integer computations (reachable from C via $\texttt{clang}$) and turns such programs into the weights of a transformer, which then executes them over millions of steps autoregressively. We illustrate how this can be used to solve combinatorial problems, tasks at which even large reasoning LLMs struggle.

Byte-level byte-pair encoding (BBPE) tokenizers are attractive for large language models (LLMs) in multilingual settings because they cover all Unicode text. Under UTF-8, however, many scripts start from a higher byte-level fallback cost than English: when no learned merge covers a character $c$, the tokenizer must emit multiple byte-derived symbols. We call this worst-case pre-merge cost the encoding floor: $$ f(c) = |\\mathrm{utf8}(c)| \\in \\{1,2,3,4\\}. $$ A higher floor inflates token budgets, shrinks usable context, and increases per-request cost. Changing the underlying text encoding can reduce this gap, but a single global encoding can also make already-efficient English spans more expensive in mixed-script text. We propose Universal Byte-Level Encoding (UBE), a dual-alphabet tokenizer with deterministic per-character routing: $$ r(c) = \\begin{cases} \\mathrm{UTF\\text{-}8}, & |\\mathrm{utf8}(c)| \\leq 2,\\\\ \\mathrm{UTF\\text{-}16}, & |\\mathrm{utf8}(c)| > 2. \\end{cases} $$ Characters satisfying $|\\mathrm{utf8}(c)| \\leq 2$ stay on the UTF-8 path, while characters satisfying $|\\mathrm{utf8}(c)| > 2$ are routed through UTF-16. This reduces token costs for Basic Multilingual Plane (BMP) scripts whose characters typically satisfy $|\\mathrm{utf8}(c)| = 3$ and have the highest token premiums, without raising costs for already-efficient spans. UBE changes only the byte representation presented to byte-pair encoding (BPE): the BPE merge rule remains standard, and exact decoding $\\mathrm{decode}(\\mathrm{encode}(s)) = s$ is preserved. Across intrinsic tokenization evaluations, UBE reduces cross-lingual dispersion in English-normalized token-count ratios $\\pi(\\ell) = m(\\ell)/m(\\mathrm{en})$, lowering cross-script token-budget disparity. In multilingual language model experiments, UBE preserves LM quality within seed variability. In the main multilingual settings, UBE reduces token counts most for high-premium scripts and also slightly lowers English token counts; these reductions translate into more usable context under fixed token budgets and faster content-matched prompt processing.

Vision Transformers (ViT) dominate computer vision. However, their reliance on rigid patch projectors hinders transfer to Earth Observation (EO), where inputs vary widely in modality, scale, and resolution. We introduce UniverSat, a ViT-style backbone built around a Universal Patch Encoder that maps patches from arbitrary spatial, spectral, and temporal resolutions, and from both optical and non-optical sensors, into a shared embedding space with a single set of weights. This enables training one model on heterogeneous multimodal corpora in self-supervision, yielding robust sensor-agnostic spatial features. We validate this approach with strong results across classification and segmentation on standard EO benchmarks from GeoBench, PANGEABench, and SpectralEarth. The code and model will be released in open-source.

Stochastic estimators are fundamental to large-scale optimization, where population quantities must be inferred from noisy oracle observations. Although influential methods such as momentum, SPIDER, STORM, and PAGE have been highly successful, their analyses are largely estimator-specific and expectation-based, obscuring the structural tradeoffs that determine reliability. In this paper, we develop a unified framework for stochastic variance-reduced estimation based on a recursion with three components: memory retention, reset probability, and a correction term for iterate movement. This framework recovers several classical estimators, motivates new second-order variants, and yields a bias-variance decomposition of estimation error. Our main result is a unified high-probability bound proved using a new dimension-free vector-valued Freedman inequality, valid for smooth normed spaces involving random sums of vector martingales. The result applies in both Euclidean and non-Euclidean settings, including the analysis of mirror-descent-based methods in Banach spaces. As applications, we obtain high-probability oracle complexities for unconstrained optimization with mirror descent, establishing the logarithmic dependence on the confidence level. We also derive the first $\tilde{\mathcal{O}}(\varepsilon^{-3})$ oracle-complexity bounds for stochastic optimization with expectation constraints, improving upon the existing $\tilde{\mathcal{O}}(\varepsilon^{-4})$ complexity by leveraging variance-reduced estimation for the first time in this setting.