NeurIPS 2026 Papers

Physics-informed neural networks (PINNs) train a single neural approximation by minimizing multiple physics- and data-derived losses, but the gradients of these losses often interfere and can stall optimization. Existing remedies typically treat this pathology either through scalar loss balancing or full-parameter-space gradient surgery, leaving it unclear which intervention is most appropriate. We show that PINN gradient conflict is not a uniform failure mode with one universal remedy. Instead, we identify distinct PINN gradient-conflict regimes, each associated with a different intervention class. Persistent directional conflict may require separate loss-indexed parameter subspaces, magnitude imbalance often favors scalar reweighting, and low or transient conflict may require no extra mitigation. To select between scalar reweighting and a lightweight architectural intervention, we propose a diagnostic-first framework. It profiles a 1000-step unmodified PINN run and, when intervention is warranted, uses one low-rank adapter per loss to create explicit loss-indexed parameter subspaces attached to a shared PINN trunk, providing each loss with a direct gradient pathway. Across more than 60 PDE configurations, including forward, inverse, multi-physics, parameter-varying, and high-dimensional problems up to 50D, persistent directional conflict dominates standard forward $K=3$ benchmarks and a natural $K=4$ thermoelastic system, where adapters combined with reweighting yield significant improvements. In contrast, $K=3$ inverse problems and natural $K=5$ and $K=6$ multi-physics systems are largely magnitude-dominated and often favor reweighting alone, while full-parameter-space gradient surgery can fail on heterogeneous parameter spaces. A regime-transition theorem and a blockwise neural tangent kernel analysis explain why gains from these separate adapter parameter subspaces arise selectively in persistent-conflict regimes.

Tandem mass spectrometry provides a high-throughput framework for identifying and quantifying proteins in complex biological samples. In computational proteomics, predicting peptide MS/MS spectra is a critical task, enabling downstream applications such as large-scale peptide identification and quantification. While deep learning architectures have substantially improved prediction accuracy, three evaluation challenges obscure the true progress of the field. First, inconsistent data preprocessing and incompatible model output spaces hinder fair model comparison. Second, flawed data splitting strategies can permit hidden sequence leakage and inflate reported performance. Third, existing evaluations typically lack comprehensive cross-species benchmarking and systematic assessment of model robustness to influential experimental conditions. To address these challenges, we propose PepSpecBench, a unified benchmark for peptide MS/MS spectrum prediction. PepSpecBench standardizes data preprocessing across complementary public datasets, enforces a strict backbone-disjoint splitting strategy to eliminate sequence leakage, and evaluates diverse architectures within a shared fragment-ion representation space. It further introduces a comprehensive multi-species evaluation suite and physically grounded metadata perturbation probes to assess model robustness and instrument awareness. We uncover previously unrecognized performance discrepancies and robustness limitations across six representative models, providing actionable insights for future model design, evaluation and practical deployment. Code and data are available at https://anonymous.4open.science/r/PepSpecBench-NeurIPS2026/.

Predicting mutation-induced changes in peptide--protein binding affinity ($\Delta\Delta G$) is central to therapeutic peptide optimization, but peptide-specific predictors remain limited by scarce labels, target overlap in supervised benchmarks, and costly molecular simulations. We introduce PepDDG, a zero-shot, training-free predictor that ranks peptide mutations by decomposing binding perturbations into three complementary information channels: energetic perturbation, geometric environment, and evolutionary compatibility. Each channel is computed from a wild-type complex structure, transformed into rank space, and combined by non-parametric Borda aggregation, without fitting fusion weights or fine-tuning neural predictors. PepDDG is motivated by rank-covariance analysis showing cross-channel fusion can improve rank correlation when channels retain complementary signal, whereas increasingly expensive refinement of a single energetic channel has diminishing returns. Empirically, the channels are individually moderate but capture distinct mutation regimes, and rank fusion consistently outperforms single-channel variants and training-free baselines. On our curated benchmark of 332 peptide-chain mutations across 33 targets, PepDDG reaches Spearman $\rho = 0.619$; the calibrated PepDDG-Cal variant reaches $\rho = 0.691$. Performance remains strong on short and cyclic peptide subsets, with $\rho = 0.813$ and $0.872$, respectively. These results support complementary evidence fusion on PDB-derived complex structures as a practical alternative to costly same-channel physical refinement for peptide $\Delta\Delta G$ ranking; a separate diagnostic shows that predicted structures can be used when no crystal structure is provided at inference time.

Chain-of-thought (CoT) reasoning with self-consistency improves performance by aggregating multiple sampled reasoning paths. In this setting, correctness is no longer tied to a single reasoning trace but to the aggregation rule over a pool of candidate paths, making aggregation uncertainty the central challenge. This issue is critical where confidently incorrect answers are far more costly than abstentions. We introduce a conformal procedure for CoT reasoning that directly addresses aggregation uncertainty. Our approach replaces majority voting with weighted score aggregation over reasoning paths and calibrates an abstention rule using conformal risk control. This approach leads to finite-sample guarantees on the confident-error rate--the probability that the system answers and is wrong. We further identify score separability as the key condition under which abstention provably improves selective accuracy, and derive closed-form expressions that predict accuracy gains from calibration data alone. The method is fully inference-time, and requires no retraining. Across four benchmarks, four open-source models, and three score classes, realized confident-error rates are consistent with the prescribed targets up to calibration-split and test-set variability. Our method achieves $90.1\\%$ selective accuracy on GSM8K by abstaining on less than $5\\%$ of problems, compared with $82\\%$ accuracy under majority-voting baseline.

Reinforcement-learning-trained agents for retrieval-augmented generation retrieve content (passages, knowledge-graph fragments, or hyperedges) and supervise training with outcome rewards on the final answer. Across these paradigms the reasoning route through retrieved content never leaves the language model's hidden state, conflating declarative knowledge with the procedural knowledge that recurs across queries of the same logical type; content-only methods consequently plateau as reasoning depth grows. We then introduce PatternBloom, an agentic RAG framework that shifts the unit of retrieval from content (what to read) to logic (how to reason). A first RL stage scores the agent's constructed evidence graph by a frozen oracle's information gain through the Information-Density Reward (IDR), a size-normalized mutual-information surrogate. High-reward trajectories are distilled into a Graph Pattern Memory (GPM) of type-abstracted reasoning skeletons, and a second RL stage uses the Pattern-Augmented Reward (PAR) to couple policy to memory, turning procedural reasoning into a learnable, externalized structural prior. A 7B PatternBloom outperforms 27 baselines by +15.0 Avg-OOD EM, with the gain compounding with reasoning depth precisely because procedural knowledge, once distilled, is reused rather than re-derived at every query.

Low-rank approximation captures one important notion of matrix simplicity: a matrix is simple if it is well described by a low-dimensional linear model. This paper studies a complementary notion. We consider matrices that become structurally simple only after permuting rows and columns, so that their mass concentrates in a prescribed geometric pattern such as a band, block-diagonal, arrowhead, or related form. We introduce a pattern distance that measures how close a matrix is to a given structural mask after optimal reordering, giving a common framework for latent ordering, community detection, and bandwidth minimization. The problem is NP-hard in general; we derive an alternating algorithm based on Linear Assignment subproblems, and establish identifiability of the target structure up to pattern automorphisms. We further introduce a pattern-plus-low-rank decomposition that combines permutation-structured fitting with a low-rank residual; it dominates pure low-rank approximation by feasibility and admits a strict rank separation on planted block-diagonal targets, with broad empirical gains across pattern families. We evaluate the framework on synthetic recovery tasks, real correlation and graph adjacency matrices, and transformer attention heads (GPT-2, Qwen2.5-0.5B and Qwen2.5-7B), showing that a single universal algorithm recovers qualitatively different forms of matrix structure.

We study how visual information is routed in vision-language models (VLMs). Using causal patching on controlled synthetic and natural datasets, we find that models rely on two distinct pathways to solve visual tasks: A \emph{direct pathway}, where visual information is retained in image token representations and read out by the final token at later layers, and a \emph{text-mediated pathway}, where visual information is first transferred to the query tokens and then read out by the final token. Across three visual tasks, we show that pathway selection is task-dependent, and that data distribution and prompt design can also modulate which pathway is used to solve the image-based query. Moreover, using attention knockouts and corrupted-input patching, we find that these pathways are flexible, under certain interventions, models can rely on the text-mediated pathway as a fallback when the usual pathway is ablated. This behavior unifies findings in prior work and shows that ablation-based interventions can reveal what models could do rather than what they normally do. Together, our results provide a mechanistic characterization of visual information flow in VLMs and highlight the flexibility of their internal mechanisms under intervention.

Predicting spatial gene expression from histopathology images enables transcriptomic analysis on archival H\&E samples where matched spatial transcriptomics data is unavailable. Gene expression is organized through cyclic regulatory circuits: transcription factor complexes are themselves products of regulated genes, aggregating upstream gene activity and modulating co-expressed downstream programs (e.g., the ISGF3 complex coordinating hundreds of interferon-stimulated genes through shared cis-regulatory elements). Existing approaches to histology-to-expression prediction, both deterministic and generative, employ generic architectures that ignore this structure. We introduce regulator attention, a mechanism that encodes this cycle by pooling gene representations into a small set of content-dependent tokens via a learned selector and routing them back through cross-attention to modulate gene-level features, in contrast to learnable token arrays whose identity is independent of the input. Within a flow matching framework conditioned on pathology foundation model embeddings, regulator attention consistently outperforms deterministic and generative baselines on correlation-based metrics across four tissue types spanning human cancer, normal tissue, and cross-species samples, with substantial gains on highly variable genes. Beyond predictive accuracy, 87.5\% of regulator tokens align with known biological pathways without supervision. Replacing them with content-independent learned queries eliminates this specialization and degrades predictive performance, isolating content-dependent token generation as the causal mechanism.

Multimodal large language models (MLLMs) are increasingly used to analyze pathology images. However, dominant multimodal benchmarks in pathology mainly score final diagnostic answers, captions, or reports. These evaluations provide limited insight into whether a model understands the multiscale visual content needed for pathology reasoning and decision-making. We introduce PathView-Bench, a vision-anchored benchmark for fine-grained and multiscale visual understanding in computational pathology. Built from $23$ public pathology imaging datasets with human-supervised labels and spatial annotations, PathView-Bench evaluates MLLM understanding in two fields of view: Region-FOV for high-resolution local regions and Slide-FOV for macro whole-slide views. By converting raw annotations into deterministic task targets, PathView-Bench enables programmatic scoring of region localization, visual recognition, quantity estimation, spatial reasoning, and insufficient-context judgment. The benchmark contains $14$ VQA-style tasks, $61,673$ images, and $308,070$ samples across $28$ organs and $7,253,526$ annotations. Evaluating $18$ representative general-purpose, medical-domain, and pathology-oriented MLLMs, we observe substantial limitations even in advanced models on fine-grained visual tasks across multiscale pathology images. PathView-Bench provides a reproducible basis for developing and evaluating pathology MLLMs with explicit multiscale visual understanding.

Flow Matching is a powerful framework for learning transport maps between probability distributions. Yet its standard single-parameter formulation is not designed to capture multi-parameter variations where the resulting transport should be path-independent. Path independence is crucial because it ensures that transformations depend only on the initial and target distributions, not on the specific path. In this work, we introduce \textit{Path-independent Flow Matching (PiFM)}, a method for learning vector fields whose induced flows yield path-independent transport between distributions. We show that PiFM generalizes Flow Matching to higher-dimensional parameter domains while enforcing structural conditions that ensure consistency of composed transformations. In addition, we show that, under suitable assumptions, PiFM approximates the Wasserstein barycenter, linking the framework to a notion of distributional interpolation. To enable practical training, we propose a tractable, simulation-free objective that regresses onto multi-parameter conditional probability paths. We showcase empirically that PiFM outperforms other approaches on both synthetic and real world data in interpolating path-independent trajectories and generating desired out of distribution samples. Our code is available at \url{https://anonymous.4open.science/r/Pi_flow_matching-C83F/}.

Dataset distillation compresses large datasets into compact synthetic sets with comparable performance in training models. Despite recent progress on diffusion-based distillation, such methods typically rely on heuristic guidance or prototype assignment over long denoising chains, which increases sampling cost and makes prototype-consistent control harder under strong guidance or low IPC. We propose \emph{Path-Guided Flow Matching (PGFM)}, the first flow matching-based framework for generative distillation, which enables deterministic synthesis by solving an ODE in a few steps. In particular, we introduce a retrieval-based prototype inversion stage that identifies prototype-consistent initial noises for class prototypes on the frozen flow manifold, and further develop an anchor-guided residual correction strategy for bounded stage-wise control. This design follows a controlled-transport principle: retrieval reduces initialization mismatch, while anchor guidance provides a bounded residual correction along the flow trajectory. Extensive experiments across high-resolution benchmarks demonstrate that PGFM matches or surpasses prior diffusion-based distillation approaches with fewer sampling steps while delivering competitive performance with improved efficiency.

Repeated AI assistance can improve immediate task performance while reducing the skill available for future independent work. We develop a mathematical framework for this long-run tradeoff. The model tracks two state variables: a latent human skill level governing expected independent performance, and a delegation level representing the learner's evolving tendency to rely on AI. Skill changes through error-driven learning under practice and decay under delegation; delegation responds to observed performance, increasing when AI-assisted work appears to outperform independent work. We analyze the resulting dynamics and contrast them with fixed delegation. With fixed delegation, skill follows a one-dimensional learning-decay process with a single stable equilibrium. With adaptive delegation, the coupled system has two attracting equilibria separated by the stable manifold of an interior saddle. The existence and geometry of this separatrix require a global phase-plane analysis of the coupled dynamics. The system is path-dependent: small differences in initial skill or reliance can lead to different long-run outcomes. We use this characterization to show that AI assistance can improve short-run performance while producing worse long-run performance than a no-AI baseline. Increasing AI capability can enlarge the basin of attraction of the low-skill equilibrium, making delegation appear beneficial for longer while increasing the risk of eventual skill loss. The qualitative picture is observed to persist across alternative specifications. Together, these results show that the risk is not AI assistance itself, but the coupling between performance-driven reliance and use-dependent skill change.

Probabilistic circuits (PCs) have emerged as generative models guaranteeing tractable probabilistic inference. However, tractability comes at the price of larger model sizes than in other neural generative models. As such, to scale PCs to high-dimensional data such as images, previous work has devised several parameter sharing techniques as well as fast evaluation protocols that consider learning PCs on independent image patches. In this work, we revisit this practice to highlight a surprising learnability issue: models that assume independence counterintuitively can yield better likelihoods than models that do not. To break this apparent bottleneck and to scale PCs further, we introduce a recursive parameter sharing construction (RePA) that fosters efficiency without assuming independence. In a set of rigorous experiments, we show that RePA can greatly reduce time and memory consumption while yielding competitive likelihoods.

Researchers often need web corpora that are not answers to a single query, but reusable collections of documents spanning many decentralized sources. We refer to this problem as thematic web data collection}: assembling semantically relevant documents that share a common theme but are scattered across heterogeneous and structurally disconnected regions of the web. Existing approaches, including traversal-based methods and LLM-driven web agents, are limited by local exploration or short-horizon stopping, preventing comprehensive collection. We formulate this task as a long-horizon information foraging problem over query-induced semantic patches, where the central challenge is deciding when to continue local exploitation and when to switch to new regions. We propose PatchScout, a multi-agent framework that instantiates a class of patch-switching policies to balance local exploitation and adaptive patch switching under a fixed budget. Experiments on four live-web topics show that PatchScout substantially improves yield and domain coverage over traversal-based and search-augmented reasoning agent baselines. In an applied social science setting with an incomplete prior collection, PatchScout discovers 102 previously unidentified entities, illustrating its ability to expand existing thematic datasets.

Long-context inference with Large Language Models (LLMs) is bottlenecked by the linearly growing memory of the key-value (KV) cache. Existing compression methods reduce the cache through token eviction or approximation, but degrade sharply at aggressive compression budgets. We propose PatchKV, a training-free framework that compensates KV cache compression methods by carrying part of the context in the model's weights. PatchKV pairs an off-the-shelf compressed KV cache with a context-specific weight patch, which is computed once at context-loading time and served for downstream queries for the context. The weight patch is derived in closed form via ridge regression, by aligning the block-wise activations of context-derived reference query tokens under the full cache and the compressed cache. Once merged into the model, the patch leaves the forward graph and per-query inference cost unchanged in the single-context, multi-query setting. Across long-context QA (SCBench with up to 170k tokens, SQuAD, NIAH) and math (GSM8K) benchmarks on three model architectures, PatchKV consistently improves cache compression methods, suggesting an alternative direction to compensate them at aggressive budgets.

An LLM safety patch can pass a benchmark while still being a poor repair. This risk is especially acute for jailbreak repairs, where the goal is to correct a specific unsafe behaviour without changing unrelated behaviours. A patch may block the exact prompts used for evaluation yet fail on close harmful variants, or suppress harmful behaviour by over-refusing benign prompts that share its wording or structure. Existing evaluation protocols primarily test whether models can be broken, while aggregate metrics, such as attack success, refusal rates, and global capability scores, cannot distinguish selective repairs from broader local suppression. To address this gap, we introduce **PatchBench**, a benchmark of empirically observed model-specific jailbreak failures that induce actionable harmful answers, designed as concrete repair targets for patching research. Starting from 27,870 prompts aggregated from 37 public jailbreak benchmark datasets, we manually curate 15,314 English prompts and query 8 open-source instruction-tuned models. We then combine WildGuard filtering, pairwise Elo ranking, and manual verification to retain only unsafe completions that answer the harmful requests. The result is a curated bank of 400 high-confidence jailbreak failures, organised by model. We further introduce **PatchBench-Local**, a local evaluation protocol that tests whether a patch is behaviourally precise. For each harmful source prompt, PatchBench-Local generates three families of local neighbours: harmful variants that preserve the malicious intent, benign prompts with matched structure, and benign prompts that reuse the key harmful terms. It evaluates both harmful-neighbour correction and benign-neighbour preservation, thus distinguishing selective repair from broader local suppression. We use PatchBench-Local and MMLU to evaluate four state-of-the-art activation steering methods. Our results show that global capability can remain nearly unchanged while local benign-neighbour regressions are severe, confirming that aggregate metrics alone can miss important collateral damage. By exposing whether patches act as precise behavioural repairs or as broad local suppressors, PatchBench-Local provides a more precise basis for developing and comparing jailbreak repair methods.

As Multimodal Large Language Models (MLLMs) continue to advance and find increasingly broad applications, a key challenge persists: the patch-based vision paradigms widely adopted by MLLMs inevitably introduce structural fragmentation when processing information-dense images such as tables and charts. Specifically, when a table row or chart axis spans multiple visual partitions, the lack of cross-partition communication causes the model to lose track of structural continuity, leading to performance fluctuations when layout changes --- a problem we term layout sensitivity. To address this fundamental limitation, we propose Patch-for-Patch (P4P), a lightweight module that can be plugged into existing patch-based vision encoders. P4P dynamically selects a sparse set of tokens as bridging anchors and uses them to establish cross-partition communication through bidirectional attention. Combined with a gated fusion strategy, P4P restores global structural coherence without the quadratic cost of global attention. Extensive experiments across Qwen2.5-VL, MiniCPM-V, and InternVL3.5 demonstrate that P4P consistently improves structured content visual understanding, notably boosting TableEval accuracy by up to 9.12\% and significantly reducing layout sensitivity. The source code is available at: https://anonymous.4open.science/r/P4P/.

Model inversion is a widely adopted technique in data-free learning that reconstructs inputs from a pretrained model through iterative optimization, without access to original data. However, its application to Vision Transformers (ViTs) incurs high computational cost due to expensive self-attention mechanisms. To address this, $\textit{Sparse Model Inversion}$ (SMI) was proposed to improve efficiency by gradually pruning seemingly unimportant patches, even claiming they are obstacles to knowledge transfer. However, our empirical findings suggest the opposite: even randomly selected patches can eventually acquire transferable knowledge over the inversion process. In fact, we further observe that removing prematurely inverted patches hinders the extraction of class-agnostic features essential for knowledge transfer, as well as class-specific features. In this paper, we propose $\textit{Patch Rebirth Inversion}$ (PRI), a novel approach that constructs multiple sparse images within a single inversion process by incrementally detaching informative patches, instead of removing unimportant ones. This strategy not only improves efficiency, but also encourages initially less informative patches to gradually accumulate more class-relevant knowledge, a phenomenon we refer to as the $\textit{Re-Birth}$ effect, thereby effectively balancing class-agnostic and class-specific knowledge. Experimental results show that PRI achieves up to 10$\times$ faster inversion than standard $\textit{Dense Model Inversion}$ (DMI) and 2$\times$ faster than SMI, while consistently outperforming SMI in accuracy and matching the performance of DMI.

Real-time jet tagging is critical for identifying short-lived particle decays in the high-throughput detectors of the Large Hadron Collider, where real-time trigger systems which are responsible for deciding which collision events to store impose strict latency and accuracy constraints. Transformer architectures achieve the highest jet tagging accuracy when compute is unconstrained, but their quadratic self-attention cost places them orders of magnitude beyond the trigger budget. Existing efficient variants reduce this cost by compressing the attention matrix or restricting it to ordered local windows, at the price of the explicit particle-particle interactions that drive substructure identification. To address this limitation, we introduce the Patch Hierarchical Attention Transformer (PHAT-JeT), which combines two mechanisms: a physics-inspired geometric message-passing module that encodes local detector-plane structure, and a hierarchical patch-based attention scheme that computes exact attention within small particle groups while preserving global context through lightweight patch-token communication. Within this compute budget, PHAT-JeT achieves state-of-the-art accuracy and background rejection among resource-constrained jet tagging models on four benchmarks (hls4ml, JetClass, Top Tagging, and Quark-Gluon). Our code is available at https://anonymous.4open.science/r/PHAT-JeT-540B/README.md