paperlist.ai

AI conference papers

AI conference paper library

Search papers from 30+ top conferences, including NeurIPS, ICLR, ICML, CVPR, ACL and AAAI. Filter by keyword, then open any paper for its abstract, authors and links.

NeurIPS 2026

Prediction-Augmented Trees for Reliable Statistical Inference

Vikram Kher, Argyris Oikonomou, Manolis Zampetakis

Machine learning (ML) is increasingly central to scientific discovery, but translating ML predictions into rigorous scientific claims still requires statistical inference on gold-standard labeled data. A setting that has emerged across the sciences provides the analyst with a small labeled sample, a much larger unlabeled sample, and a pre-trained predictor, with the goal of constructing a confidence interval for a quantity of interest, as was formalized in Angelopoulos et al. (2023). In this ML-driven scientific discovery literature, labeled data is typically the binding constraint on precision while computation is comparatively unconstrained, motivating the design question of how additional computation can extract more inferential value from a fixed labeled set, a fixed unlabeled set, and a fixed predictor. To address this question, we propose the *Prediction-Augmented Residual Tree* (PART) estimator. PART takes the same inputs as the original PPI estimator and replaces their single global rectifier with locally computed rectifiers obtained from an adaptive partition of the feature space. We show that PART outperforms existing methods by producing tighter confidence intervals across real-world datasets from ecology, astronomy, and census reports, among other domains. We then describe and analyze PAQ, an estimator that arises when considering the limit of PART when the depth of its tree grows to infinity. Under appropriate assumptions in the input data, we show that the variance of PAQ shrinks at a rate of $O(N^{-1} + n^{-4})$, showing that there are settings where the rate of $O(N^{-1}+n^{-1})$ of existing methods can be provably broken.

NeurIPS 2026

Prediction Under Imperfect Compression: A Theory of Approximate MDL

Qian Li, Xinyu Mao, Shang-Hua Teng, Guangxu Yang

Minimum Description Length (MDL) formalizes the principle of Occam's razor by optimizing the total description length: $L(\mathrm{model})+L(\mathrm{data} \ | \ \mathrm{model})$. For sequential prediction, the MDL method repeatedly selects a model with a minimum objective score of the observed prefix for the next step prediction. Classical MDL prediction theory shows that exact optimization of the MDL objective indeed provides a strong compression guarantee that supports reliable prediction. However, practical machine learning usually can only find models by approximately optimizing the objective function. To bridge this gap, this paper addresses the following fundamental question: \emph{Under what forms of approximation and regularization does approximate MDL still guarantee reliable sequential prediction?} This work offers a principled characterization. We prove that for any approximation with \emph{additive} slack \(C\) of the more general form of the \emph{balanced} MDL objective: $\lambda\cdot L(\mathrm{model})+L(\mathrm{data} \ | \ \mathrm{model})$, the cumulative expected squared prediction error is finite for all $\lambda\ge1$. The case $\lambda>1$ is proved by an affinity-telescoping argument, while the boundary case $\lambda=1$ is proved by a likelihood-ratio stopping argument based on exact static MDL bounds. Our results establish that classical MDL regularization remains robust to any fixed additive optimization error. Furthermore, we establish that our characterization of the approximate MDL framework is sharp: When $0<\lambda<1$, overfits can happen to incur infinite cumulative expected error in the universal class of estimable measures, and hence a strong form of model-complexity regularization is necessary. In addition, model selection may fail in every regularized regime $\lambda >0$, under multiplicative approximation, and thus, additive approximation is both sufficient and essential.

NeurIPS 2026

Predicting directional flexibility in proteins

Vsevolod Viliuga, Leif Seute, Matteo Tadiello, Nicolas Wolf, Frauke Gräter, Arne Elofsson

Predicting protein dynamics is a long-standing problem in computational structural biology. Often, protein function critically depends on local directed motions, such as hinge movements, catalytic loop rearrangements and domain reorientations, which can be characterized by directional flexibility and correlated structural motions of the protein backbone. While Molecular Dynamics (MD) simulations provide an established but often prohibitively expensive approach, recent deep generative models aim to reduce this cost by directly predicting conformational ensembles, emulating MD. However, due to their large size and the need to generate several states until the derived dynamical properties converge, these models remain expensive. In this work, we propose a fast SE(3)-equivariant graph neural network to directly predict dynamical descriptors, such as directional backbone flexibility and pairwise dynamic correlations, from an equilibrium structure. In a series of experiments, we show that our model matches the accuracy of substantially larger ensemble generation models while being orders of magnitude faster, and demonstrate that the proposed equivariant architecture is especially well-suited for capturing anisotropic motions in proteins.

NeurIPS 2026

Predicting and improving test-time scaling laws via reward tail-guided search

Muheng Li, Jian Qian, Wenlong Mou

Test-time scaling has emerged as a critical avenue for enhancing the reasoning capabilities of Large Language Models (LLMs). Though the straight-forward ``best-of-$N$'' (BoN) strategy has already demonstrated significant improvements in performance, it lacks principled guidance on the choice of $N$, budget allocation, and multi-stage decision-making, thereby leaving substantial room for optimization. While many works have explored such optimization, rigorous theoretical guarantees remain limited. In this work, we propose new methodologies to predict and improve scaling properties via tail-guided search. By estimating the tail distribution of rewards, our method predicts the scaling law of LLMs without the need for exhaustive evaluations. Leveraging this prediction tool, we introduce Scaling-Law Guided (SLG) Search, a new test-time algorithm that dynamically allocates compute to identify and exploit intermediate states with the highest predicted potential. We theoretically show that SLG approaches a full-information oracle at a reduced scale and, under additional regularity conditions, achieves vanishing matched-budget regret; in a Gaussian two-stage model, SLG can also match BoN with a polynomially larger sampling budget. Empirically, we validate our framework across different LLMs and reward models, confirming that tail-guided allocation consistently achieves higher reward yields than Best-of-$N$ under identical compute budgets.

NeurIPS 2026

Predicting What Changes: Causal Delta World Models for Risk-Aware LLM Agent Planning

Guo Yue, Yang Liu, Donghui Zhang, Tang Qingkang, Rong Fu, Li Aoyu

Modern LLM agents repeatedly fall into a familiar trap: they execute irreversible actions whose consequences they did not anticipate, such as confirming a non-refundable booking or sending an unrecoverable message. World models are a natural remedy, yet two dominant lines of work both fail this use case. Generative pixel and HTML world models pay a heavy compute cost to produce vivid but often unfaithful continuations of the full observation, while reward and value models only summarise how good an action is and tell the agent nothing about what it will change or whether the change can be undone. We argue that what an agent really needs is a model of what changes, not of what comes next. We introduce Causal Delta World Models (CDWM), which represent each transition as a typed structured delta over an entity, attribute and relation graph, and explicitly factor out reversibility, irreversible cost and prediction confidence. A learnable causal sparsity mask binds each action class to the delta dimensions it can plausibly affect, providing a strong inductive bias for unseen actions and environments. We post-train CDWM with a verifiable composite reward that aligns delta accuracy, action following, calibration error and closed-loop task success. The resulting world model is consumed by a risk-aware closed-loop planner that prunes catastrophic candidates, performs short-horizon model-predictive control, and abstains when confidence is low. On four interactive benchmarks (WebArena, Mind2Web, ALFWorld, ScienceWorld) and two visual control suites (Procgen and Atari 100k), CDWM trained on four NVIDIA A100 (80 GB) GPUs improves task success by 6.0 to 11.7 absolute points over strong agentic and world model baselines, reduces irreversible failures by 47 to 71 percent, lowers token cost per successful trajectory by 27 to 45 percent, and shows substantially better generalisation to unseen websites and rule perturbations. We conclude that the right unit for an agent's world model is the structured delta and not the full observa

NeurIPS 2026

Predicting Quantization Price for Selecting PTQ Configurations Before Deployment

Junbin Qiu, Jian Mu, Weitong ZHANG, Yao Shu

Weight-space post-training quantization (PTQ) must choose finite formats, granularities, quantizer families, transformations, and bits before the completed quantized model reveals its output-distribution drift. Existing PTQ methods predict important pieces of this degradation, including reconstruction error, Hessian sensitivity, transformation effects, and downstream loss, but these pieces are usually scored after fixing the quantization geometry or inside separate configuration families. We formulate weight-space PTQ as pre-deployment configuration selection using priced layer-output error. Each admissible layer configuration is treated as an error generator with a deployment cost, which induces a layer-output error covariance $\boldsymbol{\Sigma}_l(\alpha_l)$, and the full-precision model prices that covariance by downstream curvature, $\hat{\rho}_l(\alpha_l)=\frac{1}{2}\operatorname{Tr}\left(\hat{\mathbf{H}}_l\,\hat{\boldsymbol{\Sigma}}_l(\alpha_l)\right)$. The price follows from full-precision-to-quantized forward KL, whose first-order term cancels at the reference model. It turns reconstruction and diagonal scores into reduced proxies that drop price factors, while finite formats, codebooks, granularities, and equivalent transformations become comparable candidates through the covariances they induce and the costs they pay. A trace reduction then yields a calibration-time price table and a budgeted price-guided selector, making fixed-geometry bit allocation a special case rather than the organizing problem.

NeurIPS 2026

Predicting Only from Selected Evidence: A Tempered Product-of-Experts Bottleneck for Auditable EEG Diagnosis

Yinghao WANG, Shujian Yu, Duc-Han LE, Zhikai Yu, Changming Wang, Van-Tam Nguyen

Pretrained EEG backbones improve transfer performance, but downstream diagnostic heads remain difficult to audit: predictions are typically computed from unconstrained hidden representations, while explanations are often generated only post hoc. We introduce tPoE-EIB, an $\textit{evidence-information bottleneck}$ head for adapting EEG backbones under an evidence-only prediction constraint. tPoE-EIB selects temporal and channel-specific evidence, maps the resulting summaries to Gaussian experts over a shared latent variable, and fuses them via a tempered product-of-experts posterior. The classifier conditions exclusively on this latent variable, yielding an explicit decision pathway whose information flow is constrained by the expected posterior KL divergence. This formulation induces a tractable supervised objective with an information-rate penalty, while the closed-form tempered posterior mitigates overconfident aggregation of correlated evidence sources. We evaluate tPoE-EIB on pretrained EEG foundation model backbones across six diagnostic tasks: event-type classification, abnormality detection, seizure detection, cognitive decline staging, depression screening, and cerebrovascular disease classification. The evaluation spans public benchmarks and in-house clinical cohorts, covering both binary screening and fine-grained staging, as well as sparse and dense electrode montages. tPoE-EIB maintains competitive balanced accuracy while outperforming representative post-hoc explanation methods on selection-faithfulness audits, including insertion–deletion and gate-causality tests. Its structured posterior further supports integration-faithfulness evaluations, including expert-drop, posterior-reliance, and expert-disagreement tests. Overall, these results suggest that evidence-only, rate-limited fusion is a practical approach to building auditable diagnostic models on top of frozen EEG foundation backbones.

NeurIPS 2026

Predict-Project-Renoise: Sampling Diffusion Models under Hard Constraints

Omer Rochman Sharabi, Gilles Louppe

Diffusion models cannot enforce hard constraints, yet applications in the physical sciences demand exact satisfaction of conservation laws, boundary conditions, and observational consistency. In this work, we identify a corrector kernel whose unique stationary distribution is the constrained marginal at each noise level, and approximate it by iteratively projecting through the denoiser and renoising via the forward kernel. The resulting Predict-Project-Renoise (PPR) algorithm enables sampling from pretrained diffusion models under hard constraints. Its three components are each necessary: projecting through the denoiser keeps samples close to the data manifold, while renoising and iterating drive samples toward the constrained marginal. On 2D distributions, the Kuramoto-Sivashinsky equation, and global weather forecasting with a $10^8$-dimensional atmospheric model, PPR simultaneously achieves low constraint violations and high distributional fidelity, a combination that existing methods fail to deliver.

NeurIPS 2026

Preconditioned Implicit Midpoint Langevin Sampling for Non-Smooth Bayesian Imaging

Sichen Wang, Zhipeng Lu

We introduce P-IMLA, a preconditioned implicit-midpoint Langevin sampler for high-dimensional log-concave targets with non-smooth regularization. The obstruction is that the large steps enabled by matrix preconditioning are precisely where preconditioned Euler--Maruyama inflates stationary variance. Implicit midpoint removes this surplus via a Cayley-transform identity, making P-IMLA Gaussian-exact at any step size. Beyond Gaussians, a Möbius-monotonicity argument gives an $M$-Wasserstein rate governed by $\kappa_{\rm eff}$ rather than the $\kappa_f$ of unpreconditioned IMLA, while backward-error analysis shows drift-only rather than Euler-type diffusion bias. An $M$-norm Moreau envelope yields a proximal non-smooth variant with a three-term error budget. Controlled Gaussian and TV-regularized imaging experiments confirm the predicted separation: the unpreconditioned IMLA degenerates as $\kappa_f$ grows, whereas P-IMLA remains $\kappa_{\rm eff}$-governed and near variance-exact.

NeurIPS 2026

Preconditioned Flow Matching

Shadab Ahamed, Eshed Gal, Md. Shahriar Rahim Siddiqui, Simon Ghyselincks, Moshe Eliasof, Eldad Haber

Flow matching (FM) learns vector fields by regressing stochastic velocity targets along intermediate distributions $p_t$. We identify a geometric optimization bottleneck in this regression problem: when the covariance $\Sigma_t$ of $p_t$ is ill-conditioned, gradient-based training rapidly fits high-variance directions while making slow progress along low-variance ones. In an exactly solvable Gaussian setting, we prove that the excess risk is weighted by $\Sigma_t$, and that both gradient descent and stochastic gradient descent inherit condition-number-dependent convergence. We then extend the analysis to Gaussian mixtures, showing that multimodality does not average away this effect; instead, the slowest and worst-conditioned component can control optimization. Motivated by this analysis, we propose \emph{preconditioned flow matching}, a precondition-then-match framework that transforms the target distribution into a more isotropic representation, trains the main flow in the transformed space, and maps generated samples back through the inverse transformation. We show theoretically that preconditioning reshapes the intermediate FM path and improves its conditioning. Across controlled Gaussian and Gaussian-mixture experiments, latent MNIST and other high resolution image datasets up to $512{\times}512$ resolution, preconditioning improves path-conditioning diagnostics, low-eigenvalue recovery, FID, MMD, precision, and recall. Compute-matched baselines and preconditioner-quality ablations further show that the gains are not explained merely by additional preconditioner parameters, but by improved geometry of the downstream flow matching problem.

NeurIPS 2026

Precision-Pyramid: Towards real-time neural decoding for fault-tolerant quantum computing

Zhenhao Zhong, Ge Yan, SHANCHUAN LI, Pengyue Ma, Yuxuan Du

Real-time quantum error correction (QEC) is a critical bottleneck for fault-tolerant quantum computing due to strict hardware latency constraints. While neural decoders achieve state-of-the-art logical accuracy, their reliance on high-precision, compute-intensive inference precludes real-time deployment. Conversely, uniformly quantizing these models to extreme low-bit regimes for FPGAs triggers a catastrophic collapse in decoding accuracy. To overcome this precision-accuracy bottleneck, we propose Precision Pyramid ($\mathsf{PP}$), a hardware-algorithm co-designed neural decoder applicable to both surface and BB codes. $\mathsf{PP}$ features a monotonically escalating activation precision hierarchy built upon a globally weight-binarized (W1) foundation. By scaling intermediate activations down to INT2 for the massive perception stages and reserving higher precision strictly for the lightweight final logical decision, this hierarchy effectively shifts over $99\%$ of the computational workload to abundant FPGA look-up tables. Comprehensive evaluations on both simulated and hardware-calibrated noise data demonstrate that $\mathsf{PP}$ consistently suppresses logical errors below highly optimized classical baselines (e.g., MWPM and Relay-BP) while seamlessly satisfying stringent sub-microsecond latency budgets.

NeurIPS 2026

Precision As You Need: Stochastic Computing Is a Dense Adaptive Quantizer

Haoran Jin, Kangqi Zhang, Jirong Yang, Barry Lyu, Qiuyi Ding, Ruijie Gao, Nathan Bleier

Matrix multiplications dominate the inference cost of modern transformer-based vision models, yet existing efficiency techniques such as post-training quantization and mixed-precision inference are largely limited to the small set of fixed-width formats—INT4, INT8, BF16, and FP16—supported by conventional accelerators. We revisit stochastic computing (SC) as a way to lift this constraint: viewed as a dense adaptive quantizer, SC controls precision by bit-stream length L rather than a fixed datapath, while each multiplication reduces to a single AND/XNOR gate. We build a GPU library that emulates SC matrix multiplication at scale, exposes stream lengths as first-class kernel arguments, and evaluates SC end-to-end on image classification, object detection and instance segmentation, class-conditional image generation, and visual world-model planning. On top of this substrate, we develop a dynamic per-row mixed-precision policy that assigns stream length per token or group at matched average budget, requires no retraining, and uses the same SC hardware across schedules. Across tasks, SC remains competitive with fixed-format INT quantization at matched bit budgets, while per-row mixed precision helps maintain accuracy at lower average stream lengths. These results provide software-level feasibility evidence that SC can serve as a dense-precision substrate for fine-grained mixed-precision inference on modern vision transformers.

NeurIPS 2026

Precise but Uncoupled: Reviewer Precision Does Not Guarantee Critique Uptake in Multi-Agent Math Reasoning

Chih-Hsuan Yang, Jingyan Jiang, Vikram Vasudevan, Cheng-Hau Yang, Huihuo Zheng, Benoit Cote, Le Chen, Eliu Huerta, Venkatram Vishwanath, Ian Foster, Rajeev Thakur

Many recent math- and science-oriented agent systems adopt hierarchical designs with specialized reviewer roles, motivated by the idea that routing critique through a dedicated review stage should help turn wrong candidates into correct ones on hard problems. We study that expectation on 4,181 verifier-grounded Omni-MATH problems, a hard ten-tier benchmark with enough headroom to separate protocols, using matched gpt-oss-120b actors as the primary actor family. On tiers 1-2, multi-agent collaboration adds at most about 2 percentage points over the matched single-agent baseline. From tier 4 onward, the gains open sharply, reaching about 10 to 20 percentage points on tiers 6-9. In that regime, broadcast-style peer discussion attains higher final accuracy than a planner-executor-reviewer pipeline (PER). We use that divergence as a starting observation and ask when reviewer quality translates into effective solver updates. On this benchmark, the PER-broadcast accuracy gap is not accounted for by reviewer precision alone. Here reviewer precision asks how often a reviewer warning truly points to a real error. PER's reviewer has higher precision (0.861 vs. 0.644), yet correct critique is much less likely to change the next candidate the protocol carries forward, and reviewer-guided repair is correspondingly lower. These results show that reviewer detection quality and critique uptake are empirically separable. Within matched PER interventions, forcing explicit acknowledgment lowers final accuracy instead of improving follow-through, while placing reviewer guidance directly in the solver's working context partially improves follow-through without closing the gap. Taken together, these interventions point in the same direction: critique appears more likely to be acted upon when it is presented more directly in the solver's working context, although this evidence is directional rather than causal. Under reviewer-centric evaluation, a system can look strong at spotting errors yet still fail to solve more problems if the protocol does not act on those critiques.

NeurIPS 2026

Precise Debugging Benchmark: Is Your Model Debugging or Regenerating?

Miaosen Chai, Wang Bill Zhu, Shangshang Wang, Yejia Liu, Song Bian, Honghua Dong, Willie Neiswanger, Robin Jia

Unlike code completion, debugging requires localizing faults and applying targeted edits. We observe that frontier LLMs often hack unit tests by regenerating correct but over-edited solutions during debugging. To evaluate how far LLMs are from precise debugging, we introduce Precise Debugging Benchmarking (PDB), a general, dataset-agnostic framework that converts any coding dataset into a debugging benchmark with precision-aware evaluation. PDB generates buggy programs by synthesizing verified atomic bugs and composing them into multi-bug programs. We define two novel metrics, edit-level precision and bug-level recall, which measure how many necessary edits are made and how many bugs are resolved. We release two evaluation benchmarks: PDB-Single on single-line bugs and PDB-Wild on multi-line and repository-level bugs. Experiments show that frontier models, such as GPT-5.1-Codex and DeepSeek-V3.2-Thinking, achieve unit-test pass rates above 76% but exhibit precision below 45%, even when explicitly instructed to perform minimal debugging on single-line bugs. Iterative and agentic debugging strategies do not substantially improve precision or recall, highlighting the need to rethink post-training pipelines for coding models.

NeurIPS 2026

PreFT: Prefill-only finetuning for inference efficiency

Andrew Lanpouthakoun, Aryaman Arora, Zhengxuan Wu, Dhruv Pai, Benjamin Keigwin, Dan Jurafsky, Chris Potts

Large language models can now be personalised efficiently at scale using parameter efficient fine-tuning methods (PEFTs), but \emph{serving} user-specific PEFTs harms throughput, even with specialised kernels and memory management techniques. This is because, theoretically and empirically, a mismatch exists between prefill (processing a large number of tokens at once) and decode (generating a single token autoregressively): the latter has far lower throughput when serving multiple adapters. Rather than optimising performance relative to parameter count, for efficient multi-adapter serving, we instead ought to optimise performance relative to \textit{serving throughput}. We therefore propose \textbf{\texttt{PreFT}} (Prefill-only Finetuning), wherein we only apply the adapter to prefill tokens and discard it afterwards. \texttt{PreFT} significantly increases throughput with minimal effect on performance. We develop and release an efficient implementation of two prefill-only PEFTs, LoRA and ReFT, on the vLLM inference engine. We first show that serving multi-user \texttt{PreFT}s is vastly more efficient than traditional PEFTs ($1.90\times$ the throughput when serving $512$ adapters on Llama 3.1 70B). Then, we compare the performance of prefill-only vs.~all-token adapters on a variety of supervised finetuning and reinforcement learning tasks with LMs at varying scales. On SFT, we observe that the evaluation loss of \texttt{PreFT}s is higher, but can be compensated by increasing rank with nearly no reduction in throughput. On RL, we consistently find that \texttt{PreFT}s approach parity with standard PEFTs. Together, this work validates prefill-only adaptation of LLMs as a more favourable accuracy--throughput tradeoff than existing PEFTs for personalised serving.

NeurIPS 2026

PreDiff: Sequential Recommendation by Denoising Preference Distributions

Yaoqi Chen, Jianjin Zhang, Qi Chen, Weihao Han, Yujing Wang, Hao Sun

Sequential recommendation predicts which item a user will interact with next. A key property of this task is that user preferences are concentrated: only a small cluster of items is relevant at a given moment. Recent diffusion-based methods add noise to the item's embedding and score candidates by embedding similarity. Since similar items have similar embeddings, their score gaps are inherently small and easily overwhelmed by noise, causing the most relevant candidates to become indistinguishable first. We prove that moving diffusion to distribution space, where each item receives independent noise, preserves rankings better and makes the reverse denoising process easier. This shift, however, loses the semantic structure of embedding space and requires richer supervision than one-hot labels. We propose PreDiff (**Pre**ference **Diff**usion), which performs diffusion in distribution space and introduces two components to address these problems: Distribution-Guided Sparse Projection projects the preference scores into embedding space while filtering out low-relevance noise, and soft targets encode inter-item similarity to teach fine-grained ranking within the preference cluster. On four benchmarks, PreDiff achieves 8%--17% relative improvement over existing methods.

NeurIPS 2026

PreCoMem: Predictive Cognitive Memory for Self-Evolving Long-Term Dialogue Agents

Jian Zhong, Zeyu Liu, Pingchuan Cao, Rongduo Han, Shunye Tang, Guohuan Xie, Mingyuan Qin, Ran Ji, Xiao Liang, Haining Zhang, Wei Wang

Long-horizon dialogue agents must keep user memories stable enough for personalization while adapting when the user genuinely changes. Existing memory systems often append every observation or overwrite old entries by heuristic rules, which makes transient noise hard to distinguish from real belief drift. We introduce PreCoMem, a memory consolidation framework inspired by predictive coding that represents memory as confidence-weighted beliefs about the user's state. Its core signal, Effective Surprisal, combines semantic deviation and contradiction evidence, then downweights this signal when retrieval is diffuse and unreliable. According to effective surprisal, a two-threshold gate maps each turn to one of three deterministic updates: MAINTAIN reinforces supported beliefs, PROFILE stores ambiguous cues as low-confidence hypotheses, and CORRECT softly decays reliably contradicted facts. The PROFILE stage acts as an explicit waiting room, preventing premature commitment to noisy evidence. Experiments on LoCoMo, LongMemEval, and PersonaMem-v2 show consistent state-of-the-art accuracy; our new DiMoBench confirms that PreCoMem adapts to non-stationary belief drift without over-reacting to noise. The code is available at: \url{https://anonymous.4open.science/r/PreCoMem-9000}.

NeurIPS 2026

Practical Non-Stationary Graph Gaussian Processes

Senanayak Sesh Kumar Karri, Viacheslav (Slava) Borovitskiy

Matern Gaussian processes offer a principled framework for learning and uncertainty quantification on graphs. However, they struggle with heterogeneous networks. A central challenge is that variance at a node can represent genuine signal that should strongly propagate (diffuse) to neighbors, mere noise that should be isolated, or a mixture of both. While this problem has been studied in continuous spatial domains, directly translating those approaches to graphs leaves a free parameter per node, causing extreme over-parameterization. To resolve this, we introduce non-stationary Variance-Coupled Gaussian Processes (VCGPs). These models use an auxiliary signal (often easily derived from historical data) to determine relative diffusivity between nodes, tied to the regression data via a single learned hyperparameter, $\gamma$. The sign of $\gamma$ captures whether the auxiliary signal is directly or inversely proportional to node diffusivity, while its magnitude calibrates this relationship.We formally prove that $\gamma = 1$ is the unique point where VCGPs are stationary up to rescaling, ensuring that varying $\gamma$ results in genuinely non-stationary correlations. Across five real-world datasets, our VCGPs match or outperform both stationary and heavily parameterized non-stationary baselines.

NeurIPS 2026

Practical Estimation of the Bayes Optimal Fairness-Accuracy Tradeoff with Soft Labels

Mohit Sharma, Okan Koc, Amit Jayant Deshpande, Takashi Ishida, Gang Niu, Rajiv Ratn Shah, Masashi Sugiyama

There is a fundamental limit to the best prediction error of any model for a given data distribution. In classification, the Bayes error quantifies this limit and serves as both a benchmark for trained models and a criterion to detect overfitting. The state-of-the-art Bayes error estimation techniques for binary classification use only soft labels (positive class probabilities) and require no input features or auxiliary training. We extend such notions to constrained classification problems, such as fairness. Fair classification is an important case of constrained or multi-objective classification. It requires a model not only to be accurate but also to yield fair outcomes (equal or near-equal true positive rates) across different demographic groups (e.g., race and gender). This leads to an inherent fairness-accuracy tradeoff or a Pareto frontier that is used to compare fair classifiers against each other. The focus of our work is to estimate the fundamental limit for this fairness-accuracy tradeoff. We derive an explicit expression for the fair Bayes error rate (i.e., best achievable error rate subject to given fairness constraints), and construct consistent, instance-free estimators for it using only soft labels and group information. We provably bound the bias and the sample complexity of our estimators. Empirically, our method produces a robust, low-variance estimate of the optimal fairness-accuracy trade-off curve, even when the soft labels are noisy and derived from a classification model on hard labels.

NeurIPS 2026

Practical Adversarial Attacks on Stochastic Bandits via Fake Data Injection

Qirun Zeng, Eric He, Richard Hoffmann, Xuchuang Wang, Jinhang Zuo

Adversarial attacks on stochastic bandits have traditionally relied on some unrealistic assumptions, such as per-round reward manipulation and unbounded perturbations, limiting their relevance to real-world systems. We propose a more practical threat model, Fake Data Injection, which reflects realistic adversarial constraints: the attacker can inject only a limited number of bounded fake feedback samples into the learner's history, simulating legitimate interactions. We design effective attack strategies under this model, explicitly addressing both magnitude constraints (on reward values) and temporal constraints (on when and how often data can be injected). Our theoretical analysis shows that these attacks can mislead a class of bandit algorithms into selecting a target arm in nearly all rounds while incurring only sublinear attack cost. Experiments on synthetic and real-world datasets validate the effectiveness of our strategies, revealing vulnerabilities in stochastic bandit algorithms under practical adversarial scenarios.

Keep browsing

More papers from top AI conferences

30+ top conferences, by conference, year, and topic.