UAI 2026 论文列表

Large multi-modal language models are increasingly deployed in high-stakes domains, making well-calibrated uncertainty essential. Traditional Bayesian methods approximate posteriors over all model weights, which becomes intractable for modern large models. For this reason, recent work instead considers Bayesian low-rank adaptation to enable tractable posterior approximation. Due to a lack of a standardized benchmark to evaluate these approaches, it remains unclear where these methods provide meaningful benefits. To fill this gap, we introduce Bayesian Adaptation Gym (BAG), a benchmark for the Bayesian adaptation of multi-modal language models. BAG provides reference implementations of classic Bayesian baselines and state-of-the-art adaptation methods, along with a multi-modal dataset and task suite designed to probe calibration, robustness under distribution shift, and decision-making under uncertainty via active learning. Using BAG, we conduct and report extensive experiments across model sizes, datasets, and tasks to highlight the successes and failures of current Bayesian adaptation approaches. To enable further research, BAG is fully open source: https://github.com/SRI-CSL/BayesAdapt.

Sampling from high-dimensional, non-log-concave distributions with unnormalized densities is a fundamental challenge in machine learning, particularly when the exact gradient of the potential is unavailable and must be approximated via stochastic gradients that exhibit high variance under a fixed budget of gradient computations per iteration. Although variance reduction techniques such as SGD with momentum, STORM, and PAGE have demonstrated improved convergence properties in non-convex optimization, their implications for sampling from non-log-concave distributions remain largely unexplored. In this work, we develop the first unified analysis of these estimators for sampling from non-log-concave distributions. We establish improved non-asymptotic convergence rates in ε-relative Fisher information and, under a Poincaré inequality assumption, in squared total variation distance, and further prove weak convergence to the target distribution. We extend our analysis to solving inverse problems with score-based generative priors. We empirically validate our theory and demonstrate that, under a fixed gradient computations per iteration, variance-reduction techniques consistently improve sample quality in two standard imaging applications.

In this paper, we construct confidence interval (CI) for the test-unfairness of a group-fairness aware classifier trained by Stochastic Gradient Descent (SGD) type algorithms. Since these algorithms are almost invariably used to train modern machine learning models, the learned parameters—and consequently their fairness—are inherently random. We quantify this randomness, which can have far-reaching consequences, especially for sensitive applications. Viewing fairness-aware classification through the lens of constrained optimization allows us to leverage tools from stochastic approximation theory to establish the asymptotic normality of the estimated parameters. Existing work typically provides fairness guarantees either in expectation or for the exact minimizer of the constrained objective <cit.>, without accounting for the stochasticity induced by the optimization algorithm itself. To construct CIs for fairness in practice, we develop an easily parallelizable online multiplier bootstrap procedure. In doing so, we also extend theoretical guarantees for online bootstrap methods from unconstrained SGD to the constrained optimization setting which may be of independent interest. We illustrate the effectiveness of our approach through extensive simulations and experiments on benchmark datasets.

Symbolic regression (SR) has gained recent traction in AI-driven scientific discovery for learning closed-form physical laws. Yet existing methods are dominated by heuristic search or data-intensive approaches that often assume low-noise regimes and lack principled uncertainty quantification, while fully probabilistic SR formulations remain scarce. We introduce a scalable probabilistic framework for SR, VaSST, based on variational inference. VaSST uses soft symbolic trees, a continuous relaxation of symbolic expression trees in which discrete operator and feature assignments are replaced by probability distributions over allowable components. This transforms combinatorial symbolic search through an astronomically large expression space into efficient gradient-based optimization while preserving a coherent probabilistic interpretation. The learned soft representations induce posterior distributions over symbolic structures, enabling uncertainty quantification across plausible symbolic forms through posterior-aware symbolic model selection. On simulated experiments and the Feynman Symbolic Regression Database, VaSST achieves strong structural recovery and predictive accuracy compared to state-of-the-art competing SR methods.

We introduce kernel integrated R^2, a new measure of statistical dependence that combines the local normalization principle of the recently introduced integrated R^2 with the flexibility of reproducing kernel Hilbert spaces (RKHSs). The proposed measure extends integrated R^2 from scalar responses to responses taking values on general spaces equipped with a characteristic kernel, allowing to measure dependence of multivariate, functional, and structured data, while remaining sensitive to tail behaviour and oscillatory dependence structures. We establish that (i) this new measure takes values in [0,1], (ii) equals zero if and only if independence holds, and (iii) equals one if and only if the response is almost surely a measurable function of the covariates. Two estimators are proposed: a graph based method using K-nearest neighbours and an RKHS-based method built on conditional mean embeddings. We prove consistency and derive convergence rates for the graph-based estimator, showing its adaptation to intrinsic dimensionality. Numerical experiments on simulated data and a real data experiment in the context of dependency testing for media annotations demonstrate competitive power against state-of-the-art dependence measures, particularly in settings involving non-linear and structured relationships.

We prove the weighted Bayesian bootstrap, a method for approximate sampling of a posterior distribution, can be extended to sample from general constrained posterior distributions under mild assumptions. The method entails a simple algorithm that can take advantage of fast tools from convex optimization. Under regularity conditions, we show the asymptotic distribution of samples from the constrained weighted Bayesian bootstrap has a covariance matching the restricted maximum likelihood estimator, an efficient estimator. We assess the method empirically on a variety of constrained Bayesian problems, demonstrating broad applicability of the method as well as advantages over existing peer methods. The constrained weighted Bayesian bootstrap quickly samples from constrained posteriors, providing adequate uncertainty quantification for problems typically solved via optimization methods designed to deliver only a point estimate. As a case study, using constraints required in European-style option prices, uncertainty estimates of an option pricing surface are derived with constrained weighted Bayesian bootstrap.

Goal-oriented Markov Decision Processes with unavoidable dead-ends pose a significant challenge to standard planning algorithms, as the goal cannot always be reached with probability 1. In certain applications, it can be desirable to optimize efficiency only over successful trajectories while enforcing a lower bound on goal reachability, yet no existing method directly addresses this conditional formulation, whose non-linear structure makes direct optimization difficult. To overcome this, we introduce a tractable approximation. Working within an unconstrained utility-maximization framework, we derive an analytical formula that determines a value for the goal-reward parameter sufficient to guarantee that the optimal policy respects the requested reachability threshold. The proposed approach is evaluated on two common benchmark domains, Aircraft Routing and Exploding Blocksworld, and compared against state-of-the-art constrained optimization methods, specifically i-dual and Chance-Constrained. Results show that our method offers a compelling trade-off: it significantly reduces the expected cost compared to conservative lexicographic approaches (i.e. i-dual), while avoiding the failure-seeking bias inherent to methods modelling artificial zero-cost give-up action (i.e. Chance-Constrained).

Bayesian networks are well-known probabilistic graphical models that enable explainable reasoning under uncertainty. In many domains, data are scarce and fragmented across institutions, motivating collaboration and the sharing of learned models rather than individual-level records to increase the available evidence and reduce bias. Releasing a model, however, can still reveal sensitive information: adversaries may mount membership inference attacks to determine whether a specific individual contributed to the training. A common mitigation strategy is to inject noise into the learned model before its release. This perturbation may compromise the quality and interpretability of subsequent inferences. We investigate how a Bayesian network can be effectively masked rather than perturbed to protect it against attacks by leveraging credal networks, the imprecise version of Bayesian networks, allowing us to employ sets of parameters instead of exact point estimates. We formalize and compare various masking and attack strategies and investigate how privacy leakage depends on these strategies. Finally, we analyze how privacy and utility can be traded off in credal naive Bayes models, comparing them with the common noise-injection baseline. Results indicate that credal masking provides principled protection, substantially reducing attack success while yielding fine-tuned privacy-accuracy trade-offs.

Querying expert predictions or human feedback incurs explicit, often heterogeneous costs—a challenge central to crowdsourcing and LLM training. We study online learning with expert advice under observation costs, where the learner must adaptively select which experts to query (p_i is expert-dependent) to minimize cumulative cost (losses plus query costs). While classical bandits and recent paid observation models have advanced the field, they fail to capture the rich cost-adaptive selection landscape and do not provide tight, problem-dependent performance guarantees for heterogeneous costs. This paper provides the first comprehensive characterization. For adversarial losses, we prove tight minimax regret Θ(m∈{1,…,K}min{√(K/m Tlog K) + (m-1)p̅T}), interpolating between bandit and full-information regimes and revealing the optimal observation budget, where p̅ is a tight upper bound of p_is. For stochastic losses, our novel m-ELIM algorithm achieves instance-dependent regret of O(log T/m∑_i ≠ i^*1/Δ_i + (m-1)p̅T), Δ_i being the suboptimality gap of the i-th arm, showing how gap structure and costs interact. Matching lower bounds (adversarial and stochastic) establishes minimax optimality. These results provide the first rigorous framework for optimally allocating annotation budgets across heterogeneous workers in crowdsourcing systems, directly informing cost-effective strategies for collecting human feedback in adaptive learning applications.

Density aggregation is a central problem in machine learning, for instance when combining predictions from a Deep Ensemble. The choice of aggregation remains an open question with two commonly proposed approaches being linear pooling (probability averaging) and geometric pooling (logit averaging). In this work, we address this question by studying the normalized generalized mean of order r ∈ℝ∪{-∞,+∞} through the lens of log-likelihood, the standard evaluation criterion in machine learning. This provides a unifying aggregation formalism and shows different optimal configurations for different situations. We show that the regime r ∈ [0,1] is the only range ensuring systematic improvements relative to individual distributions, thereby providing a principled justification for the reliability and widespread practical use of linear (r=1) and geometric (r=0) pooling. In contrast, we show that aggregation rules with r ∉ [0,1] may fail to provide consistent gains with explicit counterexamples. Finally, we corroborate our theoretical findings with empirical evaluations using Deep Ensembles on image and text classification benchmarks.

Causal discovery algorithms such as PC often rely on faithfulness; yet XOR-type relations expose violations that PC and its weakened-faithfulness variants can miss. In such cases, a target variable X may be independent of each variable Y_i individually, while becoming dependent on Y_i once the remaining variables are conditioned on. We call a minimal set with this property a minimal dependence (MD) set: no proper subset has the same dependence pattern. MD sets of size at least two violate faithfulness. We introduce minimal-dependence faithfulness and a corresponding orientation condition, characterize the resulting DAG structure by showing that dependent members of an MD set connect to X directly or through colliders, and define the associated MD-equivalence representation. We then present MD-PC, a PC-style algorithm that detects such faithfulness violations and, under the proposed assumptions, returns the corresponding candidate class of DAGs.

This paper investigates the statistical efficiency of algebraic expressions ("estimands") used to answer causal queries. We first examine structural rules, developing a partial dominance relationship for front-door estimands that extends recent results on back-door estimands. In many models, however, structural rules alone may be insufficient to select an estimand. For such cases, we propose empirical techniques for estimating and comparing the variance of different estimands using both the graph and observational data: 1) a bootstrap-based method, and 2) a computationally simpler yet practically effective method for discrete models which estimates the Fisher information matrices. We illustrate both methods’ effectiveness on a variety of causal diagrams and estimand forms.

Multi-agent reasoning and metacognitive strategies are widely used to improve large language model (LLM) performance, yet the functional role of metacognition remains unclear. Prior methods often treat metacognition as a mechanism for generating better or more diverse reasoning. In this work, we argue that metacognition primarily functions as an information routing and compression mechanism rather than a generator of new reasoning content. We introduce MC-MAS, an inference-time framework that separates problem solving from metacognitive arbitration, where independent solvers propose candidate answers and an arbiter critiques and consolidates their outputs. Using routing-centric metrics: semantic novelty, entropy reduction, and overconfident errors across four reasoning benchmarks and multiple model settings, we show that MC-MAS introduces limited novelty but consistently reduces uncertainty and overconfidence, with accuracy gains that are modest and dataset-dependent. These results indicate that reliable improvements arise from structured arbitration rather than reflection or increased sampling alone.

Variational Autoencoders (VAEs) typically assume a factorized Gaussian posterior and isotropic normal prior, implicitly imposing mean-field independence in the latent space. Rewriting the ELBO in copula form, we isolate a dependence-mismatch term that contributes to approximation and amortization gaps. We propose a Tree-Copula VAE that replaces the mean-field posterior with a tree-structured copula while retaining flexible learned marginals. The tree is inferred per datapoint via a Chow–Liu maximum-weight spanning tree, exploiting the monotonic relationship between mutual information and copula parameters. We further study an Average-of-Trees (AoT) copula prior for prior–posterior alignment and a rank-1 Gaussian-copula likelihood for low-rank observation correlations. Experiments on correlated dSprites and Fashion-MNIST show tighter importance-weighted bounds than mean-field VAEs when latent dependence is present, and highlight the role of prior alignment and likelihood modeling beyond mean-field inference.

Cooperative Multi-Agent Reinforcement Learning (MARL) has primarily focused on direct cooperation among agents. However, many real-world systems exhibit structurally asymmetric cooperation, where agents are physically constrained and cannot directly coordinate with those on whom they depend. In such settings, accurately estimating individual contributions under high uncertainty is challenging. We propose Marginal Advantage Estimation (MAE), a representation-level contribution estimation method for cooperative MARL under this structural isolation. MAE employs a synchronised feature-masking mechanism to evaluate marginal contributions without action-level counterfactual perturbations, thereby reducing variance and providing more informative learning signals. We provide a theoretical analysis of its bias–variance properties and demonstrate consistent performance improvements across 18 tasks in Partitioned MPE and Basilisk benchmarks over strong cooperative MARL baselines.

Negation Detection (ND) aims to extract negative statements with non-factual semantics, including negative scopes, xscopes, focuses, where (x)scopes are continuous spans governed by negative cues, and focuses are the most principal negated texts in scopes. Previous studies are confined to task-specific models, and failed to develop a unified solution for (x)scopes and focuses based on the correlations of their definitions, nor implemented the mutual promotion for each task. Moreover, existing work did not design fine-grained encoders, resulting in reliance on external features, such as syntactic parse trees. To address these issues, we construct a unified negation detection paradigm with Model-Agnostic Meta-Learning (MAML) to facilitate those ND sub-tasks. Besides, inspired by reading mechanism of humans, we synthesize Cascade Linear Attention Network (CLAN) by heterogeneous linear attentions with cascade coefficients as fine-grained encoder. Experimental results on several corpora demonstrate that CLAN can outperform the state-of-the-arts.

The pivotal challenge of handling uncertainty in artificial intelligence has prompted a growing focus on learning Bayesian network (BN) structures from data in recent years. Nevertheless, most of the existing methods such as hill-climbing, variational auto-encoder and graph neural network-based algorithms continue to encounter the issues like local optimum and exponential computational complexity, etc. In response to these challenges, we propose a Transformer-based approach for learning the BN structure (T-BNSL) from an information theory perspective. Specifically, we first establish the graph skeleton of a BN by employing conditional independence tests grounded in mutual information. Subsequently, we propose a hill-climbing method with decomposable BIC scoring function to generate potential directed acyclic graphs (DAGs) and evaluate the BIC scores of a subset of these DAGs. Lastly, we use the Transformer framework with a refined attention module to predict the BIC scores of the remaining DAGs, enabling the efficient identification of the DAG with the highest score. Experimental findings demonstrate that our approach significantly surpasses state-of-the-art competitors in terms of accuracy and efficiency by several orders of magnitude when learning the BN structure.

Deep Gaussian processes (DGPs) offer a principled Bayesian framework with hierarchical uncertainty propagation, but their reliable propagation of uncertainty and out-of-distribution (OOD) detection performance remains underexplored and often unreliable in safety-critical settings. In this work, we propose a novel kernel operating in both Euclidean and Wasserstein-2 space to better account for the geometry of representation learning spaces, thus circumventing a common pathology called feature collapse, whereby inliers and outliers get mapped to similar spaces. Empirically, our approach consistently improves OOD detection in convolutional image tasks and shows improved performance on tabular datasets.

Large-scale text-to-image diffusion models pose safety and compliance risks due to their ability to generate harmful or undesirable content. Machine unlearning seeks to remove specific concepts from pretrained models while preserving overall generative quality. Low-Rank Adaptation (LoRA) enables parameter-efficient targeted forgetting, but naive LoRA-based unlearning often induces distributional shift, degrading fidelity and destabilizing sampling. In this paper, we propose UnGuide, a LoRA-based unlearning framework with stability-aware adaptive guidance. Our method dynamically modulates classifier-free guidance (CFG) based on early denoising trajectory variance. By estimating discrepancies between base and LoRA-adapted noise predictions during initial diffusion steps, UnGuide interpolates per prompt between the frozen and adapted models. High-variance trajectories receive stronger adapted-model guidance to enforce forgetting, while low-variance trajectories rely on the base model to preserve fidelity. This mechanism enables controlled concept suppression without prompt embedding modification or external segmentation. Experiments on object erasure and explicit content removal demonstrate improved forgetting efficacy and superior fidelity retention compared to existing LoRA-based approaches.