UAI 2026 Papers

In multi-task learning (MTL) negative transfer is often considered as an optimization artifact, but it can also be viewed as a consequence of limited shared capacity and weak task redundancy. We investigate this effect through a Capacity–Redundancy (CR) identity that decomposes the sum of per-task predictive informations into joint predictive information that includes label redundancy defined via total correlation, and a residual coupling term that quantifies interference left unresolved by the shared representation. Additionally, we show two key results: (i) a clustering-gap decomposition that gives a necessary and sufficient condition for clustered sharing to outperform global sharing, and (ii) a gradient–TC bridge in a Gaussian multi-task model that formally justifies gradient cosine similarity as a proxy for redundancy ordering. Empirically, we estimate Δ from validation residual correlations, showing that clustered LoRA substantially reduces Δ, outperforms size-matched random partitions, and results in statistically significant gains with multi-seed confidence intervals.

Sharing the financial impact of rare adverse events across a group can soften extreme individual burdens, but any participant made worse off by the arrangement has reason to leave. A credible mechanism must therefore provide each agent with a trustworthy cap on their future obligation and should be deployed only if the aggregate harm across participants is bounded. We formalise this as the Certified Allocation Problem: from finite data and without distributional assumptions, find a redistribution rule, produce obligation caps for every participant, and verify that no participant is made materially worse off. We propose Conformal Risk Sharing, which solves this problem by pairing an interpretable sharing policy with split conformal calibration. The sharing intensity is tuned on training data, while held-out calibration data produces distribution-free per-agent guarantees (valid under exchangeability). Experiments on synthetic and real-world data, including precipitation and energy-cooperative data, confirm that the framework can substantially reduce extreme obligations for high-risk agents while controlling harm to others.

Reliable probability estimates by classifiers are essential in high-risk applications. In practice, however, predicted probabilities are often miscalibrated, and many existing post-hoc calibration methods typically lack guarantees that a specific notion of calibration is achieved after the correction procedure is applied. We introduce a *set-based* perspective on calibration through the notion of *cumulative mass calibration* and the corresponding error measures. We propose a new calibration procedure based on conformal prediction that forms cumulative probabilities with guaranteed marginal coverage. We introduce an __adaptive temperature scaling algorithm__, with the temperature tuned for each input to satisfy the conformal coverage constraint. As we show, this procedure can be efficiently implemented. Across image classification tasks, particularly in settings with many classes, our method improves newly introduced calibration error measures (__CMCE__ and α__-CMCE__) *and* standard metrics (such as ECE, cw-ECE, MCE) over the existing baselines.

Many distributional quantities in reinforcement learning are intrinsically joint across actions, including distributions of gaps and probabilities of superiority. However, the classical Markov decision process (MDP) formalism specifies only marginal laws and leaves the joint law of counterfactual one-step outcomes across multiple possible actions at a state unspecified. We study coupled-dynamics environments with a multi-action generative interface which can sample counterfactual one-step outcomes for multiple actions under shared exogenous randomness. We propose joint MDPs (JMDPs) as a formalism for such environments by augmenting an MDP with a multi-action sample transition model which specifies a coupling of one-step counterfactual outcomes, while preserving standard MDP interaction as marginal observations. We adopt and formalize a one-step coupling regime where dependence across actions is confined to immediate counterfactual outcomes at the queried state. In this regime, we derive Bellman operators for nth-order return moments, providing dynamic programming and incremental algorithms with convergence guarantees.

We study multi-objective reinforcement learning with stochastic, vector-valued returns and propose a policy improvement principle based on multivariate kth-order stochastic dominance in the lower-orthant sense. Rather than scalarizing objectives or comparing only expected values, we compare entire return distributions using kth-order integrated cumulative distribution functions (cdfs). This yields a max-violation objective that measures the worst-case dominance gap of a candidate policy relative to the incumbent policy. Under some regularity assumptions, we show that the Orthant Dominance Policy Optimization (ORDO) algorithm we introduce drives this objective to a point that certifies ϵ-almost non-dominatedness, meaning that no policy in the class can improve the incumbent by more than ϵ. We also propose smoothing approaches for the hinge structure inherent in integrated cdfs, which improve stability and enable stable stochastic gradients in optimization.

This study proposes the General Bayes framework for policy learning. We consider decision problems in which a decision-maker chooses an action from an action set to maximize its expected welfare. Typical examples include treatment choice and portfolio selection. In such problems, the statistical target is a decision rule, and the prediction of each outcome Y(a) is not necessarily of primary interest. We formulate this policy learning problem by loss-based Bayesian updating. Our main technical device is a squared-loss surrogate for welfare maximization. We show that maximizing empirical welfare over a policy class is equivalent to minimizing a scaled squared error in the outcome difference, up to a quadratic regularization controlled by a tuning parameter ζ>0. This rewriting yields a General Bayes posterior over decision rules that admits a Gaussian pseudo-likelihood interpretation. We clarify two Bayesian interpretations of the resulting generalized posterior, a working Gaussian view and a decision-theoretic loss-based view. As one implementation example, we introduce neural networks with tanh-squashed outputs. Finally, we provide theoretical guarantees in a PAC-Bayes style.

When language models are trained by reinforcement learning (RL) to write probabilistic programs, they can artificially inflate their marginal-likelihood reward by producing programs whose data distribution fails to normalise instead of fitting the data better. We call this failure likelihood hacking (LH). We formalise LH in a core probabilistic programming language (PPL) and give sufficient syntactic conditions for its prevention, proving that a safe language fragment L_safe satisfying these conditions cannot produce likelihood-hacking programs. Empirically, we show that GRPO-trained models generating PyMC code discover LH exploits within the first few training steps, driving violation rates well above the untrained-model baseline. We implement L_safe’s conditions as SafeStan, a LH-resistant modification of Stan, and show empirically that it suppresses LH under optimisation pressure. These results show that language-level safety constraints are both theoretically grounded and effective in practice for automated Bayesian model discovery.

We target Bayesian Active Learning (AL) and Optimization (BO) for censored data regimes. While the Tobit likelihood accurately models such clipped observations, its mixed continuous-discrete nature impedes the analytical evaluation of information-theoretic acquisition functions. To address this, we investigate Censored Optimization via Bayesian Active Learning Techniques (COBALT). We establish rigorous theoretical guarantees for this framework, proving posterior consistency and the asymptotic normality of the MAP estimator under greedy maximization. Central to our framework is the derivation of a closed-form entropy for the Censored Normal distribution, enabling an analytical BALD (Bayesian Active Learning by Disagreement) score compatible with any Gaussian posterior approximations. We further underpin this method by deriving a numerically stable Evidence Lower Bound (ELBO) for censored atoms, utilizing robust approximations of the log-normal cumulative density. Empirical evaluations using our open-source implementation demonstrate COBALT’s best accuracy–compute trade-off among censored-likelihood methods in learning GP posteriors and effectiveness on a variety of benchmarks.

Imbalanced regression, where certain continuous target ranges are severely underrepresented, poses a fundamental challenge for predictive modeling under uncertainty. Existing oversampling methods rely on local interpolation, which fails to capture complex conditional distributions in rare target regions. We propose TabOversample, a conditional diffusion framework that generates high-fidelity synthetic tabular samples for underrepresented targets via a relevance-weighted denoising objective. We establish four theoretical results: equivalence of relevance weighting to maximum likelihood under a reweighted measure, targeted probability mass amplification in rare regions, quality guarantees for generate-then-filter sampling through order statistics, and a formal connection to distributionally robust optimization over a chi-squared uncertainty set. Across nine benchmark imbalanced regression datasets—four numerical-dominant and five categorical-rich—and against thirteen oversampling and reweighting baselines evaluated over ten seeds, TabOversample attains the best relevance-weighted error (SERA) on eight of nine datasets and the best rare-region RMSE on all nine, substantially improving prediction accuracy in rare target regions while maintaining overall performance.

A bilevel optimization problem consists of two optimization problems nested as an upper- and a lower-level problem, in which the optimality of the lower-level problem defines a constraint for the upper-level problem. This paper considers Bayesian optimization (BO) for the case that both the upper- and lower-levels involve expensive black-box functions. Because of its nested structure, bilevel optimization has a complex problem definition, by which bilevel BO has not been widely studied compared with other standard extensions of BO such as multi-objective or constraint problems. We propose an information-theoretic approach that considers the information gain of both the upper- and lower-optimal solutions and values. This enables us to define a unified criterion that measures the benefit for both level problems, simultaneously. Further, we also show a practical lower bound based approach to evaluating the information gain. We empirically demonstrate the effectiveness of our proposed method through several benchmark datasets.

Identifiability in representation learning is commonly evaluated using standard metrics (e.g., *MCC, R^2, DCI*) on synthetic benchmarks with known ground-truth factors. These metrics are assumed to reflect recovery up to the equivalence class guaranteed by identifiability theory. We show that this assumption holds only under specific structural conditions: each metric implicitly encodes assumptions about both the data-generating process (DGP) and the encoder. When these assumptions are violated, metrics become misspecified and can produce systematic false positives and false negatives. Such failures occur both within classical identifiability regimes and in post-hoc settings where identifiability is most needed. We introduce a taxonomy separating DGP assumptions from encoder geometry, use it to characterise the validity domains of existing metrics, and release an evaluation suite for reproducible stress testing and comparison.

We study event prediction from news text when intermediate drivers are not expert-annotated. Directly prompting large language models (LLMs) for forecast probabilities can yield miscalibrated outputs and explanations that are hard to audit. We propose the Probabilistic Reliability and Interpretability System, PRISM, which separates semantic extraction from probabilistic decision making. PRISM uses LLMs only to extract interpretable factors, treats repeated extractions as noisy measurements of latent states, and performs Bayesian inference to produce posterior predictive probabilities that propagate extraction and measurement uncertainty. For empirical reliability under temporal dependence, we add low-capacity post-hoc calibration and time-adaptive conformal prediction and report calibration and coverage diagnostics. PRISM also reports factor-contrast association summaries with uncertainty and explicit interpretation boundaries. On a UCDP–GDELT conflict benchmark, PRISM achieves AUROC 0.821 and ECE 0.058 on the test set; at 90

We consider the problem of differentially private (DP) convex empirical risk minimization (ERM). While the standard DP-SGD algorithm is theoretically well-established, practical implementations often rely on shuffled gradient methods that traverse the training data sequentially rather than sampling with replacement in each iteration. Despite their widespread use, the theoretical privacy-accuracy trade-offs of private shuffled gradient methods (DP-ShuffleG) remain poorly understood, leading to a gap between theory and practice. In this work, we leverage privacy amplification by iteration (PABI) and a novel application of Stein’s lemma to provide the first empirical excess risk bound of DP-ShuffleG. Our result shows that data shuffling results in worse empirical excess risk for DP-ShuffleG compared to DP-SGD. To address this limitation, we propose Interleaved-ShuffleG, a hybrid approach that integrates public data samples in private optimization. By alternating optimization steps that use private and public samples, Interleaved-ShuffleG effectively reduces empirical excess risk. Our analysis introduces a new optimization framework with surrogate objectives, varying levels of noise injection, and a dissimilarity metric, which can be of independent interest. Our experiments on diverse datasets and tasks demonstrate the superiority of Interleaved-ShuffleG over several baselines.

Conformal prediction provides a distribution-free coverage guarantee for predictive inference, yet its validity degrades under distribution shift, a common challenge in real-world deployment. We introduce DiffConf, a framework that uses score-based diffusion models as expressive priors over the data-generating process to enable adaptive conformal inference under temporal and covariate distribution shifts. The main idea is that the score function learned by a diffusion model encodes rich geometric information about the data manifold, which can be repurposed to construct nonconformity scores that are sensitive to distributional changes. We derive a diffusion-guided conformity score that integrates the learned score field with a lightweight online recalibration mechanism, providing finite-sample marginal coverage guarantees even when the data distribution evolves over time. Theoretically, we establish that DiffConf achieves asymptotic conditional coverage under mild regularity conditions on the drift rate, and we prove a regret bound that scales gracefully with the complexity of the distribution shift. Experiments on synthetic benchmarks, real-world tabular regression tasks, and high-dimensional image datasets demonstrate that DiffConf produces prediction sets that are simultaneously valid and more efficient than existing adaptive conformal methods, reducing average set size by 6–39

We study online adaptation of a pre-trained base classifier to streaming unlabeled data under label shift, where the marginal label proportions in the stream differ from those in the offline training data. We consider the case where the base classifier’s model class may be misspecified, motivating a separate, calibrated auxiliary classifier used solely to estimate the target label proportions. While many works have studied this setting, it is less understood how calibration quality affects the performance of the adapted base classifier in an online setting. In this paper, we thoroughly analyze the estimation error of the adapted base classifier after the deployment of a novel algorithm that estimates the target label proportions in an online fashion and dynamically adapts the base classifier using the estimated proportions. We decompose the error into a term that vanishes over time and a term determined by calibration quality. Moreover, we characterize an explicit trade-off between calibration granularity and finite-sample calibration error, and propose a novel calibration strategy which effectively balances this trade-off.

Conformal prediction (CP) is a widely used frequentist framework to quantify uncertainty by constructing prediction sets with user-specified marginal coverage guarantees. In practice, CP is typically applied on top of probabilistic classifiers, which are able to express aleatoric but not epistemic uncertainty. In this paper, we consider the question of how to optimally employ CP on top of a more expressive formalism, namely credal sets, which can express both aleatoric and epistemic uncertainty. More specifically, we propose probabilistic Bernoulli prediction sets and derive a variant that achieves conditional coverage for valid credal sets while remaining minimal in expected size. We then address the more realistic scenario in which the validity of the credal sets is not guaranteed. Assuming access to calibration data with ground-truth distributions over labels, we apply conformal risk control to BPS and derive a PAC-style guarantee: with high probability over the data, the achieved conditional coverage is at least the desired level. We validate our theoretical findings empirically over various datasets.

We study multi-armed bandits under network interference, where each unit’s reward depends on its own treatment and those of its neighbors in a given graph. This induces an exponentially large action space, making standard approaches computationally impractical. We propose a novel algorithm that uses the local graph structure to minimize regret. We derive a graph-dependent upper bound on cumulative regret that improves over prior work. Additionally, we provide the first lower bounds for bandits with arbitrary network interference, where each bound involves a distinct structural property of the graph. These bounds show that for both dense and sparse graphs, our algorithm is nearly optimal, with matching upper and lower bounds up to logarithmic factors. When the interference graph is unknown, a variant of our algorithm is Pareto optimal: no algorithm can uniformly outperform it across all instances. We complement our theoretical results with numerical experiments, showing that our approach outperforms the baseline methods.

Federated Learning (FL) enables heterogeneous clients to collaboratively train a shared model without centralizing their raw data, offering an inherent level of privacy. However, gradients and model updates can still leak sensitive information, while malicious servers may mount adversarial attacks such as Byzantine manipulation. These vulnerabilities highlight the need to address differential privacy (DP) and Byzantine robustness within a unified framework. Existing approaches, however, often rely on unrealistic assumptions such as bounded gradients, require auxiliary server-side datasets, or fail to provide convergence guarantees. We address these limitations by proposing Byz-Clip21-SGD2M, a new algorithm that integrates robust aggregation with double momentum and carefully designed clipping. We prove high-probability convergence guarantees under standard L-smoothness and σ-sub-Gaussian gradient noise assumptions, thereby relaxing conditions that dominate prior work. Our analysis recovers state-of-the-art convergence rates in the absence of adversaries and improves utility guarantees under Byzantine and DP settings. Empirical evaluations on CNN and MLP models trained on MNIST further validate the effectiveness of our approach.

We study problems where two agents seek to minimize an objective that is known only to one of the agents. This setting arises in human-machine and AI-AI interactions where one agent’s objective is private knowledge inaccessible to the other, referred to as the "helper". We propose a game-theoretic learning algorithm that provably converges to optimal policies through repeated interaction without solving an inverse problem to recover the unknown objective. Importantly, the helper agent has no access to the cost function, its values, or its gradients, and operates under action-only feedback. Despite the information limitation, we establish convergence guarantees under Polyak-Lojasiewicz and Lipschitz-gradient assumptions. We validate the approach through two sets of experiments: human-machine interaction and a cart-pole environment with a reinforcement learning agent. In each of these experiments, the helper uses our proposed algorithm. Across scalar and multidimensional action spaces, we demonstrate consistent convergence under action-only feedback.