paperlist.ai

AI conference papers

AI conference paper library

Search papers from 30+ top conferences, including NeurIPS, ICLR, ICML, CVPR, ACL and AAAI. Filter by keyword, then open any paper for its abstract, authors and links.

arXiv

On the Condition Number Dependency in Bilevel Optimization

Lesi Chen, Kaiyi Ji, Jingzhao Zhang

Bilevel optimization minimizes an objective function, defined by an upper-level problem whose feasible region is the solution of a lower-level problem. We study the oracle complexity of finding an $ε$-stationary point with first-order methods when the upper-level problem is nonconvex, and the lower-level problem is strongly convex. Recent works achieve a $\tilde{\mathcal{O}}(\bar κ_y^{7/2} ε^{-2})$ upper bound that is near-optimal in $ε$. In this work, we establish a new $Ω(κ_y^{5/2} ε^{-2})$ lower bound, where $κ_y \le \bar κ_y$ is the lower-level condition number. Our lower bound establishes the first provable gap {in terms of condition number dependency} between bilevel problems and minimax problems in this setup, and \textit{is tight up to logarithmic factors when the lower-level function is quadratic.} Our lower bounds can be extended to various settings. (1) For second-order and arbitrarily smooth problems, we show lower bounds of $Ω(κ_y^{9/4} ε^{-7/4})$ and $Ω(κ_y^{13/6} ε^{-5/3})$, respectively. (2) For convex--strongly-convex problems, we improve the previously best lower bound (Ji and Liang, JMLR 2022) from $Ω(κ_y /\sqrtε)$ to $Ω(κ_y^{3/2} / \sqrtε)$. (3) For stochastic nonconvex--strongly-convex problems, we also show the lower bounds of $Ω(κ_y^4 ε^{-4})$ and $Ω(κ_y^{9/2} ε^{-4})$ for stochastic Hessian-vector-product and stochastic first-order methods, respectively.

arXiv

Hyperellipsoid Density Sampling: Exploitative Sequences to Accelerate High-Dimensional Numerical Optimization

Julian G. Soltes

The curse of dimensionality remains a persistent challenge in modern optimization problems. Expanding the search space into higher dimensions exponentiates the sparsity of uniform sample distributions, rendering traditional quasi-Monte Carlo (QMC) sequences increasingly inefficient. This paper introduces a non-uniform sampling strategy to accelerate high-dimensional optimization. This method, Hyperellipsoid Density Sampling (HDS), generates samples as hyperellipsoids overlapping throughout the parameter space. Utilizing a series of unsupervised learning techniques, a non-uniform sequence is generated to exploit the interior regions of the hypervolume. If prior information about optima is known, the distribution can be biased towards the known regions, making HDS versatile for many numerical applications. HDS was evaluated against Sobol, a highly uniform QMC sampling method, using differential evolution (DE) on the challenging and widely benchmarked set of 29 CEC2017 test functions. The results show statistically significant improvements in final solution geometric mean error (p<0.05), with average performance gains ranging from 37% in 10D to 11% in 100D. This paper demonstrates the efficacy of HDS as an exploitative alternative to uniform QMC sampling.

arXiv

Gradient Boosted Mixed Models: Flexible Estimation of Mean and Variance Components for Clustered Data

Mitchell L. Prevett, Francis K. C. Hui, Zhi Yang Tho, A. H. Welsh, Anton H. Westveld

We introduce a novel way to combine gradient boosting with mixed effects models, whereby the mean and variance components are learned jointly as functions of covariates via likelihood-based gradients. Gradient Boosted Mixed Models (GBMixed) estimates a nonparametric fixed effects function characterizing the overall mean of the response, while also allowing the random effects covariance matrix along with the residual variance to depend on covariates in a flexible manner. We demonstrate how GBMixed facilitates covariate-dependent random effect predictions, and subsequently point predictions and prediction intervals for individual treatment effects, that can adapt between population-level and cluster-level information. Experiments and applications to two real-world datasets show that GBMixed can accurately recover complex nonlinear fixed effect functions and covariate-dependent covariances in a linear mixed model, while also improving point and probabilistic predictive performance compared with several existing approaches such as parametric linear mixed models, Natural Gradient Boosting, and Gaussian Process Boosting. In simulations where the variance components are designed to vary as a function of covariates, GBMixed reduces the mean squared error in recovering the random effects variance function by a factor of eight relative to linear mixed models and Gaussian Process Boosting.

arXiv

HyWA: Architecture-Preserving Personalized Voice Activity Detection for Full-Duplex Voice Assistants

Hamed Jafarzadeh Asl, Amin Edraki, Mahsa Ghazvini Nejad, Masoud Asgharian, Mohammadreza Sadeghi, YUANHAO YU, Vahid Partovi Nia

Voice activity detection (VAD) serves as an early gate in voice-assistant pipelines for smart devices. Because conventional VADs respond to speech from any speaker, nearby conversations and residual assistant playback lead to unwanted triggers, degrade the user experience, and waste computational resources. Personalized voice activity detection (PVAD) addresses this limitation by detecting speech only from an enrolled target speaker. Existing PVAD methods typically incorporate speaker information into model inputs or hidden representations. These approaches require VAD architectural changes that increase engineering and requalification costs in deployment. We present HyWA, a hypernetwork-based weight-adaptation method that converts an established VAD into a PVAD, while preserving its acoustic interface and inference topology. HyWA generates speaker-conditioned weights once at enrollment and requires no per-user optimization. We apply HyWA to pretrained Alibaba's FSMN and NVIDIA's MarbleNet VAD models. Evaluations on synthetic and real-user test data show consistent improvements in target-speaker discrimination and false-positive suppression. In an integrated full-duplex barge-in pipeline, replacing generic VAD with HyWA-based PVAD reduces observed false-interruption detections from 88.9% to only 9.9%.

arXiv

On The Statistical Limits of Self-Improving Agents

Charles L. Wang, Keir Dorchen, Peter Jin

We develop a learning-theoretic framework for analyzing self-improving agents by decomposing self-modification into five axes. Within this framework, we prove a sharp boundary: under standard i.i.d. assumptions, distribution-free PAC learnability is preserved if and only if the policy-reachable family remains uniformly capacity-bounded. If reachable capacity can grow without bound, utility-rational self-changes can make learnable tasks unlearnable. We further introduce a simple Two-Gate guardrail -- a validation-improvement requirement plus a capacity cap -- that preserves this boundary and yields standard VC-rate guarantees. The broader implication is that self-modification must be constrained not only by objectives, but also by structural conditions that preserve the statistical prerequisites for learning. As AI systems become increasingly intelligent and autonomous, this framework provides a foundation for the statistical theory of self-improvement.

arXiv

Terminal-Bench-LILT: Multilingual Agentic Coding Benchmark Grounded in Language, Region, and Culture

Yunsu Kim, Kaden Uhlig, Ashwin Purohit, Milind Agarwal, Patrick Simianer, Anil Arslan, Kiarash Mokhtari, Thomas Zenkel, Johannes Mosig, Gabriel Bretschner, Shamik Bose, Joern Wuebker, John DeNero

Most evaluations for coding agents are conducted exclusively in English, which does not reflect real-world multilingual deployment. We present Terminal-Bench-LILT, a suite of 300 authentic coding tasks in ten languages: Arabic, Czech, German, Spanish, Hindi, Japanese, Korean, Serbian, Turkish, and Chinese. Each task targets issues specific to non-English software development that have no direct English equivalent, e.g., internationalization, encoding, text normalization, and cultural conventions. All tasks are authored by native-speaker programmers and validated through a multi-stage quality control pipeline. Evaluation of six frontier models reveals that even the strongest model reaches only 63.1\% pass rate, with many tasks unsolved by any model. Performance varies substantially by language and does not track general coding benchmark rankings, highlighting that multilingual coding competence is a distinct and underexplored capability axis. Sample tasks are available at https://github.com/lilt/terminal-bench-lilt

arXiv

PromptKWS: A Novel Prompt-Guided Open-Vocabulary Keyword Spotting Framework

Gaopeng Xu, Chengfei Li, Xianliang Wang, Lin Zhu, Juan Wei, Wenpeng Li, Jianwei Niu, Jie Gao

In this paper, we present PromptKWS, a novel Prompt-guided keyword spotting (KWS) framework to improve the accuracy of open vocabulary KWS systems. In specific terms, we introduce the Prompt Phrases Prediction Network (PPN), an encoder-decoder architecture designed to effectively extract keyword prompts embeddings. we employ the PPN encoder to encode the keyword prompts and infuse the prompt embedding into the Prompt-guided KWS encoder by utilizing a Prompt-acoustic Multi-head Cross-attention (MHCA). Experiments show that PromptKWS improves the wakeup rate by over 10% compared to baseline system. Notably, another strength of PromptKWS is its ability to effectively leverage keyword prompts for adapting to complex real-world environments involving noise and pronunciation variations. In comparison to purely acoustic models, which often struggle in such situations, PromptKWS demonstrates remarkable performance, with an average accuracy improvement of over 15% in test sets.

arXiv

Diffusion-Based Impedance Learning for Contact-Rich Manipulation Tasks

Noah Geiger, Tamim Asfour, Neville Hogan, Johannes Lachner

Learning-based methods excel at robot motion generation but remain limited in contact-rich physical interaction. Impedance control provides stable and safe contact behavior but requires task-specific tuning of stiffness and damping parameters. We present Diffusion-Based Impedance Learning, a framework that bridges these paradigms by combining generative modeling with energy-consistent impedance control. A Transformer-based Diffusion Model, conditioned via cross-attention on measured external wrenches, reconstructs simulated Zero-Force Trajectories (sZFTs) that represent contact-consistent equilibrium behavior. A SLERP-based quaternion noise scheduler preserves geometric consistency for rotations on the unit sphere. The reconstructed sZFT is used by an energy-based estimator to adapt impedance online through directional stiffness and damping modulation. Trained on parkour and robot-assisted therapy demonstrations collected via Apple Vision Pro teleoperation, the model achieves sub-millimeter positional and sub-degree rotational accuracy using only tens of thousands of samples. Deployed in real-time torque control on a KUKA LBR iiwa, the approach enables smooth obstacle traversal and generalizes to unseen tasks, achieving 100% success in multi-geometry peg-in-hole insertion. The code for all experiments is publicly available on GitHub and videos of the experiments are available on the project website.

arXiv

GLAM: Efficient Continual Learning at Scale via Grouped LoRA Adapter Merging

Irene Testa, Luigi Quarantiello, Eric Nuertey Coleman, Samrat Mukherjee, Julio Hurtado, Vincenzo Lomonaco

The ability to learn continuously over time remains a major challenge for modern machine learning systems, even in the era of Foundation Models. While the rich representations learned by large pre-trained models can partially mitigate catastrophic forgetting, they still struggle to adapt efficiently to evolving data distributions. A key challenge remains, how to continually add new knowledge to a large pretrained model in a way that is scalable and computationally efficient over long task sequences. In this work, we introduce GLAM, a simple and effective framework for class-incremental continual learning based on LoRA adapters merging. For each task, GLAM trains a lightweight low-rank adapter with an importance scalar, incurring minimal computational overhead. Adapters are then pruned, rescaled, and sequentially grouped to enable structured knowledge reuse and limit parameter growth. At inference, all groups are combined into a single module, ensuring constant computational cost regardless of the number of tasks. We evaluate GLAM on vision benchmarks with sequences of up to 50 tasks, significantly extending beyond standard protocols. GLAM achieves the highest accuracy across the evaluated benchmarks. Compared with the baseline attaining the highest average accuracy across benchmarks, it uses 16--19\% of the trainable parameters and reduces training time by approximately 63--74\%, demonstrating efficient and scalable continual learning. Our source code is publicly available at https://github.com/atlas-luiss/GLAM.

arXiv

Behavioral Inference at Scale: The Fundamental Asymmetry Between Motivations and Belief Systems

Jason Starace, Terence Soule

How much information about an agent's underlying values can be recovered from its observable behavior? This question matters for any approach that infers agent properties from action sequences, yet remains empirically open at scale. We address it through controlled experiments: LLM-based agents (Llama 3.1-8B) assigned one of 36 behavioral profiles (9 belief systems x 4 motivations) generate over 1.5 million behavioral sequences in grid-world environments, providing ground truth unavailable in human behavioral studies. After filtering, classifiers train and evaluate on a shared canonical dataset of 10,338 episodes and 1,200,834 sequences. A fundamental asymmetry emerges in both magnitude and structure. Motivations achieve 98-100% accuracy and recover 97% of available mutual information across all architectures. Belief systems plateau at 24% for LSTMs regardless of capacity, and even transformers reach only 34.0%, recovering 16.3% of available information, a 6.1x asymmetry in extraction efficiency. Per-alignment accuracy ranges from 23.2% (Lawful Neutral) to 59.4% (Chaotic Evil). Confusion analysis maps the failure structure: a neutral zone of behavioral ambiguity centers on True Neutral, absorbing misclassified samples from adjacent Neutral and Good alignments whose prosocial or balance-keeping behavior lacks distinctive signal. Combined inference yields 12.2x improvement over random baseline for full 36-class profile classification, with the bottleneck located entirely in belief system inference. Signal enhancement and explanatory queries yield only marginal LSTM gains (+3.8%), confirming the recurrent ceiling is architectural rather than data-limited. Whether the transformer's 34.0% ceiling reflects a similar architectural-class limit or a more fundamental bound remains open. These results characterize what behavioral observation can and cannot reveal about LLM agent values.

arXiv

TS-Mob: Social and Geographical-Aware Time Series Foundation-Model Framework for Human Mobility Prediction

Massimiliano Luca, Ciro Beneduce, Bruno Lepri

Short-term forecasting of aggregated human mobility flows supports urban planning, intelligent transportation systems, and emergency response, yet existing models often require substantial mobility history and learn spatial structure implicitly through grids or graphs. Time series foundation models provide strong temporal priors but typically lack explicit geographic and social conditioning for origin-destination interactions. We introduce TS-Mob, a framework that conditions a fine-tuned time series foundation model (TimesFM) forecaster on a gravity-inspired destination-attractiveness index that encodes geographic and social signals computed from open data (living population, centroid distances, and Overture POI counts), together with weather covariates. Evaluated on commonly used benchmarks like Bike New York City, Taxi Beijing, and a nation-scale Spain origin-destination matrix estimated through mobile phone data, TS-Mob outperforms classical, deep spatio-temporal, and a set of foundation model-based baselines across RMSE, MAE, and CPC, with gains up to 78.71% lower MAE and 137.93% higher CPC over the best classical baseline, and up to about 4.27% lower RMSE over the strongest foundation baseline. Stratified analyses further show robustness across different temporal regimes, like weekdays/weekends.

arXiv

Generalized Linear Markov Decision Process

Sinian Zhang, Kaicheng Zhang, Ziping Xu, Zongqi Xia, Jue Hou, Tianxi Cai, Doudou Zhou

Offline reinforcement learning for longitudinal studies often faces two linked challenges: rewards may be binary or bounded, and reward observations may be available only for a subset of trajectories or time points even when the corresponding state-action-next-state histories are available. Linear Markov decision process methods are tractable because Bellman backups remain linear, but they require linear rewards and do not indicate how transition-only observations should be used. We introduce GRASP-MDP, Generalized Reward And Semi-supervised Pessimism for Markov Decision Processes, a reward-transition separated framework addressing both issues through one Bellman decomposition. It preserves linear transition dynamics while modeling reward means through generalized linear models. Although this breaks the usual linear Bellman form, the backup remains explicit as a nonlinear reward component plus a linear continuation component, yielding a nonlinear-plus-linear Bellman-complete class. In the resulting recursion, observed rewards estimate the generalized reward component, while all available transitions estimate the continuation component. The resulting pessimistic value iteration controls the two estimation errors separately without reward imputation. Finite-sample guarantees show that transition-only observations reduce transition-estimation error while reward uncertainty remains governed by observed rewards. Simulations and a multiple sclerosis electronic health record application illustrate the empirical benefit of retaining transition-only observations.

arXiv

Demystifying Adversarial Robustness in Diffusion Models: Compression, Randomness, and Geometry

Liu Yuezhang, Xue-Xin Wei

Recent studies suggest that diffusion models significantly improve the empirical adversarial robustness of deep neural network models. While intuitive explanations have been proposed, the mechanisms underlying diffusion-based robustness remain largely unclear. This work aims to demystify how diffusion models improve adversarial robustness. We observe that diffusion models surprisingly increase the $\ell_p$ distance to clean samples, thus rejecting the hypothesis that purification denoises perturbed images closer to the clean ones. Next, we provide a unifying account of the robustness improvement in diffusion-based purification by decomposing it into two sources: (i) gradient masking induced by randomness; (ii) compression of the image space. First, we find that the purified images are heavily influenced by the internal randomness of diffusion models. This randomness leads to gradient masking that cannot be removed by the previously proposed remedy, i.e., expectation-over-transformation (EOT). The improvement in robustness due to randomness is determined by the cosine similarity of the optimal vs. empirical attack directions, as predicted by a hyperspherical cap model of the adversarial regions. Second, we find that, when fixing the randomness, diffusion models substantially compress the image space. Importantly, we discover a lawful relationship between the model's ability to compress the image space and the genuine adversarial robustness gain. Further theoretical analyses show that convergent score fields encoded in diffusion models explain these compression effects. Our findings reveal new insights into the mechanisms underlying diffusion-based purification, and offer guidance for developing more effective and principled adversarial purification systems.

arXiv

High-Dimensional Calibration from Swap Regret

Maxwell Fishelson, Noah Golowich, Mehryar Mohri, Jon Schneider

We study online calibration of multi-dimensional forecasts over an arbitrary convex set $P \subset \mathbb{R}^d$ relative to an arbitrary norm $|\cdot|$. We connect this to external regret minimization for online linear optimization (OLO): if one can guarantee $O(\sqrt{ρT})$ worst-case regret after $T$ rounds when actions are drawn from $P$ and losses from the dual $|\cdot|_*$ unit norm ball, then one can obtain $ε$-calibrated forecasts after $T = \exp(\tilde O(ρ/ε^2))$ rounds. When $P$ is the $d$-dimensional simplex and $|\cdot|$ is the $\ell_1$-norm, the $O(\sqrt{T\log d})$ experts regret bound yields $ε$-calibrated forecasts after $T = \exp(\tilde O(\log d/ε^2)) = d^{\tilde O(1/ε^2)}$ rounds, recovering a recent result of Peng (2025). Interestingly, our algorithm obtains this guarantee without requiring access to any online linear optimization subroutine or knowledge of the optimal rate $ρ$ -- in fact, our algorithm is identical for every setting of $P$ and $|\cdot|$. Instead, we show that the optimal regularizer for the above OLO problem can be used to upper bound the above calibration error by a swap regret, which we then minimize by running the recent TreeSwap algorithm (Dagan et al., 2024; Peng and Rubinstein, 2024) with Follow-The-Leader as a subroutine. The resulting algorithm is highly efficient and plays a distribution over simple averages of past observations in each round. Finally, we prove that any online calibration algorithm that guarantees $εT$ $\ell_1$-calibration error over the $d$-dimensional simplex requires $T \geq \exp(\mathrm{poly}(1/ε))$ (assuming $d \geq \mathrm{poly}(1/ε)$). This strengthens the corresponding $d^{Ω(\log(1/ε))}$ lower bound of Peng (2025), and shows that an exponential dependence on $1/ε$ is necessary.

arXiv

Local Fréchet functional regression in manifolds from time-correlated bivariate curve data

M. D. Ruiz-Medina, A. Torres-Signes

Under mild conditions, a least-squares local linear Fréchet curve predictor is derived for a response and a regressor evaluated in a separable Hilbert space. The conditions that allow the implementation of the local linear Fréchet functional predictor in the ambient L2-space of vector functions, with values in the time-varying tangent space of a compact Riemannian manifold, are established. An intrinsic local linear Fréchet curve predictor on such a manifold is then proposed, based on a weighted Fréchet mean approach. Its asymptotic optimality is proved. Simulations and a real-data application are considered to analyze the finite-sample performance of the empirical versions of both predictors, compared with a geodesic Nadaraya-Watson-type curve predictor. In the real-data application, the functional prediction of the time-varying spherical coordinates of the Earth's magnetic field is addressed usingobservations through time of the geocentric latitude and longitude of the NASA MAGSAT spacecraft.

arXiv

Bayesian Federated Cause-of-Death Classification and Quantification Under Distribution Shift

Yu Zhu, Jason Teng, Zehang Richard Li

In regions lacking medically certified causes of death, verbal autopsy (VA) is a widely used tool to ascertain the cause of death through interviews with caregivers. Data collected by VAs are often analyzed using probabilistic algorithms. The performance of these algorithms often degrades due to distribution shift across populations. Most existing VA algorithms rely on centralized training, requiring full access to training data for joint modeling. This can be infeasible due to privacy and logistical constraints. In this paper, we propose a novel Bayesian Federated Learning (BFL) framework that avoids data sharing across multiple training sources. Our method supports individual-level cause-of-death classification and population-level quantification of cause-specific mortality fractions in a target domain with limited or no local labeled data. The proposed framework is modular, computationally efficient, and compatible with a wide range of existing VA algorithms as base models, facilitating flexible deployment in real-world mortality surveillance systems. We validate the performance of BFL through extensive experiments on two real-world VA datasets under varying levels of distribution shift scenarios. Our results show that BFL significantly outperforms single-domain base models and performs comparably to or better than joint modeling.

arXiv

KKL Observer Synthesis for Nonlinear Systems via Physics-Informed Learning

M. Umar B. Niazi, John Cao, Matthieu Barreau, Karl Henrik Johansson

This paper proposes a novel learning approach for designing Kazantzis-Kravaris or nonlinear Luenberger (KKL) observers for autonomous nonlinear systems. The design of a KKL observer involves finding an injective map that transforms the system state into a higher-dimensional observer state, whose dynamics is linear and stable. The observer's state is then mapped back to the original system coordinates via the inverse map to obtain the state estimate. However, finding this transformation and its inverse is quite challenging. We propose learning the forward mapping using a physics-informed neural network, and then learning its inverse mapping with a conventional feedforward neural network. Theoretical guarantees for the robustness of state estimation against approximation error and system uncertainties are provided, including non-asymptotic learning guarantees that link approximation quality to finite sample sizes. The effectiveness of the proposed approach is demonstrated through numerical simulations on benchmark examples, showing better generalization capability outside the training domain compared to state-of-the-art methods.

arXiv

A variational Bayes approach to inference for low-dimensional parameters in high-dimensional linear regression

Ismaël Castillo, Alice L'Huillier, Kolyan Ray, Luke Travis

We propose a scalable variational Bayes method for statistical inference for a single or pre-specified low-dimensional subset of the coordinates of a high-dimensional parameter in sparse linear regression. Our approach relies on assigning a mean-field approximation to the nuisance coordinates and carefully modelling the conditional distribution of the target given the nuisance. This requires only a preprocessing step and preserves the computational advantages of mean-field variational Bayes, while ensuring accurate and reliable inference for the target parameter, including for uncertainty quantification. We investigate the numerical performance of our algorithm, showing that it performs competitively with existing methods. We further establish accompanying theoretical guarantees for estimation and uncertainty quantification in the form of a Bernstein--von Mises theorem for a possibly growing subset.

arXiv

Convergence of Sign-based Random Reshuffling Algorithms for Nonconvex Optimization

Zhen Qin, Zhishuai Liu, Pan Xu

signSGD is attractive in nonconvex optimization because it communicates sign-valued rather than full-precision gradients. Several standard analyses assume independent stochastic-gradient samples, whereas a common finite-sum implementation reshuffles the data and processes them sequentially. We study this variant, signSGD with random reshuffling (SignRR), and show that reshuffling does not in general repair the bias created by discarding gradient magnitudes. In particular, on a one-dimensional two-component strongly convex quadratic, the expected gradient norm at every SignRR inner iterate equals $1/2$. We complement this impossibility result with an alignment-explicit finite-time bound $O(\log(nT)/\sqrt{nT}+\varepsilon_{\mathrm{align}})$, where $\varepsilon_{\mathrm{align}}$ measures the averaged loss of descent caused by component-sign misalignment. A horizon-tuned constant stepsize improves the vanishing term to $O(1/\sqrt{nT})$, and a remaining-set alignment condition yields a residual-free $O(1/\sqrt{nT})$ guarantee. The alignment term is upper bounded by twice the averaged mean absolute gradient error and, in turn, by twice an averaged coordinatewise conditional root-mean-square error. As a variance-reduced alternative, we analyze SignRVR, which signs an SVRG estimator anchored at the beginning of every epoch. A pathwise argument gives a residual-free guarantee with an $O(\sqrt{d/T})$ averaged $\ell_1$-stationarity bound.

arXiv

Weighted Sequential Bayesian Inference for Non-Stationary Linear Contextual Bandits

Nicklas Werge, Yi-Shan Wu, Abdullah Akgül, Melih Kandemir

In non-stationary linear contextual bandits, existing efficient algorithms typically rely on the Weighted Regularized Least-Squares (WRLS) estimator. Because WRLS only provides point estimates, previous methods typically construct surrogate distributions when aiming to perform Bayesian-like randomized exploration. To more properly establish the Bayesian principles, we introduce Weighted Sequential Bayesian (WSB) inference, which forms a sequence of posteriors over a sequence of non-stationary reward parameters. This Bayesian take allows us to isolate the influence of initial beliefs into a dynamic prior penalty evaluated through the posterior covariance, which typically decreases over time. Building on this framework, we instantiate three WSB-based algorithms for exploration: WSB-LinUCB, WSB-RandLinUCB, and WSB-LinTS. By extending a refined drift analysis to randomized exploration without requiring local norms, we establish frequentist regret guarantees that match state-of-the-art WRLS-based baselines. Empirically, WSB's dynamic prior penalty reduces over-conservatism, allowing our algorithms to consistently match or exceed their WRLS-based counterparts. Lastly, we also provide a simplified proof for the time-uniform concentration of vector-valued martingales, a critical subroutine used throughout the literature, that might be of independent interest.

Keep browsing

More papers from top AI conferences

30+ top conferences, by conference, year, and topic.