AI conference papers

AI conference paper library

Search papers from 30+ top conferences, including NeurIPS, ICLR, ICML, CVPR, ACL and AAAI. Filter by keyword, then open any paper for its abstract, authors and links.

arXiv

Dissecting Federated-Graph Aggregation under Domain Shift: Importance-Aware Aggregation via Empirical Analysis

Zhanting Zhou, Kahou Tam, Zeyu Ma, Ziqiang Zheng

Federated graph learning (FGL) trains a shared graph model across clients whose local graphs differ in node features, labels, and connectivity while keeping raw graph data decentralized. Although graph-domain shifts across clients can severely degrade the global model, existing FGL approaches for graph-domain shift mainly adapt local representations, propagation, or graph-derived collaboration, while the server typically applies the same aggregation rule to every parameter coordinate. In this work, we analyze how graph-domain shifts affect server-side aggregation. We find that, as clients optimize under distinct graph-domain conditions, they gradually concentrate their strongest updates on different parameter coordinates, making important update coordinates less shared across clients. Consequently, standard averaging can weaken these domain-specific signals even when client updates are not directly opposed. These findings reveal an aggregationinduced signal dilution effect: parameter coordinates strongly expressed by only a subset of domains are attenuated by nearzero contributions from the remaining domains. Motivated by these findings, we propose FedIA, a lightweight serverside aggregation method that calibrates parameter selection and client weighting. FedIA identifies important coordinates =within each layer and adjusts client aggregation weights according to their contributions on the selected subspace, without modifying local graph training or the standard clientupdate payload. Experiments on Twitch Gamers and WikiNet using two graph-learning backbones and nine FL/FGL baselines show improvements of up to 11.58 percentage points, with particularly strong gains under domain skew.

arXiv

Accelerating SGDM via Learning Rate and Batch Size Schedules: A Lyapunov-Based Analysis

Yuichi Kondo, Hideaki Iiduka

We analyze the convergence behavior of stochastic gradient descent with momentum (SGDM) under dynamic learning-rate and batch-size schedules by introducing a novel and simpler Lyapunov function. We extend the existing theoretical framework to cover three practical scheduling strategies commonly used in deep learning: a constant batch size with a decaying learning rate, an increasing batch size with a decaying learning rate, and an increasing batch size with an increasing learning rate. Our results reveal a clear hierarchy in convergence: a constant batch size does not guarantee convergence of the expected gradient norm under our Lyapunov-based analysis, whereas an increasing batch size does, and simultaneously increasing both the batch size and learning rate achieves a provably faster decay. Empirical results validate our theory, showing that dynamically scheduled SGDM significantly outperforms its fixed-hyperparameter counterpart in convergence speed. We also evaluated a warmup schedule in experiments, which empirically outperformed all other strategies in convergence behavior.

arXiv

CLIP-Guided Backdoor Defense through Entropy-Based Poisoned Dataset Separation

Binyan Xu, Fan Yang, Xilin Dai, Di Tang, Kehuan Zhang

Deep Neural Networks (DNNs) are susceptible to backdoor attacks, where adversaries poison training data to implant backdoor into the victim model. Current backdoor defenses on poisoned data often suffer from high computational costs or low effectiveness against advanced attacks like clean-label and clean-image backdoors. To address them, we introduce CLIP-Guided backdoor Defense (CGD), an efficient and effective method that mitigates various backdoor attacks. CGD utilizes a publicly accessible CLIP model to identify inputs that are likely to be clean or poisoned. It then retrains the model with these inputs, using CLIP's logits as a guidance to effectively neutralize the backdoor. Experiments on 4 datasets and 11 attack types demonstrate that CGD reduces attack success rates (ASRs) to below 1% while maintaining clean accuracy (CA) with a maximum drop of only 0.3%, outperforming existing defenses. Additionally, we show that clean-data-based defenses can be adapted to poisoned data using CGD. Also, CGD exhibits strong robustness, maintaining low ASRs even when employing a weaker CLIP model or when CLIP itself is compromised by a backdoor. These findings underscore CGD's exceptional efficiency, effectiveness, and applicability for real-world backdoor defense scenarios. Code: https://github.com/binyxu/CGD.

arXiv

What Must a Fairness Audit Report When Demographic Data Is Incomplete?

Yash Vardhan Tomar

Fairness audits are a key component of responsible machine-learning deployment. Yet what such an audit must disclose, when the protected labels it depends on are incomplete, remains unsettled. In this work, we focused on the rates a fairness audit publishes and on what an oversight reader needs beside them. We paired every published rate with a matched baseline drawn from the same audit, one hiding protected labels and one varying only the run seed. Across ACS/Folktables tasks, missingness settings that kept some protected labels moved the selected mitigation less than an ordinary rerun did. At zero protected-label access, candidates collapsed to empirical risk minimization, so the apparent exception there reflected the candidate set's composition. Equalized-odds threshold optimization most often regressed an intersectional subgroup, but that rate fell back to its baseline once we kept only the configurations an auditor would accept. Any accuracy it lost fell on the population as heavily as on the worst-off cell, so the mechanism is levelling down. The one effect that survived was a change in which cell is worst-off. Overall, our results highlight that a published audit rate should be reported with the baseline needed to interpret it, the candidate set it came from, and its intersectional effects, before it is treated as evidence about a deployed model.

arXiv

pychop: Emulating Low-Precision Arithmetic in Numerical Methods and Neural Networks

Erin Carson, Xinye Chen

Motivated by the growing demand for reduced-precision arithmetic in computational science, we exploit lower-precision emulation in Python---widely regarded as the dominant programming language for numerical analysis and machine learning. Low-precision paradigms have revolutionized deep learning by enabling more efficient computation and reduced memory footprint while maintaining model fidelity. To better enable numerical experimentation with and exploration of reduced-precision computation, we developed the \texttt{pychop}, which supports customizable floating-point formats and a comprehensive set of rounding modes in Python, allowing users to benefit from fast, reduced-precision emulation in numerous applications. \texttt{pychop} also introduces interfaces for {array and tensor backends}, enabling efficient reduced-precision emulation on GPUs for neural network training and inference with unparalleled flexibility. In this paper, we offer a comprehensive exposition of the design and applications of \texttt{pychop}, establishing it as a foundational tool for advancing mixed-precision algorithms. Furthermore, we present empirical results on reduced-precision emulation for image classification and object detection using published datasets, illustrating the sensitivity of the use of low precision and offering valuable insights into its quantization-aware training and post-quantization impacts. \texttt{pychop} enables in-depth investigations into the effects of numerical precision, facilitates the development of novel hardware accelerators, and integrates seamlessly into existing deep learning workflows.

arXiv

Gradient-Free Continual Learning

Grzegorz Rypeść

Neural networks are notorious for forgetting old skills when taught new ones - a problem known as catastrophic forgetting. Standard continual learning techniques try to fix this by saving old data or relying on complex gradient updates, but these methods fail when past data cannot be stored due to memory or privacy constraints. To solve this, we propose EvoCL, a gradient-free approach that uses evolutionary algorithms to update the network without needing old data or gradients. EvoCL uses a lightweight adapter module to translate saved representations from past tasks into the model's current space, allowing it to learn new tasks while keeping past knowledge intact. Across multiple benchmarks, EvoCL matches or exceeds standard performance under strict memory constraints, offering a simple and flexible new direction for continual learning. The code to reproduce these results is available at https://github.com/grypesc/EvoCL.

arXiv

Re-examining Granger Causality with Causal Bayesian Networks and Reichenbachs Principles

S. A. Adedayo

Granger causality (GC) is widely used to infer directed relationships in time-series data. However, its predictive criterion does not by itself distinguish direct causal effects from dependencies induced by common causes, indirect paths, collider conditioning, or model misspecification. We revisit this limitation by interpreting bivariate and multivariate GC through causal Bayesian networks and Reichenbachs common cause principles. Under explicit graphical assumptions, bivariate GC provides a marginal dependence check, while multivariate GC tests whether the same association persists after conditioning on relevant histories. This view motivates causalised Granger causality (c-GC), which combines the two decisions, and c-GC*, a more conservative variant with a richer conditioning set. We validate both methods on synthetic dynamical systems, established time-series causal discovery benchmarks, Sachs protein-signalling data, and Lorenz-96 simulations. The results show that the proposed criteria recover plausible causal structure in settings with delayed effects, cycles, bidirectional links, and nonlinear or noisy dynamics. The framework clarifies how GC-style inference can be given a causal interpretation without treating temporal prediction alone as sufficient evidence of causation.

arXiv

Learning to Detect Cyber Attacks: Neural Anomaly Detection for Cybersecurity with Theoretical Insights

Tian-Yi Zhou, Matthew Lau, Jizhou Chen, Wenke Lee, Xiaoming Huo

In cybersecurity practice, new forms of cyberattacks continuously emerge, deliberately designed to evade defense systems that rely on previously observed behaviors. Motivated by this challenge, we propose a neural network-based method for anomaly detection that does not rely on (1) prior knowledge of anomaly distributions or (2) the availability of real anomalies during training. Our proposed method trains a neural network classifier using only normal samples, combining the supervision from synthetic anomalies, and is particularly suitable when collecting real anomaly samples is expensive or impractical. The trained classifier is proven to attain minimax excess risk, and more importantly, it is guaranteed to learn the boundary of the normal region. Once the normal region is well estimated, the model can detect a wide range of anomalies without requiring explicit modeling of their distributions. Extensive experiments across cybersecurity, industrial, and medical anomaly detection tasks demonstrate that our method is consistently robust and competitive compared to state-of-the-art baselines. Notably, in the context of network intrusion detection, our approach significantly enhances the detection of difficult and previously unseen cyberattacks compared to other baselines.

arXiv

Bridging AI and Energy Forecasting: An Autonomous Workflow with Customized Toolkit

Zhixian Wang, Leandro Von Krannichfeldt, Qingsong Wen, Chaoli Zhang, Liang Sun, Shirui Pan, Yi Wang

Energy forecasting is crucial for the power grid, but fundamentally different from general time series analysis: it highly relies on covariates like meteorological factors, and its goals must align with actual power grid operations, such as risk assessment and system reliability. In order to bridge the huge gap between advanced machine learning forecasting models and actual power grid demand, this paper proposes an autonomous forecasting workflow based on LLMs. As a virtual analyst, the agent replaces tedious manual adjustments by autonomously analyzing data features, dynamically orchestrating optimal forecasting pipelines, and generating actionable analysis reports for decision-makers. The carefully arranged pipeline directly addresses the demand of the power grid through probabilistic forecasting for uncertainty quantification. Furthermore, the framework acts as an automated testbed, enabling seamless A/B testing of specific algorithmic plugins across various architectures to evaluate their empirical effectiveness. As the foundation of the algorithm, the underlying toolkit integrates 31 advanced temporal architectures and 6 customized exogenous modules, resulting in 146 highly configurable variants. In addition, to provide experience references for the agents, we established a large-scale benchmark on 21 energy datasets and released new high-quality renewable energy datasets with meteorological factors.

arXiv

Deep R Programming

Marek Gagolewski

Deep R Programming is a comprehensive and in-depth introductory course on one of the most popular languages for data science. It equips ambitious students, professionals, and researchers with the knowledge and skills to become independent users of this potent environment so that they can tackle any problem related to data wrangling and analytics, numerical computing, statistics, and machine learning. This textbook is a non-profit project. Its online and PDF versions are freely available at <https://deepr.gagolewski.com/>.

arXiv

MSGNN: A Spectral Graph Neural Network Based on a Novel Magnetic Signed Laplacian

Yixuan He, Michael Permultter, Gesine Reinert, Mihai Cucuringu

Signed and directed networks are ubiquitous in real-world applications. However, there has been relatively little work proposing spectral graph neural networks (GNNs) for such networks. Here we introduce a signed directed Laplacian matrix, which we call the magnetic signed Laplacian, as a natural generalization of both the signed Laplacian on signed graphs and the magnetic Laplacian on directed graphs. We then use this matrix to construct a novel efficient spectral GNN architecture and conduct extensive experiments on both node clustering and link prediction tasks. In these experiments, we consider tasks related to signed information, tasks related to directional information, and tasks related to both signed and directional information. We demonstrate that our proposed spectral GNN is effective for incorporating both signed and directional information, and attains leading performance on a wide range of data sets. Additionally, we provide a novel synthetic network model, which we refer to as the Signed Directed Stochastic Block Model, and a number of novel real-world data sets based on lead-lag relationships in financial time series.

arXiv

Heterogeneous Ranking in Industrial-Scale Recommender Systems: A Case Study

Di Bai, Jintao Liu, Zhenwei Tang, Peifan Wu, Nada Al-Thawr, Luoshu Wang

Heterogeneous recommendation feeds present complex challenges that extend beyond those found in highly homogeneous environments (e.g., music-only or video-only closed-ecosystem platforms). In Google Discover, a unified feed integrates diverse content sourced from the decentralized open web, including web articles, long-form and short-form videos, user-generated content (UGC), and beyond. Different content types exhibit distinct feature densities and user interaction patterns. Building a unified ranking model that sustains high performance across such heterogeneity, while avoiding negative transfer or majority bias, remains a significant industrial challenge. This paper presents an end-to-end case study on the industrial-scale multi-task ranking of heterogeneous feeds, grounded in real-world deployment. We introduce HA-MoE, a heterogeneity-adaptive multi-gated mixture-of-experts architecture that incorporates explicit heterogeneity context into both gating networks and expert representations. This approach enables effective specialization without significantly increasing operational overhead. To support reliable deployment, we introduce LENS, a lightweight observability framework that provides interpretable diagnostics of expert specialization and tracks this functional heterogeneity across continuous retraining. We evaluate our method using Dual-Level AUC (DL-AUC), a heterogeneity-aware evaluation metric that combines global ranking performance with cross-segment ranking correctness. Offline evaluations on a large-scale industrial dataset demonstrate consistent improvements over baseline models. Furthermore, online A/B testing confirms gains in feed activity and exploration metrics. Together, offline and online results validate the effectiveness of our approach for managing heterogeneity in industrial-scale recommender systems.

arXiv

Policy Gradient Steering: Interventions from Behavioral Objectives

Yoann Poupart, Aurélie Beynier, Nicolas Maudet

Activation steering has emerged in large language models as a lightweight alternative for dynamically changing a model's behavior at inference time. However, we show that existing steering methods fail to steer even a simple policy in a two-route gridworld environment. To address this limitation, we propose Policy Gradient Steering (PGS), which formulates steering as a reinforcement learning problem. PGS accumulates gradients of a temporary behavioral objective over a small set of rollouts or demonstrations to construct a removable task vector. We first demonstrate the calibration and reversibility of PGS in a two-route gridworld environment. Using chess puzzles, we then evaluate independently fitted PGS vectors both in isolation and in combination, finding that compatible tactical objectives accumulate constructively. Finally, in competitive football, we show that PGS can alter specific team behaviors and that its effects transfer across opponents. Together, these results show that policy gradients provide a natural interface for constructing temporary and composable behavioral adaptations across diverse decision-making domains.

arXiv

Recognition and Label-Free Adaptation Across Recording Sessions in Surface-EMG Gesture Decoding

Jethro Odeyemi, W. J. Zhang

Recognition accuracy obtained during a recording session does not persist when a user puts on the electrodes again after the electrodes had previously been removed. The electrodes may have moved slightly, the skin may be drier or wetter, or the elbow may be positioned differently; these factors all contribute to day-to-day variability and therefore represent a major obstacle to implementing successful pattern-recognition based myoelectric control systems in daily practice. However, simply recalibrating a user's hand for 20 min at every doff/don event is a clearly unrealistic expectation. A montage-agnostic encoder built for cross-user, cross-montage transfer is trained here using data collected during a particular recording session, and then applied to data collected later in a different recording session without adjusting anything, on the ten intact subjects of NinaPro DB6. The performance of this approach is compared to that of a per-user LDA classification pipeline, and to that of two published approaches that only rely on source data collected from the same recording session. Carried unchanged across recording sessions, the encoder retains 0.688 macro-F1 against 0.540 for the per-user pipeline, and, on the per-window metric the published baselines use, sits above both published source-only results, a band of two points that locates the encoder rather than ranking it. Of five label-free test-time adaptations, only feature-statistic alignment improves every subject; batch-normalisation re-estimation, a standard method in the domain-adaptation literature, collapses this architecture entirely. Aligning the encoder's feature statistics to the new session recovers about what a single labelled calibration repetition would.

arXiv

A Montage-Agnostic Encoder for Calibration-Light Cross-User Gesture Recognition from Surface Electromyography

Jethro Odeyemi, W. J. Zhang

Pattern-recognition control promises a myoelectric prosthesis that responds to many intended gestures rather than one or two, but the promise has stayed in the laboratory. A recogniser trained on one person rarely transfers to the next, and useful performance usually demands a fresh round of labelled calibration from the end user. A montage-agnostic encoder is introduced that reads each electrode with shared weights and locates it by its physical coordinate rather than its index, so one architecture ingests any channel count without montage-specific parameters. Trained across users, it exceeds a per-user Hudgins and linear-discriminant classifier by 0.234 macro-F1 on DB1 for every held-out subject and by 0.108 on DB2, and falls below it on the ten-subject DB5. Each of the encoder's three key components individually accounts for more than half of its 3-shot macro F1 in an otherwise budget-matched ablation study. A controlled subject-count sweep shows the margin is close to flat from nine training subjects to thirty-nine, so the training pool binds only as a stability floor below which cross-user training fails to converge; what tracks the direction of the comparison across the three databases is instead the strength of the per-user baseline, which signal fidelity sets. Comparing against an LDA baseline depends on budget spent training models and on how good that baseline is, and self-supervised pretraining had no benefits once a supervised model was adequately trained.

arXiv

Strategies for Milestone-driven Start-ups in Multi-activity Settings

Zhengli Wang

New venture start-ups need to ``survive'' through multiple stages of reaching milestone targets. We investigate the strategies for start-ups in a milestone-oriented setting. We examine a model of an entrepreneurial start-up firm, where its state is captured by a diffusion process. The entrepreneur can choose between multiple activities (or controls), which incur different cost and determine the drift and the variance of the process. Depending on whether the process reaches a fixed upper boundary or a lower one, the start-up firm succeeds or fails. Continuous-time stochastic models with multiple ($\ge 3$) controls are typically very challenging to deal with. In this work, we are able to completely solve for the optimal policy and provide an explicit characterization of its structure. In particular, the optimal policy only uses controls from a set characterized by a so-called efficient frontier curve that orders the controls by two intuitive measures: riskiness (drift-to-volatility ratio) and cost-effectiveness (drift-to-cost ratio). A unique feature of our model is that depending on the model parameters, the efficient frontier curves can be of different types, resulting in qualitatively different structures of the optimal policy. As far as we know, this is the first study that analyzes a stochastic control model which admits efficient frontier curves of different types. Our work provides start-up firms with intuitive measures to evaluate their activities and offers valuable insights on how the optimal strategies in a milestone-oriented setting change qualitatively contingent upon the specific scenario. We believe the results provide a foundational block in the study of entrepreneurial decision-making.

arXiv

Memory Efficient Tabular Foundation Models

Shuting Luo, Monika Mikhail Kanaan, Cameron Gordon, Anna Leontjeva, Simon Lucey

Tabular Foundation Models, such as TabPFN, have received a large amount of recent attention due to their performance on in-context tabular machine learning tasks, which often exceeds classical baselines. However, practical deployment considerations of these models has received less attention. In this paper we investigate the memory requirements for these models. We demonstrate that employing model compression approaches can enable memory reductions of up to 7.6 with similar levels of performance, reducing deployment requirements by nearly 87%. Our work provides insight to practitioners seeking efficient deployment of these models in practical settings.

arXiv

Strategy, Not Payoffs: A Behavioural Embedding of Normal-Form Games

Joshua Caiata, Sreepriya Pulyassary, Xiang Li, Kate Larson

Learning a strategic task changes more than what is directly taught: fine-tuning on one game can either enhance or degrade an agent's ability to reason in another. Understanding and predicting this transfer of strategic capabilities, however, remains a key challenge for large language models (LLMs). Normal-form games provide an ideal testbed for analyzing this phenomenon, as they feature explicitly defined payoffs and well-characterized equilibrium behaviours. In this work, we investigate whether game embeddings can explain and predict changes in LLM strategic capabilities following fine-tuning across different games. We propose a lightweight two-feature embedding that captures fundamental behavioural demands: the entropy of the Nash equilibrium and the sensitivity of optimal responses to an opponent's action. We show that while existing published structural embeddings primarily memorize game identities and fail to generalize, our behavioural embedding reliably predicts performance changes on held-out games. These results demonstrate that the transfer of strategic capabilities in LLMs is not dictated by the payoff geometry of a game, but by the underlying structure of the decision-making behaviour it requires.

arXiv

Can an Actor-Critic Optimization Framework Improve Analog Design?

Sounak Dutta, Fin Amin, Sushil Panda, Jonathan Rabe, Yuejiang Wen, Paul Franzon

Analog design often slows down because even small changes to device sizes or biases require expensive simulation cycles, and high-quality solutions typically occupy only a narrow part of a very large search space. While existing optimizers reduce some of this burden, they largely operate without the kind of judgment designers use when deciding where to search next. This paper presents an actor-critic optimization framework (ACOF) for analog sizing that brings that form of guidance into the loop. Rather than treating optimization as a purely black-box search problem, ACOF separates the roles of proposal and evaluation: an actor suggests promising regions of the design space, while a critic reviews those choices, enforces design legality, and redirects the search when progress is hampered. This structure preserves compatibility with standard simulator-based flows while making the search process more deliberate, stable, and interpretable. Across our test circuits, ACOF improves the top-10 figure of merit by an average of 38.9% over the strongest competing baseline and reduces regret by an average of 24.7%, with peak gains of 70.5% in FoM and 42.2% lower regret on individual circuits. By combining iterative reasoning with simulation-driven search, the framework offers a more transparent path toward automated analog sizing across challenging design spaces.

arXiv

Concurrent training methods for Kolmogorov-Arnold networks: Disjoint datasets and FPGA implementation

Andrew Polar, Michael Poluektov

The present paper introduces concurrency-driven enhancements to the training algorithm for the Kolmogorov-Arnold networks (KANs) that is based on the Newton-Kaczmarz (NK) method. Prior research shows that KANs trained using the NK-based approach outperform classical neural networks (multilayer perceptrons - MLPs) both in terms of accuracy and training time. Up to now, the fundamental limitation of the algorithm has been the sequential computation of the updates - each update depends on the results of the previous step, obstructing parallelisation; even though parallelisation of some parts of the algorithm, such as the evaluation of the basis functions, has already been proposed and tested. However, substantial acceleration is achievable. Three complementary concurrency-driven novel strategies are proposed in the present paper: (i) a pre-training procedure tailored to the NK updates' structure, (ii) training on disjoint subsets of data together with models' merging, and (iii) a division-free customisation of the algorithm for field-programmable gate arrays (FPGAs), which is implemented and tested directly on the device. Computational experiments are used to assess the scalability and to compare the authors' KAN implementation with popular MLP packages. All presented experimental results are fully reproducible, with the complete source codes available online.

Keep browsing

More papers from top AI conferences

30+ top conferences, by conference, year, and topic.