AI conference papers

AI conference paper library

Search papers from 30+ top conferences, including NeurIPS, ICLR, ICML, CVPR, ACL and AAAI. Filter by keyword, then open any paper for its abstract, authors and links.

arXiv

Non-Stationary Functional Bilevel Optimization

Jason Bohne, Ieva Petrulionyte, Michael Arbel, Julien Mairal, Paweł Polak

Functional bilevel optimization (FBO) provides a powerful framework for hierarchical learning in function spaces, yet current methods are limited to static offline settings and perform suboptimally in online, non-stationary scenarios. We propose SmoothFBO, the first algorithm for non-stationary FBO with both theoretical guarantees and practical scalability. SmoothFBO introduces a time-smoothed stochastic hypergradient estimator that reduces variance through a window parameter, enabling stable outer-loop updates with sublinear regret. Importantly, the classical parametric bilevel case is a special reduction of our framework, making SmoothFBO a natural extension to online, non-stationary settings. Empirically, SmoothFBO consistently outperforms existing FBO methods in non-stationary hyperparameter optimization and model-based reinforcement learning, demonstrating its practical effectiveness. Together, these results establish SmoothFBO as a general, theoretically grounded, and practically viable foundation for bilevel optimization in online, non-stationary scenarios.

arXiv

Boltzmann generators for amorphous particle systems

Louis Grenioux, Leonardo Galliano, Ludovic Berthier, Giulio Biroli, Marylou Gabrié

Sampling configurations in thermodynamic equilibrium is a long-standing challenge in statistical physics. Boltzmann generators address this problem by employing generative models to propose independent configurations, which are then reweighted via importance sampling using exact likelihood evaluations. Recent Boltzmann Generators based on continuous normalizing flows and flow matching have achieved significant success for particle systems and biomolecules. However, these approaches have not been extended to amorphous materials (glasses), for which equilibrium sampling is notoriously slow. Because of their disordered structure, the invariances and geometrical constraints of amorphous materials differ from those of crystals and biomolecules, preventing the direct use of existing generative models. Here, we develop Boltzmann Generators tailored to amorphous materials by building the required equivariances directly into Riemannian stochastic interpolants. Our framework incorporates periodic boundary conditions and particle symmetries using equivariant graph neural networks. Numerical experiments demonstrate that enforcing physical symmetries significantly improves the accuracy of Boltzmann Generators, but also reveal an intrinsic limitation of the continuous-flow formulation: accumulated numerical errors during likelihood integration break time-reversibility, compromising exact thermodynamic reweighting. These results reveal a fundamental challenge for continuous-flow generative models in statistical mechanics and call for alternative approaches that preserve exact thermodynamic consistency.

arXiv

Exploiting the Alternatives: Coordinated Learning via Hierarchical RL for Dynamic VNEAP

Ali Al Housseini, Cristina Rottondi, Sebastian Troia, Omran Ayoub

Virtual Network Embedding (VNE) is a key enabler of network slicing, yet most formulations assume that each Virtual Network Request (VNR) has a fixed topology. Recently, VNE with Alternatives (VNEAP) was introduced to capture malleable VNRs, where each request can be instantiated using one of several functionally equivalent topologies that trade resources differently. This flexibility can improve embedding feasibility, but only if the orchestrator can jointly select suitable alternatives and embed them under dynamic arrivals. This paper proposes HRL-VNEAP, a hierarchical reinforcement learning approach for dynamic VNEAP. A high-level policy selects the most suitable alternative topology (or rejects the request), and a low-level policy embeds the chosen topology onto the substrate network. Experiments on realistic substrate topologies under varying arrival rates show that naive exploitation strategies provide only modest gains, whereas HRL-VNEAP outperforms state of the art approaches, improving acceptance ratio by up to 22%, and net profit by up to 20%. An offline MILP upper bound is also used on tractable instances to quantify the remaining optimality gap.

arXiv

Black box behavioural modelling: Predicting human activity schedules with a deep conditional generative approach

Fred Shone, Tim Hillel

Modelling the complexity and diversity of human activity scheduling behaviour is inherently challenging. We demonstrate ActVAE, a deep conditional-generative machine learning approach for the modelling of activity schedules. Suitable for application in activity-based demand modelling frameworks, schedules are modelled as conditional on individual, household and schedule information, such as age, income, and access to public transit. We demonstrate the rapid generation of precise, realistic and diverse schedules dependent on input labels. We extensively evaluate and compare model capabilities against baseline models using a joint-density estimation framework. In addition to providing a novel alternative to existing scheduling approaches, our work highlights the value of explicitly modelling the randomness of complex and diverse human behaviours.

arXiv

Limit Theorems for Stochastic Gradient Descent in High-Dimensional Single-Layer Networks

Parsa Rangriz

This paper studies the high-dimensional scaling limits of online stochastic gradient descent (SGD). Building on the work of Ben Arous, Gheissari, and Jagannath on the effective dynamics of SGD, we study the critical scaling regime of the step size for single-layer networks. Below this regime the effective dynamics are governed by deterministic (ballistic) limits, whereas at the critical scale a correction term emerges that changes the phase diagram. Near the fixed points of these dynamics, we show that the diffusive (SDE) limit of the rescaled correlation is an Ornstein-Uhlenbeck process. More precisely, it is mean-reverting whenever the information exponent is at least three. At information exponent two the drift has no universal sign, and the fixed point may become repelling; we show this explicitly for phase retrieval, where the sign is determined by the step size and the noise level. These results illustrate the limitations of deterministic scaling limits in capturing stochastic fluctuations in high-dimensional learning dynamics.

arXiv

Structure-Preserving Physics-Informed Neural Network for the Korteweg--de Vries (KdV) Equation

Victory Obieke, Emmanuel Oguadimma

Physics-Informed Neural Networks (PINNs) offer a flexible framework for solving nonlinear partial differential equations (PDEs), yet conventional implementations often fail to preserve key physical invariants during long-term integration. This paper introduces a \emph{structure-preserving PINN} framework for the nonlinear Korteweg--de Vries (KdV) equation, a prototypical model for nonlinear and dispersive wave propagation. The proposed method embeds the conservation of mass and Hamiltonian energy directly into the loss function, ensuring physically consistent and energy-stable evolution throughout training and prediction. Unlike standard \texttt{tanh}-based PINNs~\cite{raissi2019pinn,wang2022modifiedpinn}, our approach employs sinusoidal activation functions that enhance spectral expressiveness and accurately capture the oscillatory and dispersive nature of KdV solitons. Through representative case studies -- including single-soliton propagation (shape-preserving translation), two-soliton interaction (elastic collision with phase shift), and cosine-pulse initialization (nonlinear dispersive breakup) -- the model successfully reproduces hallmark behaviors of KdV dynamics while maintaining conserved invariants. Ablation studies demonstrate that combining invariant-constrained optimization with sinusoidal feature mappings accelerates convergence, improves long-term stability, and mitigates drift without multi-stage pretraining. These results highlight that computationally efficient, invariant-aware regularization coupled with sinusoidal representations yields robust, energy-consistent PINNs for Hamiltonian partial differential equations such as the KdV equation.

arXiv

Spectral functions in Minkowski quantum electrodynamics from neural reconstruction

Rodrigo Carmo Terin

We study neural reconstructions of quenched rainbow quantum electrodynamics (QED) Dyson--Schwinger benchmarks in Minkowski-related kinematics. Using the dispersive formulation as motivation, we separate the Euclidean Fukuda--Kugo equation, the spectral unitary equations, and the modified unitary equations. The Fukuda--Kugo benchmark is solved directly and shows the expected zero crossing above the critical region $α_c=π/3$. Neural reconstructions with free output reproduce this behavior, while positivity-constrained ansätze fail in the supercritical regime. Thus, spectral positivity should be treated as a diagnostic of the Lehmann representation, not imposed blindly as a neural constraint.

arXiv

torchsom: The Reference PyTorch Library for Self-Organizing Maps

Louis Berthier, Ahmed Shokry, Maxime Moreaud, Guillaume Ramelet, Eric Moulines

This paper introduces torchsom, an open-source Python library that provides a reference implementation of the Self-Organizing Map (SOM) in PyTorch. This package offers three main features: (i) dimensionality reduction, (ii) clustering, and (iii) friendly data visualization. It relies on a PyTorch backend, enabling (i) fast and efficient training of SOMs through GPU acceleration, and (ii) easy and scalable integrations with PyTorch ecosystem. Moreover, torchsom follows the scikit-learn API for ease of use and extensibility. The library is released under the Apache 2.0 license with 90% test coverage, and its source code and documentation are available at https://github.com/michelin/TorchSOM.

arXiv

Crashing Waves vs. Rising Tides: Findings on AI Automation from Thousands of Worker Evaluations of Labor Market Tasks

Matthias Mertens, Adam Kuzee, Brittany S. Harris, Harry Lyu, Wensu Li, Jonathan Rosenfeld, Meiri Anto, Martin Fleming, Neil Thompson

We characterize AI automation as a continuum between crashing waves, in which capabilities jump abruptly across narrow task sets, and rising tides, in which capabilities improve continuously and broadly. Using evidence from more than 6,000 text-based, LLM-addressable tasks derived from the U.S. Department of Labor's O*NET taxonomy and over 60,000 evaluations by experienced workers, we find little evidence of crashing waves (contrary to existing views). Instead, rising tides are the primary form of AI progress. AI performance is high and improving rapidly across many tasks. In 2024-Q2, models completed text-based tasks that take humans about 1.5 hours to complete with roughly 60% success, rising above 70% by 2025-Q3. If recent trends in AI capability growth persist, frontier LLMs will be able to complete most text-based tasks at minimally sufficient quality with 88%-97% success by 2030.

arXiv

EZASP - Facilitating the Usage of ASP

Rafael Martins, Matthias Knorr, Ricardo Gonçalves

Answer Set Programming (ASP) is a declarative programming language used for modeling and solving complex combinatorial problems. It has been successfully applied to a number of different real-world problems. However, learning its usage can prove challenging as the declarative language, from a conceptual perspective, differs substantially from imperative programming, and programs are not required to adhere to any particular structure, offering arguably almost too much freedom for a beginner. Recently, a new methodology called Easy Answer Set Programming (Easy ASP) has been introduced that aims to aid in this learning process by focussing on a well-defined fragment of the ASP language and introducing additional structure to the programs. However, while this methodology can indeed be employed, to the best of our knowledge, no tool integrates its features currently. In this paper, we present EZASP, a Visual Studio Code extension designed to support the development of ASP programs following the Easy ASP methodology. It covers and extends the language fragment of Easy ASP and provides the user with warnings in the case of deviations from the methodology as well as the possibility to automatically reorder the program. Complementarily, it also adds syntax error highlighting, including detection of non-safe variables directly while editing, and configurability, as all features can be optionally disabled. A small user study in the context of university teaching suggests that these features are benefitial for both new and experienced users.

arXiv

Semantics at an Angle: When Cosine Similarity Works Until It Doesn't

Kisung You

Cosine similarity is a standard comparison rule for learned representations in information retrieval, natural language processing, computer vision, and multimodal learning. Its popularity is well founded: it removes positive radial scale, is computationally convenient, and often matches objectives that train normalized embeddings. These same properties also delimit what cosine can express. Normalization discards radial variation; anisotropic representations can compress angular contrast; high-dimensional neighborhoods can develop hubs; and a symmetric, uncalibrated score may be mismatched to the relation of interest. This article offers a selective review organized around a simple principle: the usefulness of cosine similarity depends jointly on the learned representation, any normalization or post-processing, the scoring rule, and the downstream decision. We derive the main geometric identities, distinguish failure mechanisms that are often conflated, review representative evidence about embedding norms, and describe objective-matched, geometry-aware, hubness-aware, norm-aware, and learned alternatives. The central conclusion is conditional rather than adversarial: cosine's positive-scale invariance is justified when radial variation is nuisance or fixed by the representation contract, but its angular geometry and downstream decision must still be validated.

arXiv

Classifier-pruned Bayesian optimization for particle accelerator tuning: Exploring temporally structured manifold of 6D beam phase space

Mahindra Rautela, Alan Williams, Alexander Scheinker

Complex dynamical systems, such as particle accelerators, often require intricate and time-consuming tuning procedures to achieve optimal performance. In many cases, these procedures must also estimate the optimal system parameters governing the dynamics of a spatiotemporal beam, making the task a high-dimensional optimization problem. To address this, we propose a Classifier-pruned Bayesian Optimization-based Latent space Tuner (CBOL-Tuner), a framework for efficient exploration within a temporally-structured latent manifold of 6D beam phase space. The CBOL-Tuner integrates a conditional variational autoencoder for latent space representation, a long short-term memory network for temporal dynamics, a lightweight neural network for parameter estimation, and a classifier-pruned Bayesian optimizer to adaptively search and filter the latent space for optimal solutions.

arXiv

Evolutionarily Stable Stackelberg Equilibrium

Sam Ganzfried

We present a new solution concept called evolutionarily stable Stackelberg equilibrium (SESS). We study the Stackelberg evolutionary game setting in which there is a single leading player and a symmetric population of followers. The leader selects an optimal mixed strategy, anticipating that the follower population plays an evolutionarily stable strategy (ESS) in the induced subgame and may satisfy additional ecological conditions. We consider both leader-optimal and leader-pessimal selection among ESSs, which arise as special cases of our framework. Prior approaches to Stackelberg evolutionary games either define the follower response via evolutionary dynamics or assume rational best-response behavior, without explicitly enforcing stability against invasion by mutations. We present algorithms for computing SESS in discrete and continuous games, and validate the latter empirically. Our model applies naturally to biological settings; for example, in cancer treatment the leader represents the physician and the followers correspond to competing cancer cell phenotypes.

arXiv

Post-Hoc Reasoning in Chain of Thought: Decoding and Steering Pre-Committed Answers

Kyle Cox, Darius Kianersi, Adrià Garriga-Alonso

As chain of thought (CoT) has become central to scaling reasoning capabilities in large language models (LLMs), it has also emerged as a promising tool for interpretability, suggesting the opportunity to understand model decisions through verbalized reasoning. However, the utility of CoT toward interpretability depends upon its faithfulness---whether the model's stated reasoning reflects the underlying decision process. We provide mechanistic evidence that instruction-tuned models often determine their answer before generating CoT. Training linear probes on residual stream activations at the last token before CoT, we can predict the model's final answer with >0.9 AUC on most tasks. We find that these directions are not only predictive, but also causal: steering activations along the probe direction often flips model answers, with flip rates substantially exceeding norm-matched orthogonal baselines across most model-dataset pairs. When steering induces incorrect answers, we observe two distinct failure modes: confabulation (fabricating false premises) and non-entailment (stating correct premises but drawing unsupported conclusions). While post-hoc reasoning may be instrumentally useful when the model has a correct pre-CoT belief, these failure modes suggest it can result in undesirable behaviors when reasoning from a false belief.

arXiv

Exploiting Exogenous Structure for Sample-Efficient Reinforcement Learning

Jia Wan, Sean R. Sinclair, Devavrat Shah, Martin J Wainwright

We study a structured class of Markov Decision Processes, known as Exo-MDPs, in which the state space is partitioned into exogenous and endogenous components. Exogenous states evolve stochastically, independent of the agent's actions, while endogenous states evolve deterministically based on both state components and actions. Exo-MDPs capture many operations research settings, including inventory control, resource management, and ride-sharing. Our first contribution is structural: we establish a representational equivalence between discrete MDPs, Exo-MDPs, and discrete linear mixture MDPs. Our second contribution is statistical. We characterize the minimax regret of learning in Exo-MDPs when the effective dimension r is small relative to the endogenous state and action spaces. When the exogenous states are unobserved, we prove matching upper and lower regret bounds of order $Θ(Hr \sqrt{K})$ over $K$ episodes of horizon $H$, where $r$ is the effective dimension of the Exo-MDP. When exogenous states are observed, the minimax regret improves to $Θ(H\sqrt{ r K})$, revealing a $Θ(\sqrt{r})$ statistical gap due to observation of the exogenous states. These results show that Exo-MDPs decouple sample complexity from action space and endogenous state space. We validate these insights with experiments on inventory control and resource allocation.

arXiv

OmniCustom: Sync Audio-Video Customization Via Joint Audio-Video Generation Model

Maomao Li, Zhen Li, Kaipeng Zhang, Guosheng Yin, Zhifeng Li, Dong Xu

Existing mainstream video customization methods focus on generating identity-consistent videos based on given reference images and textual prompts. Benefiting from the rapid advancement of joint audio-video generation, this paper proposes a more compelling new task: sync audio-video customization, which aims to synchronously customize both video identity and audio timbre. Specifically, given a reference image $I^{r}$ and a reference audio $A^{r}$, this novel task requires generating videos that maintain the identity of the reference image while imitating the timbre of the reference audio, with spoken content freely specifiable through user-provided textual prompts. To this end, we propose OmniCustom, a powerful DiT-based audio-video customization framework that can synthesize a video following reference image identity, audio timbre, and text prompts all at once in a zero-shot manner. Our framework is built on three key contributions. First, identity and audio timbre control are achieved through separate reference identity and audio LoRA modules that operate through self-attention layers within the base audio-video generation model. Second, we introduce a contrastive learning objective alongside the standard flow matching objective. It uses predicted flows conditioned on reference inputs as positive examples and those without reference conditions as negative examples, thereby enhancing the model ability to preserve identity and timbre. Third, we train OmniCustom on our constructed large-scale, high-quality audio-visual human dataset. Extensive experiments demonstrate that OmniCustom outperforms existing methods in generating audio-video content with consistent identity and timbre fidelity. Project page: https://omnicustom-project.github.io/page/.

arXiv

CASC: Causal Adversarial Subspace Clustering for Multivariate Spatiotemporal Data

Francis Ndikum Nji, Vandana Janeja, Jianwu Wang

Deep subspace clustering plays a critical role in applications involving multivariate spatiotemporal data, such as sea ice monitoring, disease spread analysis, and tracking neuro-degeneration over time. Despite recent advances, existing methods primarily rely on geometric self-expressiveness, assume static subspace structures, and often fail to capture causal dependencies, local spatial interactions, and long-range temporal dynamics inherent in complex spatiotemporal systems. To address these limitations, we propose a novel Causal Adversarial Subspace Clustering (CASC) framework for discovering evolving latent regimes in high-dimensional spatiotemporal data. CASC integrates a U-Net-inspired deep adversarial clustering architecture with stacked FAConvLSTM layers to preserve spatial and temporal structure while learning robust latent representations. A graph attention transformer-based self-expressive network is introduced to jointly model local spatial relationships, global dependencies, and long-range temporal interactions. Furthermore, we propose two new learning objectives: (1) a Causal Subspace Preservation Loss that aligns self-expression coefficients with latent causal relationships, encouraging clusters to reflect underlying causal processes rather than simple feature similarity, and (2) a Dynamic Temporal Subspace Evolution Loss that captures evolving subspace structures and temporal regime transitions in nonstationary environments. Together, these components transform deep subspace clustering from a correlation-driven paradigm into a causal-temporal regime discovery framework.

arXiv

Automatic knot selection in smooth additive models

Nicolás Carrizosa, Vanesa Guerrero, María Durbán

B-spline regression constitutes a widely used framework for nonparametric modeling. The performance of this methodology depends on specifying the number and placement of changepoints, known as knots, prior to the estimation process. Such knot sequence determines the dimension of the B-spline basis used to represent the regression function and the number of coefficients to be estimated. Therefore, the knots' choice affects the model's flexibility, influencing its smoothness and goodness-of-fit. Traditionally, this problem has been addressed either by explicitly selecting knots, via knot-selection algorithms, or by regularization methods, such as P-splines, which automatically tune the regressor's smoothness. The latter have become the standard in generalized additive models (GAMs). In contrast, knot-selection techniques, frequently neglected because of computational or modeling limitations, provide certain advantages which can be valuable in some contexts. In this work, we introduce a novel explicit knot-selection technique for GAMs based on an extension of the adaptive splines (A-splines) knot selection methodology, combined with a customized Fellner-Schall scheme for tuning the associated parameters. Our approach is evaluated on various synthetic and real datasets and compared with P-splines and state-of-the-art knot-selection techniques. The results indicate comparable performance, while producing models built on a substantially smaller number of basis elements.

arXiv

Nipping the Butterfly Effect in the Bud: Self-Output Fine-Tuning for Autoregressive Weather Prediction

Yun-Ye Cai, Hsuan-Tien Lin

Long-horizon weather forecasting is a fundamental challenge in atmospheric science, for which autoregressive Deep Learning Weather Prediction (DLWP) has emerged as the primary paradigm. Although the autoregressive pipeline is highly scalable and flexible, its prediction errors grow rapidly over long forecasting horizons. In this work, we study this error growth phenomenon from both theoretical and empirical perspectives. Our analysis reveals that the growth is driven by a feedback loop between output errors and input distribution shifts. Specifically, the autoregressive process amplifies small initial output errors, which progressively corrupt subsequent input distributions, echoing the butterfly effect in atmospheric science and ultimately deteriorating forecasting accuracy over longer horizons. Furthermore, we show that this distributional shift originates at the earliest stage of inference, with out-of-distribution signatures detectable as early as the first autoregressive step. To mitigate this issue, we propose \textbf{Self-Output Fine-Tuning (SOFT)}, a plug-and-play strategy that leverages the model's own one-step predictions to calibrate the biased input distribution encountered at the first step. Extensive experiments demonstrate that, despite its simplicity, SOFT achieves state-of-the-art performance on long-horizon forecasting tasks and substantially reduces both prediction errors and distributional discrepancy. The success of SOFT highlights the importance of reexamining the fundamental pipeline of deep learning weather prediction, representing a critical pipeline advance for atmospheric science.

arXiv

Spectral Transformation for Layer-wise Global Rank Discovery in Federated LoRA for Vision Transformers

Hariharan Ramesh, Jyotikrishna Dass

Fine-tuning Vision Transformers (ViTs) with low-rank adapters (LoRA) promises better communication efficiency under federated setup, yet existing aggregation strategies face fundamental limitations. Independently averaging these LoRA factors is mathematically inconsistent, introducing cross-term aggregation error. In contrast, approaches that preserve heterogeneous client ranks by concatenating local adapters on the server substantially increase download cost and often require merging global LoRA updates into pretrained weights on the clients, causing reinitialization lag and unstable convergence. Other approaches further increase server-side overhead by reconstructing dense weight updates or training auxiliary models to refine aggregation error. In this work, we propose SpecTraL, spectral transformation for layer-wise global rank discovery, that resolves these challenges within a unified design. SpecTraL stacks local LoRA modules from clients and performs orthonormal Householder Transformation of the stacked adapters directly in the low-rank latent space, eliminating dense reconstruction of the global update and any auxiliary refinement on the server. By leveraging the Spiked Covariance Model from Random Matrix Theory, SpecTraL analytically separates the global consensus signal from non-IID noise, discovering optimal layer-wise global ranks without manual hyperparameter tuning. To match local ranks in subsequent rounds, we introduce a padding-aware initialization framework that lets clients incorporate residual LoRA dimensions without re-merging them into the pre-trained base model. Experiments on federated fine-tuning of ViT-B/16 and ViT-L/16 over DomainNet and NICO++ demonstrate improved accuracy-communication trade-offs, reduced server computation, and elimination of hyperparameter search for rank selection. Our code is available at https://github.com/DASS-Lab-Group/SpecTraL

Keep browsing

More papers from top AI conferences

30+ top conferences, by conference, year, and topic.