paperlist.ai

AI conference papers

AI conference paper library

Search papers from 30+ top conferences, including NeurIPS, ICLR, ICML, CVPR, ACL and AAAI. Filter by keyword, then open any paper for its abstract, authors and links.

arXiv

Mechanistic Interpretability-Guided Selective Fine-Tuning of Vision-Language Models for Centimeter-Level Flood Depth Estimation

Nafis Fuad, Xiaodong Qian, Dongxiao Zhu

Urban flooding poses an escalating threat to transportation infrastructure, yet no operational system provides real-time, street-level flood-depth estimates at centimeter resolution. This paper presents three vision-language models fine-tuned for continuous flood-depth estimation from street-level imagery: FloodLlama-Dense, a fully fine-tuned QLoRA baseline, and FloodLlama-MI5 and FloodLlama-MI6, interpretability-guided sparse variants that fine-tune only the top five and six causally relevant cross-attention layers identified through mechanistic interpretability analysis, respectively. Training uses an approximately 610,000-image subset of a 2.81-million-image synthetic corpus generated in Unreal Engine 5. The dataset combines single-vehicle subsets with 5 cm depth increments and mixed-vehicle subsets with 1 cm depth increments, spanning seven vehicle types, four weather conditions, and flood depths from 0 to 40 cm. FloodLlama-Dense achieves an MAE of 0.40 cm, an RMSE of 1.97 cm, and an Acc@5cm of 97.59%. Mechanistic interpretability analysis combining linear probing, logit lens, centered kernel alignment (CKA), and cross-attention entropy reveals a two-stage adaptation pattern: layers L13-L22 restructure visual representations, while depth first becomes linearly decodable at layer L23. FloodLlama-MI5 and FloodLlama-MI6 leverage this insight by fine-tuning only five or six of the eight cross-attention layers, achieving an 86-88% reduction in trainable parameters (6.55-7.86 million versus 54.4 million) with minimal accuracy loss. On a real-world benchmark, FloodLlama-MI6 achieves 98.62% accuracy, compared with 86.61% for the published STURM-FloodDepth baseline.

arXiv

XEns-CKD: An Explainable Ensemble-Based Approach for Chronic Kidney Disease Stage Detection

Rehan Ahmad, Gousia Habib, Muhammad Shaban, Ishfaq Ahmad Malik

Chronic kidney disease (CKD) is a silent disease. Its progression may not significantly hamper a person's daily routine. Human kidney function can be classified as normal or as one of the five stages of CKD. Early detection of the CKD stage can help patients understand the functional status of their kidneys and follow medical advice to slow CKD progression. In this paper, we propose XEns-CKD, a novel ensemble vision transformer-based scheme for CKD stage classification using ultrasound images. Three ViTs were trained on a private ultrasound image dataset using different training parameters. The performance of each ViT was evaluated using macro sensitivity, macro specificity, macro precision, macro F1-score, macro Youden index, the Matthews correlation coefficient (MCC), and macro balanced accuracy. The ensemble model achieved an overall classification accuracy of 86.36%. This work also emphasizes identifying and interpreting kidney regions affected by CKD progression. Explainable artificial intelligence techniques, including LIME, LRP, Attention-Min, and Attention-Max, were used to improve model transparency and clinical trust. An attention map combining the Attention-Min and Attention-Max results effectively identified and interpreted kidney regions affected during CKD progression from one stage to another. The attention map also highlighted the effects of CKD progression in these regions. Compared with existing methods, the proposed method classified the five CKD stages and normal kidney status with a 4% improvement in accuracy.

arXiv

MVMD: A Multi-View Approach for Enhanced Mirror Detection

Yidan Shen, Yu Wen, Chen Zhang, Xin Fu, Renjie Hu

In 3D reconstruction, mirrors introduce significant challenges by creating distorted and fragmented spaces, resulting in inaccurate and unreliable 3D models. As 3D reconstruction typically relies on multi-view images to capture different perspectives of a scene, detecting and labeling mirrors in multi-view images before reconstruction can effectively address this issue. However, existing methods focus solely on single-image detection, overlooking the rich information provided by multi-view setups. To overcome this limitation, we propose MVMD, a novel Multi-View Mirror Detection method, along with the first database specifically designed for mirror detection in multi-view scenes. The design of MVMD is grounded in the inherent associations between objects seen from different views and those reflected inside and outside of mirrors. These relationships are learned through cross- and self-attention mechanisms. MVMD consists of three key blocks: the Inter-Views Block tracks the shifts of objects within mirrors caused by changes in viewpoint; the Intra-View Block detects object reflections inside mirrors; and the Refinement Block sharpens mirror boundaries and enhances detected details. Experimental results show that our method improves accuracy by up to 2.6% and IoU by up to 11.1%, compared to single-image mirror detection techniques. This substantial improvement makes MVMD particularly effective for computer vision tasks, especially in enhancing the accuracy of 3D reconstruction in mirror-dense environments.

arXiv

Learning Physical Interaction: A Survey of Tactile- and Force-aware Robot Learning

Shilin Shan, Chuhao Zhou, Ruize Wang, Xinyan Chen, Xiangyu Chen, Xinyu Zhou, Boyu Ma, Iris Yuxuan Hu, Jingliang Li, Celeste Yuxuan Hu, Geng Li, Guohao Chen, Tianrui Zhu, Zhe Li, Yanjie Ze, Haoran Geng, Zhiyang Dou, Jianxin Bi, Yuejiang Liu, Jianshu Zhou, Jiachen Li, Paul Liang, Tatsuya Harada, Robert Katzschmann, Harold Soh, Na Li, Edward Johns, Danica Kragic, Jan Peters, Wojciech Matusik, Masayoshi Tomizuka, Jitendra Malik, Jianfei Yang

Physically grounded robot intelligence requires robots to perceive, reason about, and regulate their interactions with the physical world. This capability is particularly critical in contact-sensitive manipulation, where successful task execution depends not only on visual perception and motion generation, but also on force regulation and adaptive control. In this context, recent robot learning methods have made substantial progress by integrating force, tactile, vision, language, and proprioceptive sensing into learned manipulation policies. In parallel, many systems adopt multi-phase architectures that combine high-level policies, action-refinement modules, and low-level controllers to bridge semantic task understanding with reactive physical execution. Despite these advances, existing surveys have not explicitly reviewed force- and tactile-aware robot learning from a unified perspective that jointly captures multimodal sensing and multi-phase system design. This survey addresses this gap by proposing TF-ART, a Tactile/Force-Aware Robot learning Taxonomy for multimodal and multi-phase frameworks, which maps individual methods into a unified hierarchical structure. The framework characterizes how recent works organize observation modalities, encode and fuse heterogeneous sensory inputs, generate and refine actions across multiple phases, and connect learned policies to reactive robot-end control. Building on this methodological view, we further examine the task settings and infrastructure requirements of physical interaction, thereby integrating both algorithmic and practical perspectives on force- and tactile-aware robot learning.

arXiv

AeroDPO: Unleashing Lightweight UAV Navigation with High-Fidelity Perception and Automated Preference Optimization

Peng Xu, Chengcheng Wang, Shaohua Wan

Vision-Language Navigation for Unmanned Aerial Vehicles (UAV-VLN) requires rapid and reactive control in complex 3D environments. Recent minimalist end-to-end paradigms show great promise but typically rely on massive language models containing billions of parameters, incurring prohibitive latency for real-world edge deployment. In this paper, we challenge this parameter-heavy reliance. Comprehensive cross-scale evaluations reveal the critical insight that perception quality fundamentally outweighs language reasoning capacity. We demonstrate that a lightweight 2B model equipped with high-fidelity visual inputs completely matches the overall success rates of massive 7B baselines. However, this minimalist policy exposes a fundamental robustness flaw inherent to pure Behavior Cloning (BC). Lacking explicit negative feedback, the agent fails to internalize robust spatial constraints and exhibits alarming collision rates in out-of-distribution (OOD) scenarios. To overcome this vulnerability without relying on unscalable human annotations, we propose AeroDPO, a zero-cost automated Direct Preference Optimization pipeline driven by deterministic physical simulation state rollback. Upon detecting collisions, the system autonomously rewinds the environment to extract causal reasoning errors as rejected actions, applies decoupled privileged interventions to synthesize collision-avoidance preferred maneuvers, and leverages an offline vision language inspector to filter visual ambiguities. By equipping our 2B model with this automated data flywheel, AeroDPO boosts success rates to 49.16% on unmapped scenarios while drastically suppressing collision rates, establishing a new SOTA for autonomous aerial agents.

arXiv

Self-Evolving Recommendation System: End-To-End Autonomous Model Optimization With LLM Agents

Haochen Wang, Yi Wu, Daryl Chang, Li Wei, Lukasz Heldt

Optimizing large-scale machine learning systems, such as recommendation models for global video platforms, requires navigating a massive hyperparameter search space and, more critically, designing sophisticated optimizers, architectures, and reward functions to capture nuanced user behaviors. Achieving substantial improvements in these areas is a non-trivial task, traditionally relying on extensive manual iterations to test new hypotheses. We propose a self-evolving system that leverages Large Language Models (LLMs), specifically those from Google's Gemini family, to autonomously generate, train, and deploy high-performing, complex model changes within an end-to-end automated workflow. The self-evolving system consists of an Offline Agent (Fast Loop) that performs high-throughput hypothesis generation to optimize for proxy metrics, and an Online Agent (Slow Loop) that validates candidates against delayed north star business metrics in live production. Our agents act as specialized Machine Learning Engineers (MLEs): they exhibit deep reasoning capabilities, discovering novel improvements in optimization algorithms and model architecture, and formulating innovative reward functions that target long-term user engagement. The effectiveness of this approach is demonstrated through several successful production launches at YouTube, confirming that autonomous, LLM-driven evolution can surpass traditional engineering workflows in both development velocity and model performance.

arXiv

Optimized Piecewise Affine Abstractions of Neural Networks with Learnable Activation Functions

Noah Schwartz, Chandra Kanth Nagesh, Sriram Sankaranarayanan, Ramneet Kaur, Tuhin Sahai, Susmit Jha

We present a generalized framework for the range verification of neural networks featuring non-linear activation functions. Our approach first constructs an ``optimized piecewise affine abstraction" of the network that replaces each non-linear activation function by a piecewise affine (PWA) function plus a bounded error. Such PWA functions are readily amenable to existing neural network verification techniques using specializations of linear arithmetic SMT solvers and mixed-integer optimization approaches. However, there are infinitely many ways to abstract each node, with a natural tradeoff between the number of pieces used, the global error bound, and the complexity of the resulting verification problem. We propose a dynamic programming (DP) algorithm to systematically compute the optimized PWA abstraction for general activation functions, guaranteeing tighter output bounds. The algorithm combines a local DP approximation at each node with a global error bound, yielding a variant of the knapsack problem for deciding how to allocate a fixed budget on the total number of pieces across units so as to minimize the worst-case error bound between the network and its approximation. Although the knapsack problem is itself NP-hard, we can use pseudo-polynomial DP algorithms as well as approximation schemes to solve it efficiently. Crucially, our approach is broadly applicable to diverse networks consisting of non-linear activations, including standard Multi-Layer Perceptrons (MLPs) and recently proposed architectures such as Kolmogorov-Arnold Networks (KANs). Over a series of KAN benchmarks spanning 20 to 22,000 parameters, our approach yields output bounds that are consistently of smaller width than uniform PWA allocation. The overall time taken is roughly comparable while the overhead for computing the optimized abstraction is subsumed by the time taken to compute output bounds.

arXiv

Physics-Informed Singular-Value Learning for Cross-Covariances Forecasting in Financial Markets

Efstratios Manolakis, Christian Bongiorno, Rosario Nunzio Mantegna

Recent advances in nonlinear shrinkage yield asymptotically optimal cleaners for large covariance matrices and have been extended to empirical cross-covariances via singular-value shrinkage. However, these approaches rely on stationarity and bounded-spectrum assumptions that are violated by real equity returns, which exhibit dependence drift and macroscopic common modes. We propose a physics-informed neural estimator that parameterizes the cleaned cross-covariance matrix in the empirical singular-vector basis and learns a nonlinear map from empirical singular values and marginal projections to cleaned singular values, recovering the cleaning performances of the analytical solution as a limiting case. On U.S. equity data, the learned correction not only improves out-of-sample cross-covariance prediction but also translates these statistical gains into better tracking-error minimization for portfolio replication. Furthermore, it remains stable in regimes where the analytical cross-covariance estimation deteriorates with universe size, suggesting an interpolation between sample-size denoising and a learned forecast correction under non-stationary dependence.

arXiv

DCC: Data-Centric Compilation of Machine Learning Kernels for Processing-In-Memory Architectures

Peiming Yang, Sankeerth Durvasula, Ivan Fernandez, Mohammad Sadrosadati, Onur Mutlu, Gennady Pekhimenko, Christina Giannoula

High-performance Host processors can integrate Processing-In-Memory (PIM) devices, which can accelerate memory-intensive kernels of Machine Learning (ML) models, including Large Language Models (LLMs), by leveraging the large memory bandwidth available at PIM cores. However, Host processor needs consecutive elements distributed across DRAM banks, while PIM cores need consecutive elements within their local banks. This necessitates data rearrangements in ML kernel execution that pose significant performance and programmability challenges, further exacerbated by the need to support diverse PIM devices. Current compilation approaches lack systematic optimization for diverse ML kernels and multiple PIM devices, and may largely ignore data rearrangement costs during the compute code optimization step. We show that data rearrangements and compute code optimization are interdependent, and need to be jointly optimized during the tuning process. Therefore, we design DCC, the first data-centric ML compiler for PIM systems that jointly co-optimizes data rearrangements and compute code in a unified tuning process. DCC integrates a multi-layer PIM abstraction to support multiple PIM backends. DCC enables effective co-optimization of data partitioning strategies with compute loop partitioning schemes. DCC applies PIM-specific code optimizations, and leverages a fast and accurate performance prediction model to select the bestperforming code schedule for a given kernel on a target PIM architecture. Our evaluations in various individual ML kernels show that DCC achieves up to 7.68x speedup (2.21x average) on HBM-PIM, and up to 13.17x speedup (3.92x average) on AttAcc PIM, over GPU-only execution. In end-to-end LLM inference, DCC on AttAcc accelerates GPT-3 and LLaMA-2 by 4.52x average (up to 7.71x in LLaMA-2) over GPU. DCC is open-sourced at https://github.com/SPIN-Research-Group/DCC.

arXiv

DeepDefense: Robust Learning via Layer-Wise Gradient-Feature Alignment

Ci Lin, Tet Yeap, Iluju Kiringa

Deep neural networks are known to be vulnerable to adversarial perturbations, which are small, carefully crafted inputs that lead to incorrect predictions. In this paper, we propose DeepDefense, a novel defense framework that applies Gradient-Feature Alignment (GFA) regularization across multiple layers to suppress adversarial vulnerability. By aligning input gradients with internal feature representations, DeepDefense promotes a smoother loss landscape in tangential space, also known as feature space, thereby reducing the model's sensitivity to adversarial noise. We provide insights into how adversarial perturbations can be decomposed into radial and tangential components and demonstrate that alignment suppresses loss variation in tangential space, where most attacks are effective. Empirically, our method achieves significant improvements in robustness against both gradient-based and optimization-based attacks. For example, on CIFAR-10, CNN models trained with DeepDefense outperform standard adversarial training by up to 15.2 percent under APGD attacks and 24.7 percent under FGSM attacks. Against optimization-based attacks such as DeepFool and EADEN, DeepDefense requires perturbation magnitudes that are 20 to 30 times larger to cause misclassification, indicating stronger decision boundaries and a flatter loss landscape. Our approach is architecture-agnostic, simple to implement, and highly effective, offering a promising direction for improving the adversarial robustness of deep learning models.

arXiv

Estimating Treatment Effects in Networks under Unknown Exposure Mappings

Daan Caljon, Jente Van Belle, Wouter Verbeke

Estimating heterogeneous treatment effects in network settings is complicated by interference, meaning that the outcome of an instance can be influenced by the treatment status of others. Existing causal machine learning approaches that account for interference usually rely on a prespecified exposure mapping that summarizes how others' treatments affect the outcome of a given instance, a simplification that is often inappropriate. We propose HINet, a neural method that combines an expressive GNN outcome model with network-aware domain-adversarial training. The outcome model learns an exposure-relevant neighborhood representation jointly with outcome prediction, allowing HINet to capture heterogeneous interference without prespecifying an exposure mapping. HINet's adversarial component uses both node and neighborhood information to promote balance in the learned representations with respect to treatment assignment. We further derive a population-level generalization bound and introduce two metrics for evaluation across counterfactual networks. An empirical evaluation on synthetic and semi-synthetic network datasets demonstrates that HINet performs consistently across diverse exposure mappings, while methods based on a prespecified mapping can perform poorly when it is misspecified.

arXiv

Learned Digital Over-the-Air Computing for Federated Edge Learning

Antonio Tarizzo, Mohammad Kazemi, Deniz Gündüz

Over-the-air (OTA) aggregation enables federated edge learning (FEEL) by exploiting the superposition property of the wireless channel to merge communication with computation, eliminating the need to schedule and decode devices individually. Analog OTA schemes transmit uncoded updates but are sensitive to noise, fading, and power misalignment, motivating more robust digital alternatives. However, state-of-the-art (SoTA) digital OTA designs that combine unsourced random access (URA) with compressed sensing struggle in the low signal-to-noise ratio (SNR) regimes common in Internet of Things (IoT) deployments, where symbol recovery and active-device estimation become unreliable. We propose a learned digital OTA framework that jointly trains a URA codebook with an unrolled approximate message passing (AMP)-based decoder. The learned decoder incorporates per-layer damping, residual scaling, temperature-controlled Bayesian denoising, and a lightweight convolutional neural network (CNN) refinement, while the codebook is optimised end-to-end through a factorised parameterisation. At near-perfect-aggregation accuracy, the proposed design extends the viable SNR range by approximately 7\,dB over the SoTA baseline at the same uplink overhead, and generalises across models, activity levels, and heterogeneous data.

arXiv

Conditional Deep Levy Models for Exotic Derivatives: History-Aware Path Generation and P-Q Payoff Diagnostics

Helin Zhao, Junchi Shen

We develop and audit a history-aware financial path generator based on Denoising Levy Probabilistic Models (DLPMs) for conditional equity-index path generation. The model combines symmetric alpha-stable diffusion noise with a conditional U-Net observing the contract state, 60- and 252-day return histories, and pre-start trend, drawdown, and volatility state. A chronological protocol evaluates one frozen model on 6,824 untouched windows from eight Chinese and U.S. equity indices. The generator attains terminal CRPS 0.0485, path energy score 0.3368, and a generated-to-realized volatility ratio of 0.882, improving on contract-only DLPM and Gaussian diffusion controls. An unconditional historical block bootstrap remains competitive on pooled marginal metrics; against a stronger state-matched empirical control using the same index, tenor, volatility, trend, and drawdown information, DLPM lowers terminal CRPS from 0.0633 to 0.0485 (23.3%), with joint calendar-block confidence intervals below zero from 20 to 150 days. We then embed the learned physical-measure distribution in a transparent P-Q payoff diagnostic against an independently fitted, martingale-corrected Student-t GARCH(1,1) benchmark. At a pre-specified 0.05 dealer spread and a 1%-of-spot materiality threshold, the aligned unconditional diagnostic is 0.91% of initial spot for vanilla calls and 0.36% for Asian calls. Drift-neutral and one-time-delta counterfactuals decompose this gap: physical direction is a large component, leaving a 0.33% vanilla residual and a near-zero Asian residual. The P-Q framework is a belief-based payoff diagnostic, distinct from an option-surface-calibrated pricing engine.

arXiv

Self-composing neural operators for high-frequency and multiscale PDE surrogates

Juncai He, Xinliang Liu, Jinchao Xu

Addressing the computational challenges of high-frequency and multiscale partial differential equations (PDEs), this work introduces a self-composing neural operator (SC-NO) framework. Inspired by classical fixed-point iterative solvers (e.g., multigrid, domain decomposition), the proposed architecture constructs a deep operator by repeatedly applying a single, parameter-efficient backbone block. This design mimics the update step of a numerical solver, allowing the model to progressively resolve complex solution features without increasing the parameter count. For practical training, we develop an adaptive ``Train-and-Unroll'' strategy that grows the composition depth during training, acting as a curriculum from shallow to deep self-composed models. We demonstrate the efficacy of this framework on the Helmholtz equation for ultrasound computed tomography (USCT), a problem characterized by high-frequency wave propagation in highly heterogeneous media. By instantiating the backbone with a multigrid-inspired architecture, the SC-NO effectively mitigates the spectral bias often observed in standard operator learning baselines. Numerical experiments show that our method reduces the prediction error significantly compared to Fourier Neural Operators (FNO) and their variants in the 300--500 kHz regime. Furthermore, we provide theoretical analysis linking the self-composition depth to approximation accuracy.

arXiv

Learning Graph-Indexed Trajectory Patterns for Stochastic On-Time Arrival Routing

Yuanhang Wang, Xing Wei, Duoxiang Zhao, Zezhou Zhang, Hao Qin, Yuqi Ouyang

Correlated link travel times create decision-relevant patterns in partial route histories. In stochastic on-time arrival (SOTA) routing, each route prefix forms a variable-length, graph-indexed sequence in which traversed-edge identities, realized travel times, and route order jointly indicate the reliability of downstream actions. We present GPG-HT, a history-conditioned Transformer policy that learns a trajectory representation from this structured sequence together with the current node, destination, and remaining budget. Edge-time cross-attention and sequence encoding capture dependencies within the observed history, while decoder cross-attention maps the resulting trajectory memory and decision context to an online distribution over feasible outgoing edges. A history-conditioned generalized policy-gradient objective trains the representation from terminal on-time outcomes. Experiments on the Sioux Falls and Anaheim road-network topologies with simulated correlated link times show that GPG-HT achieves higher mean on-time arrival probabilities than representative optimization and reinforcement-learning baselines. Paired common-pool evaluation confirms statistically significant gains in all six network-budget settings, reaching 2.82-3.27 percentage points on Sioux Falls and 0.36-1.04 percentage points on Anaheim. Correlated, independent, shuffled-history, no-history, and architecture controls further demonstrate that GPG-HT learns decision-relevant structure from graph-indexed route prefixes.

arXiv

Machine-Precision Prediction of Low-Dimensional Chaotic Systems from Noise-Free Data

Christof Schötz, Niklas Boers

Low-dimensional chaotic systems such as the Lorenz-63 model are commonly used to benchmark system-agnostic methods for learning dynamics from data. This study shows that learning from noise-free observations in such systems can be achieved up to machine precision: using ordinary least squares regression on high-degree polynomial features with 512-bit arithmetic, a system-agnostic method is introduced that matches the accuracy of standard 64-bit numerical ODE solvers using the systems' governing equations. For the Lorenz-63 system, the method achieves valid prediction times of 36 Lyapunov times, and even up to 105 Lyapunov times with favorable precision configurations, dramatically outperforming prior work, which reaches 13 Lyapunov times at most. The results are further validated on Thomas' Cyclically Symmetric Attractor, a non-polynomial chaotic system that is considerably more complex than the Lorenz-63 model, and similar results extend to higher dimensions using the spatiotemporally chaotic Lorenz-96 model. These findings suggest that forecasting low-dimensional chaotic systems from noise-free data is effectively a solved problem.

arXiv

ARMOR: Robust Reinforcement Learning-based Control for UAVs under Physical Attacks

Pritam Dash, Ethan Chan, Nathan P. Lawrence, Karthik Pattabiraman

Unmanned Aerial Vehicles (UAVs) depend on onboard sensors for perception, navigation, and control. However, these sensors are susceptible to physical attacks, such as GPS spoofing, that can corrupt state estimates and lead to unsafe behavior. While reinforcement learning (RL) offers adaptive control capabilities, existing safe RL methods are ineffective against such attacks. We present ARMOR (Adaptive Robust Manipulation-Optimized State Representations), an attack-resilient, model-free RL controller that enables robust UAV operation under adversarial sensor manipulation. Instead of relying on raw sensor observations, ARMOR learns a robust latent representation of the UAV's physical state via a two-stage training framework. In the first stage, a teacher encoder, trained with privileged attack information, generates attack-aware latent states for RL policy training. In the second stage, a student encoder is trained via supervised learning to approximate the teacher's latent states using only historical sensor data, enabling real-world deployment without privileged information. Our experiments show that ARMOR outperforms conventional methods, ensuring UAV safety. Additionally, ARMOR improves generalization to unseen attacks and reduces training cost by eliminating the need for iterative adversarial training.

arXiv

Sampling Decisions: Exact Path-Space Control for Physics-Informed Generative Sampling

Michael Chertkov, Hamidreza Behjoo, Sungsoo Ahn

Scientific generative models must turn tractable local decisions into globally correlated samples that respect physical constraints. We introduce Sampling Decisions, a finite-horizon framework in which a structured object is assembled on a growing state graph and corrected globally by an exact path-space control law. For a prescribed Gibbs target, the corrected law is the unique relative-entropy projection of a sequential prior and is realized by a Doob h-transform with a linear backward desirability recursion. The same object admits equivalent interpretations as a KL-optimal controller, a one-sided Schrodinger transport, and an ideal value or flow function for autoregressive and GFlowNet-type generation. A route-resolved formulation yields a finite-particle algorithm, whose transition kernels and terminal law converge as the path budget grows. For binary graphical models, we prove a structural cancellation theorem: every fixed singleton-product prior disappears from the population correction, so improved one-point marginals do not alter the exact generative dynamics. The relevant information is conditional and prefix dependent. Statistical physics supplies this structure through a Local-Boltzmann prior that absorbs interactions as spins are revealed, while optimal path-space control supplies the missing look-ahead field from the unrevealed subgraph. On exactly enumerable Ising grids, this physics-informed proposal increases effective sample size by factors of about ten to nearly one thousand relative to product proposals and reaches the exact-target reference band at the tested budgets. On a larger 10*10 grid, beyond exact enumeration, the same hierarchy persists: Local-Boltzmann guidance avoids the severe weight collapse of product proposals and approaches a long-run MCMC baseline on the reported diagnostics.

arXiv

Defending Membership Inference Attacks via Privacy-aware Sparsity Tuning

Hengxiang Zhang, Qiang Hu, Hongxin Wei

Over-parameterized models are typically vulnerable to membership inference attacks, which aim to determine whether a specific sample is included in the training of a given model. Previous Weight regularizations (e.g., L1 regularization) typically impose uniform penalties on all parameters, leading to a suboptimal tradeoff between model utility and privacy. In this work, we first show that only a small fraction of parameters substantially impact the privacy risk. In light of this, we propose Privacy-aware Sparsity Tuning (PAST), a simple fix to the L1 Regularization, by employing adaptive penalties to different parameters. Our key idea behind PAST is to promote sparsity in parameters that significantly contribute to privacy leakage. In particular, we construct the adaptive weight for each parameter based on its privacy sensitivity, i.e., the gradient of the loss gap with respect to the parameter. Using PAST, the network shrinks the loss gap between members and non-members, leading to strong resistance to privacy attacks. Extensive experiments demonstrate the superiority of PAST, achieving a state-of-the-art balance in the privacy-utility trade-off.

arXiv

Information-Theoretic Foundations for Machine Learning

Hong Jun Jeon, Benjamin Van Roy

The progress of machine learning over the past decade is undeniable. In retrospect, it is both remarkable and unsettling that this progress was achievable with little to no rigorous theory to guide experimentation. Despite this fact, practitioners have been able to guide their future experimentation via observations from previous large-scale empirical investigations. In this work, we propose a theoretical framework which attempts to provide rigor to existing practices in machine learning. To the theorist, we provide a framework which is mathematically rigorous and leaves open many interesting ideas for future exploration. To the practitioner, we provide a framework whose results are simple, and provide intuition to guide future investigations across a wide range of learning paradigms. Concretely, we provide a theoretical framework rooted in Bayesian statistics and Shannon's information theory which is general enough to unify the analysis of many phenomena in machine learning. Our framework characterizes the performance of an optimal Bayesian learner as it learns from a stream of experience. Unlike existing analyses that weaken with increasing data complexity, our theoretical tools provide accurate insights across diverse machine learning settings. Throughout this work, we derive theoretical results and demonstrate their generality by apply them to derive insights specific to settings. These settings range from learning from data which is independently and identically distributed under an unknown distribution, to data which is sequential, to data which exhibits hierarchical structure amenable to meta-learning, and finally to data which is not fully explainable under the learner's beliefs (misspecification). These results are particularly relevant as we strive to understand and overcome increasingly difficult machine learning challenges in this endlessly complex world.

Keep browsing

More papers from top AI conferences

30+ top conferences, by conference, year, and topic.