ACML 2022 论文列表

Learning to control from complex observations remains a major challenge in the application of model-based reinforcement learning (MBRL). Existing MBRL methods apply contrastive learning to replace pixel-level reconstruction, improving the performance of the latent world model. However, previous contrastive learning approaches in MBRL fail to utilize task-relevant information, making it difficult to aggregate observations with the same task-relevant information but the different task-irrelevant information in latent space. In this work, we first propose Constrained Contrastive Reinforcement Learning (C2RL), an MBRL method that learns a world model through a combination of two contrastive losses based on latent dynamics and task-relevant state abstraction respectively, utilizing reward information to accelerate model learning. Then, we propose a hyperparameter $\beta$ to balance two kinds of contrastive losses to strengthen the representation ability of the latent dynamics. The experimental results show that our approach outperforms state-of-the-art methods in both the natural video and standard background setting on challenging DMControl tasks.

Multimodal sentiment analysis (MSA) is gaining traction as a critical tool for understanding human behavior and enabling a wide range of applications. Since data of different modalities might lie in completely distinct spaces, it is very challenging to perform effective fusion and analysis from asynchronous multimodal streams. Most of previous works focused on aligned fusion, which is unpractical in real-world scenarios. The recent Multimodal Transformer (MulT) approach attends to model the correlations between elements from different modalities in an unaligned manner. However, it collects temporal information by self-attention transformer which is a sequence model, implying that interactions across distinct time steps are not sufficient. In this paper, we propose the Citculant-interactive Transformer Network with dimension-aware fusion (CITN-DAF), which enables parallel computation of different modalities among different time steps and alleviates inter-modal temporal sensitivity while preserving intra-modal semantic order. By incorporating circulant matrices into the cross-modal attention mechanism, CITN-DAF is aimed to examine all conceivable interactions between vectors of different modalities. In addition, a dimension-aware fusion method is presented, which projects feature representations into different subspaces for an in-depth fusion. We evaluate CITN-DAF on three commonly used sentiment analysis benchmarks including CMU-MOSEI, CMU-MOSI and IEMOCAP. Extensive experimental results reveal that CITN-DAF is superior in cross-modal semantic interactions and outperforms the state-of-the-art multimodal methods.

In contrast to risk-averse multi-armed bandit (MAB), where one aims for a best risk-sensitive arm while having a risk-neutral attitude when running the risk-averse MAB algorithm, in this paper, we aim for a best arm with respect to the mean like in the standard MAB, but we adopt a risk-averse attitude when running a standard MAB algorithm. Conditional value-at-risk (CVaR) of the regret is adopted as the metric to evaluate the performance of algorithms, which is an extension of the traditional expected regret minimization framework. For this new problem, we revisit several classic algorithms for stochastic and non-stochastic bandits, UCB, MOSS, and Exp3-IX with its variants and propose parameters with good theoretically guaranteed CVaR-regret, which match the results of the expected regret and achieve (nearly-)optimality up to constant. In the non-stochastic setting, we show that implicit exploration achieves a trade-off between the variability of the regret and the regret in expectation. Numerical experiments are conducted to validate our results.

Vision-language models have recently made impressive strides, primarily driven by large-scale training on web data. While pioneering works such as CLIP and ALIGN show significant improvements, these are focused on English data as it is easy to source them from the web. Towards serving non-English-speaking demographics, we consider various methods for generating multilingual data and find that a simple bootstrapping mechanism works surprisingly well. Specifically, just using English image captions data and text-only multilingual translation pairs we train a fairly strong multilingual vision-language model and then leverage it to create a much cleaner version of the multilingual image captions dataset we collected. We demonstrate that this dataset which was used to train Bletchley result in a strong multi-modal and multilingual model which reaches strong performance across several multilingual zero-shot tasks. Specifically, Bletchley achieves state-of-the-art results on multilingual COCO, Multi30k sets, IGLUE WIT and xFlickr&CO datasets.

With the global metamorphosis of the beauty industry and the rising demand for beauty products worldwide, the need for a robust makeup recommendation system has never been more. Despite the significant advancements made towards personalised makeup recommendation, the current research still falls short of incorporating the context of occasion and integrating feedback for users. In this work, we propose BeautifAI, a novel recommendation system, delivering personalised occasion-oriented makeup recommendations to users. The proposed work’s novel contributions, including incorporating occasion context to makeup recommendation and a region-wise method using neural embeddings, set our system apart from the current work in makeup recommendation. We also propose real-time makeup previews and continuous makeup feedback to provide a more personalised and interactive experience to users.

The stochastic multi-armed bandit problem has been widely studied under the stationary assumption. However in real world problems and industrial applications, this assumption is often unrealistic because the distributions of rewards may change over time. In this paper, we consider the piece-wise iid non-stationary stochastic multi-armed bandit problem with unknown change-points and we focus on the change of mean setup. To solve the latter, we propose a change-point based framework where we study a class of change-detection based optimal bandit policies that actively detects change-point using the restarted Bayesian online change-point detector and then restarts the bandit indices. Analytically, in the context of regret minimization, our proposal achieves a $\mathcal{O}(\sqrt{A T K_T })$ regret upper-bound where $K_T$ is the overall number of change-points up to the horizon $T$ and $A$ is the number of arms. The derived bound matches the existing lower bound for abruptly changing environments. Finally, we demonstrate the cumulative regret reduction of the our proposal over synthetic Bernoulli rewards as well as Yahoo! datasets of webpage click-through rates.

Despite their theoretical appealingness, Bayesian neural networks (BNNs) are left behind in real-world adoption, mainly due to persistent concerns on their scalability, accessibility, and reliability. In this work, we develop the BayesAdapter framework to relieve these concerns. In particular, we propose to adapt pre-trained deterministic NNs to be variational BNNs via cost-effective Bayesian fine-tuning. Technically, we develop a modularized implementation for the learning of variational BNNs, and refurbish the generally applicable exemplar reparameterization trick through exemplar parallelization to efficiently reduce the gradient variance in stochastic variational inference. Based on the the lightweight Bayesian learning paradigm, we conduct extensive experiments on a variety of benchmarks, and show that our method can consistently induce posteriors with higher quality than competitive baselines, yet significantly reducing training overheads.

Accurate forecasting of multivariate time series is an extensively studied subject in finance, transportation, and computer science. Fully mining the correlation and causation between the variables in a multivariate time series exhibits noticeable results in improving the performance of a time series model. Recently, some models have explored the dependencies between variables through end-to-end graph structure learning without the need for predefined graphs. However, current models do not incorporate the trade-off between efficiency and flexibility and make insufficient use of the information contained in time series in the design of graph structure learning algorithms. This paper alleviates the above issues by proposing Balanced Graph Structure Learning for Forecasting (BGSLF), a novel and effective deep learning model that joins graph structure learning and forecasting. Technically, BGSLF leverages the spatial information into convolutional operations and extracts temporal dynamics using the diffusion convolutional recurrent network. The proposed framework emphasizes the trade-off between efficiency and flexibility by introducing Multi-Graph Generation Network (MGN) and Graph Selection Module. In addition, a method named Smooth Sparse Unit (SSU) is designed to sparse the learned graph structures, which conforms to the sparse spatial correlations in the real world. Extensive experiments on four real-world datasets demonstrate that our model achieves state-of-the-art performances with minor trainable parameters. Our code is publicly available at https://github.com/onceCWJ/BGSLF.

Realistic use of neural networks often requires adhering to multiple constraints on latency, energy and memory among others. A popular approach to find fitting networks is through constrained Neural Architecture Search (NAS). However, previous methods use complicated predictors for the accuracy of the network. Those predictors are hard to interpret and sensitive to many hyperparameters to be tuned, hence, the resulting accuracy of the generated models is often harmed. In this work we resolve this by introducing Bilinear Interpretable Neural Architecture Search (BINAS), that is based on an accurate and simple bilinear formulation of both an accuracy estimator and the expected resource requirement, together with a scalable search method with theoretical guarantees. The simplicity of our proposed estimator together with the intuitive way it is constructed bring interpretability through many insights about the contribution of different design choices. For example, we find that in the examined search space, adding depth and width is more effective at deeper stages of the network and at the beginning of each resolution stage. Our experiments show that BINAS generates comparable to or better architectures than other state-of-the-art NAS methods within a reduced search cost for each additional generated network, while strictly satisfying the resource constraints.

Myocardial Infarctions (MI) or heart attacks are among the most common medical emergencies globally. Such an episode often has mild or varied symptoms, making it hard to diagnose and respond in a timely manner. An electrocardiogram (ECG) is used to analyze the heart’s electrical activity and, through this help, clinicians detect and localize a heart attack. However, interpretation of the ECG is made manually by trained professionals. In order to make this diagnosis more efficient, multiple methods have tried to automate the MI detection and localization process. In this work, we aim to create a more effective method of MI detection by restructuring the localization as a multi-label classification (MLC) problem, in which one set of attributes can belong to one or more classes. For this classification, features like the ST-deviation, T wave amplitude, and R-S ratios have been extracted and fed into the MLC model, which in our case, is a chain classifier of random forest. This proposed model will have five classes as the target, which represent the locations where an MI can occur. Our method achieves the best overall hamming accuracy of 81.49% in a k-fold cross validation test, with the highest accuracy for an individual class being 97.72% for anterior.

With the Internet of Everything (IoE) nowadays, monitoring edge systems is essential for coordinating everything into an IoE web. However, it is hard to monitor edge systems due to limited system information and limited sensors. To infer system information and provide robust monitoring capability, machine learning models were used to approximate mapping rules between different measurements. However, mapping rule learning using traditional machine learning tools is one way only, e.g., from measurement variables to the state vector variables. And, it is hard to be reverted, leading to over-fitting because of inconsistency between the forward and inverse learnings. Hence, we propose a structural deep neural network framework to provide a coherent two-way functional approximation. For physical regularization, we embed network size into the number of variables in the latent layers. We also utilize state sensors in the ‘latent layer’ to guide other latent variables to create state sets. The performance of reconstruction for the two-way mapping rule is validated extensively using test cases in the engineering, physics, and mathematical analysis domain.

To provide intelligent and personalized models for clients, personalized federated learning (PFL) enables learning from data, identifying patterns, and making automated decisions in a privacy-preserving manner. PFL involves independent training for multiple clients with synchronous aggregation steps. However, the assumptions made by existing works are not realistic given the heterogeneity of clients. In particular, the volume and distribution of collected data vary in the training process, and the clients also vary in their available system configurations, which leads to vast heterogeneity in the system. To address these challenges, we present an \textit{asynchronous} method (AsyPFL), where clients learn personalized models w.r.t. local data by making the most informative parameters less volatile. The central server aggregates model parameters asynchronously. In addition, we also reformulate PFL by unifying both synchronous and asynchronous updating schemes with an asynchrony-related parameter. Theoretically, we show that AsyPFL’s convergence rate is state-of-the-art and provide guarantees of choosing key hyperparameters optimally. With these theoretical guarantees, we validate AsyPFL on different tasks with non-IID and staleness settings. The results indicate that, given a large proportion of irregular clients, AsyPFL excels at empirical performance compared with vanilla PFL algorithms on non-IID and IID cases.

Human activity recognition (HAR) attracts widespread attention from researchers recently, and deep learning is employed as a dominant paradigm of solving HAR problems. The previous techniques rely on domain knowledge or attention mechanism extract long-range dependency in temporal dimension and cross channel correlation in sensor’s channel dimension. In this paper, a HAR model with positional attention (PA), termed as PA-HAR, is presented. To enhance the features in both sensor’s channel and temporal dimensions, we propose to split the sensor signals into two 1D features to capture the long-range dependency along the temporal-axis and signal’s cross-channel information along the sensor’s channel-axis. Furthermore, we embed the features with positional information by encoding the generated features into pairs of temporal-aware and sensor’s channel-aware attention maps and weighting the input feature maps. Extensive experiments based on five public datasets demonstrate that the proposed PA-HAR algorithm achieves a competitive performance in HAR related tasks compared with the state-of-the-art approaches.

Most existing deep neural networks (DNNs) are easily disturbed by slight noise. However, there are few researches on physical attacks by deploying lighting equipment. The light-based physical attacks has excellent covertness, which brings great security risks to many vision-based applications (such as self-driving). Therefore, we propose a light-based physical attack, called adversarial laser spot (AdvLS), which optimizes the physical parameters of laser spots through genetic algorithm to perform physical attacks. It realizes robust and covert physical attack by using low-cost laser equipment. As far as we know, AdvLS is the first light-based physical attack that perform physical attacks in the daytime. A large number of experiments in the digital and physical environments show that AdvLS has excellent robustness and covertness. In addition, through in-depth analysis of the experimental data, we find that the adversarial perturbations generated by AdvLS have superior adversarial attack migration. The experimental results show that AdvLS impose serious interference to advanced DNNs, we call for the attention of the proposed AdvLS.

Recently, introspective models like IntroVAE and S-IntroVAE have excelled in image generation and reconstruction tasks. The principal characteristic of introspective models is the adversarial learning of VAE, where the encoder attempts to distinguish between the real and the fake (i.e., synthesized) images. However, due to the unavailability of an effective metric to evaluate the difference between the real and the fake images, the posterior collapse and the vanishing gradient problem still exist, reducing the fidelity of the synthesized images. In this paper, we propose a new variation of IntroVAE called Adversarial Similarity Distance Introspective Variational Autoencoder (AS-IntroVAE). We theoretically analyze the vanishing gradient problem and construct a new Adversarial Similarity Distance (AS-Distance) using the 2-Wasserstein distance and the kernel trick. With weight annealing on AS-Distance and KL-Divergence, the AS-IntroVAE are able to generate stable and high-quality images. The posterior collapse problem is addressed by making per-batch attempts to transform the image so that it better fits the prior distribution in the latent space. Compared with the per-image approach, this strategy fosters more diverse distributions in the latent space, allowing our model to produce images of great diversity. Comprehensive experiments on benchmark datasets demonstrate the effectiveness of AS-IntroVAE on image generation and reconstruction tasks.

Deducing the contribution of each agent and assigning the corresponding reward to them is a crucial problem in cooperative Multi-Agent Reinforcement Learning (MARL). Previous studies try to resolve the issue through designing an intrinsic reward function, but the intrinsic reward is simply combined with the environment reward by summation in these studies, which makes the performance of their MARL framework unsatisfactory. We propose a novel method named Attention Individual Intrinsic Reward Mixing Network (AIIR-MIX) in MARL, and the contributions of AIIR-MIX are listed as follows: \textbf{(a)} we construct a novel intrinsic reward network based on the attention mechanism to make teamwork more effective. \textbf{(b)} we propose a Mixing network that is able to combine intrinsic and extrinsic rewards non-linearly and dynamically in response to changing conditions of the environment. We compare AIIR-MIX with many State-Of-The-Art (SOTA) MARL methods on battle games in StarCraft II. And the results demonstrate that AIIR-MIX performs admirably and can defeat the current advanced methods on average test win rate. To validate the effectiveness of AIIR-MIX, we conduct additional ablation studies. The results show that AIIR-MIX can dynamically assign each agent a real-time intrinsic reward in accordance with their actual contribution.

Recurrent Neural Networks are an integral part of modern machine learning. They are good at performing tasks on sequential data. However, long sequences are still a problem for those models due to the well-known exploding/vanishing gradient problem. In this work, we build on recent approaches to interpreting the gradient problem as instability of the underlying dynamical system. We extend previous approaches to systems with top-down feedback, which is abundant in biological neural networks. We prove that the resulting system is stable for arbitrary depth and width and confirm this empirically. We further show that its performance is on par with LSTM and related approaches on standard benchmarks.

ESC (Environmental Sound Classification) is an active area of research in the field of audio classification that has made significant progress in recent years. The current mainstream ESC methods are based on increasing the dimension of the extracted audio features and therefore draw on the two-dimensional convolution methods used in image processing. However, two-dimensional convolution is expensive to train and the complexity of the corresponding model is usually very high. In response to these issues, we propose a novel two-stream neural network model by the idea of disentanglement, which uses onedimensional convolution for feature extraction to disentangle the audio features into the time and frequency domains separately. Our approach balances computational pressure with classification accuracy well. The accuracy of our approach on the Urbansound 8k and Esc-10 datasets was 98.51% and 97.50%, respectively, which exceeds that of most models. Meanwhile, the model complexity is also lower.

In the past few years, supervised-based deep learning methods has yielded good results in skin lesions diagnosis tasks. Unfortunately, obtaining large of labels for medical images is expensive and time consuming. In this paper, we propose a self-improving skin lesions diagnosis (SISLD) framework to explore useful information in unlabeled data. We first propose a semi-supervised model ${f}$, which combining consistency and class-balanced pseudo-labeling to make full use of unlabeled data in scenarios with sparse manually labeled samples, and obtain a teacher model ${f_{t}}$ by semi-supervised self-training. Then, we introduce self-distillation method to enable knowledge distillation for the diagnosis of skin lesions. Finally, we measure diagnostic effectiveness in the context of label sparsity and class imbalance. The experiments on skin lesion images dataset ISIC2018 shows that SISLD achieves significant improvements in AUC, Accuracy, Specificity and Sensitivity.