INFOCOM 2024 Papers

IEEE International Conference on Computer Communications

Decision breakdown

256papers

Counted from the official paper list; may differ slightly from the number the conference announced.

  • Other / unlabeled
    256100.0%

Top 10 topics (click to filter)

No topic data yet.

Trends for all 207 research topics

Browse all papers

Scheduling Stochastic Traffic With End-to-End Deadlines in Multi-hop Wireless Networks

Christos Tsanikidis, Javad Ghaderi

Expand abstract

Scheduling deadline-constrained packets in multihop networks has received increased attention recently. However, there is very limited work on this problem for wireless networks where links are subject to interference. The existing algorithms either provide approximation ratio guarantees which diminish in quality as parameters of the network scale, or hold in an asymptotic regime when the time horizon, network bandwidth, and packet arrival rates are scaled to infinity, which limits their practicality. While attaining a constant approximation ratio has been shown to be impossible in the worst-case traffic setting, it is unclear if the same holds under stochastic traffic, in a non-asymptotic setting. In this work, we show that, in the stochastic traffic setting, constant approximation ratio or near-optimal algorithms can be achieved. Specifically, we propose algorithms that attain Ω((1 − ϵ)/β) or Ω(1 − ϵ) fraction of the optimal value, when the number of channels is ${\text{C}} = \Omega \left( {\frac{{\log (L/\varepsilon )}}{{{\varepsilon ^2}}}} \right)$ or ${\text{C}} = \Omega \left( {\frac{{{\chi ^ \star }\log (L/\varepsilon )}}{{{\varepsilon ^3}}}} \right)$respectively, where L is the maximum route length of packets, χ⋆is the fractional chromatic number of the network’s interference graph, and β is its interference degree. This marks the first near-optimal results under nontrivial traffic and bandwidth assumptions in a non-asymptotic regime.

DPBalance: Efficient and Fair Privacy Budget Scheduling for Federated Learning as a Service

Yu Liu, Zibo Wang, Yifei Zhu, Chen Chen

Expand abstract

Federated learning (FL) has emerged as a prevalent distributed machine learning scheme that enables collaborative model training without aggregating raw data. Cloud service providers further embrace Federated Learning as a Service (FLaaS), allowing data analysts to execute their FL training pipelines over differentially-protected data. Due to the intrinsic properties of differential privacy, the enforced privacy level on data blocks can be viewed as a privacy budget that requires careful scheduling to cater to diverse training pipelines. Existing privacy budget scheduling studies prioritize either efficiency or fairness individually. In this paper, we propose DPBalance, a novel privacy budget scheduling mechanism that jointly optimizes both efficiency and fairness. We first develop a comprehensive utility function incorporating data analyst-level dominant shares and FL-specific performance metrics. A sequential allocation mechanism is then designed using the Lagrange multiplier method and effective greedy heuristics. We theoretically prove that DPBalance satisfies Pareto Efficiency, Sharing Incentive, Envy-Freeness, and Weak Strategy Proofness. We also theoretically prove the existence of a fairness-efficiency tradeoff in privacy budgeting. Extensive experiments demonstrate that DPBalance outperforms state-of-the-art solutions, achieving an average efficiency improvement of 1.44× ~ 3.49×, and an average fairness improvement of 1.37×~24.32×.

Silent Observers Make a Difference: A Large-scale Analysis of Transparent Proxies on the Internet

Rui Bian, Lin Jin, Shuai Hao, Haining Wang, Chase Cotton

Expand abstract

Transparent web proxies have been widely deployed on the Internet, bridging the communications between clients and servers and providing desirable benefits to both sides, such as load balancing, security monitoring, and privacy enhancement. Meanwhile, they work silently as clients and servers may not be aware of their existence. However, due to their invisibility and stealthiness, transparent proxies remain understudied for their behaviors, suspicious activities, and potential vulnerabilities that could be exploited by attackers. To better understand transparent proxies, we design and develop a framework to systematically investigate them in the wild. We identify two major types of transparent web proxies, named FDR and CPV, respectively. FDR is a type of transparent proxy that independently performs Forced DNS Resolution during interception. CPV is a type of transparent proxy that presents Cache Poisoning Vulnerability. We perform a large-scale measurement to detect each type of transparent web proxy and scrutinize their security implications. In total, we observe 32,246 FDR and 11,286 CPV cases through our acquired vantage points. We confirm that these two types of transparent proxies are distributed globally — FDRs are observed in 98 countries and CPVs are observed in 51 countries. Our work highlights the issues of vulnerable transparent proxies and provides insights for mitigating such problems.

Your Mega-Constellations Can Be Slim: A Cost-Effective Approach for Constructing Survivable and Performant LEO Satellite Networks

Zeqi Lai, Yibo Wang, Hewu Li, Qian Wu, Qi Zhang, Yunan Hou, Jun Liu, Yuanjie Li

Expand abstract

Recently we have witnessed the active deployment of mega-constellations with hundreds to thousands of low-earth orbit (LEO) satellites, targeting at constructing LEO satellite networks (LSN) to provide ubiquitous Internet services globally. However, while the massive deployment of LEO satellites can improve the network survivability and performance of an LSN, it also involves additional sustainable challenges such as higher deployment cost, risk of satellite conjunction and space debris.In this paper, we investigate an important research problem facing the upcoming satellite Internet: from a network perspective, how many satellites exactly do we need to construct a survivable and performant LSN? To answer this question, we first formulate the survivable and performant LSN design (SPLD) problem, which aims to find the minimum number of needed satellites to construct an LSN that can provide sufficient amount of redundant paths, required link capacity and acceptable latency for traffic carried by the LSN. Second, to efficiently solve the tricky SPLD problem, we propose MegaReduce, a requirement-driven constellation optimization mechanism, which can calculate feasible solutions for SPLD in polynomial time. Finally, we conduct extensive trace-driven simulations to verify MegaReduce’s cost-effectiveness in constructing survivable and performant LSNs on demand, and showcase how MegaReduce can help optimize the incremental deployment and long-term maintenance of future satellite Internet.

Dynamic Learning-based Link Restoration in Traffic Engineering with Archie

Wenlong Ding, Hong Xu

Expand abstract

Fiber cuts reduce network capacity and take a long time to fix in optical wide-area networks. It is important to select the best restoration plan that minimizes throughput loss by reconfiguring wavelengths on remaining healthy fibers for affected IP links. Recent work studies optimal restoration plan or ticket selection problem in traffic engineering (TE) in a one-shot setting of only one TE interval (5 minutes). Since fiber repair often takes hours, in this work, we extend to consider restoration ticket selection with traffic dynamics over multiple intervals.To balance restoration performance with reconfiguration overhead, we perform dynamic ticket selection every T time steps. We propose an end-to-end learning approach to solve this T-step ticket selection problem as a classification task, combining traffic trend extraction and ticket selection in the same learning model. It uses convolution LSTM network to extract temporal and spatial features from past demand matrices to determine the ticket most likely to perform well T steps down the road, without predicting future traffic or solving any TE optimization. Trace-driven simulation shows that our new TE system, Archie, reduces over 25% throughput loss and is over 3500x faster than conventional demand prediction approach, which requires solving TE many times.

Learning-based Scheduling for Information Gathering with QoS Constraints

Qingsong Liu, Weihang Xu, Zhixuan Fang

Expand abstract

The problem of scheduling packets from multiple sources over unreliable channels has attracted much attention due to its great practicability in the Internet of things systems. Most previous work focuses on the throughput/energy consumption/operational cost optimization or the setting that the channel information is known a priori. In this paper, we consider a more generic setting to this problem where packets from different sources have different values, and each heterogeneous source has a distinct Quality of Service (QoS) requirement. The information about packet value and channel reliability is unknown in advance, and the controller schedules sources over time to maximize its collected packet values while providing a QoS guarantee for each source. For the stationary case where packet values are independent and identically distributed (i.i.d.), we propose an efficient learning policy based on linear-programming (LP) methodology. Our proof shows that it meets the QoS constraint of each source and only incurs a logarithmic regret. In the special case that the channel reliability is known a priori, our algorithm can further guarantee a bounded regret. Furthermore, in the case of non-stationary packet values, we apply the sliding window technique to our LP-based algorithm and prove that it still guarantees a sublinear regret while meeting each source’s QoS requirement. Finally, we provide numerical simulations to support our theoretical results.

OREO: O-RAN intElligence Orchestration of xApp-based network services

Federico Mungari, Corrado Puligheddu, Andres Garcia-Saavedra, Carla Fabiana Chiasserini

Expand abstract

The Open Radio Access Network (O-RAN) architecture aims to support a plethora of network services, such as beam management and network slicing, through the use of third-party applications called xApps. To efficiently provide network services at the radio interface, it is thus essential that the deployment of the xApps is carefully orchestrated. In this paper, we introduce OREO, an O-RAN xApp orchestrator, designed to maximize the offered services. OREO’s key idea is that services can share xApps whenever they correspond to semantically equivalent functions, and the xApp output is of sufficient quality to fulfill the service requirements. By leveraging a multi-layer graph model that captures all the system components, from services to xApps, OREO implements an algorithmic solution that selects the best service configuration, maximizes the number of shared xApps, and efficiently and dynamically allocates resources to them. Numerical results as well as experimental tests performed using our proof-of-concept implementation, demonstrate that OREO closely matches the optimum, obtained by solving an NP-hard problem. Further, it outperforms the state of the art, deploying up to 35% more services with an average of 30% fewer xApps and a similar reduction in the resource consumption.

Network Can Help Check Itself: Accelerating SMT-based Network Configuration Verification Using Network Domain Knowledge

Xing Fang, Feiyan Ding, Bang Huang, Ziyi Wang, Gao Han, Rulan Yang, Lizhao You, Qiao Xiang, Linghe Kong, Yutong Liu, Jiwu Shu

Expand abstract

Satisfiability Modulo Theories (SMT) based network configuration verification tools are powerful tools in preventing network configuration errors. However, their fundamental limitation is efficiency, because they rely on generic SMT solvers to solve SMT problems, which are in general NP-complete. In this paper, we show that by leveraging network domain knowledge, we can substantially accelerate SMT-based network configuration verification. Our key insights are: given a network configuration verification formula, network domain knowledge can (1) guide the search of solutions to the formula by avoiding unnecessary search spaces; and (2) help simplify the formula, reducing the problem scale. We leverage these insights to design a new SMT- based network configuration verification tool called NetSMT. Extensive evaluation using real-world topologies and synthetic network configurations shows that NetSMT achieves orders of magnitude improvements compared to state-of-the-art methods.

BiSwift: Bandwidth Orchestrator for Multi-Stream Video Analytics on Edge

Lin Sun, Weijun Wang, Tingting Yuan, Liang Mi, Haipeng Dai, Yunxin Liu, Xiaoming Fu

Expand abstract

High-definition (HD) cameras for surveillance and road traffic have experienced tremendous growth, demanding intensive computation resources for real-time analytics. Recently, offloading frames from the front-end device to the back-end edge server has shown great promise. In multi-stream competitive environments, efficient bandwidth management and proper scheduling are crucial to ensure both high inference accuracy and high throughput. To achieve this goal, we propose BiSwift, a bi-level framework that scales the concurrent real-time video analytics by a novel adaptive hybrid codec integrated with multi-level pipelines, and a global bandwidth controller for multiple video streams. The lower-level front-back-end collaborative mechanism (called adaptive hybrid codec) locally optimizes the accuracy and accelerates end-to-end video analytics for a single stream. The upper-level scheduler aims to accuracy fairness among multiple streams via the global bandwidth controller. The evaluation of BiSwift shows that BiSwift is able to real-time object detection on 9 streams with an edge device only equipped with an NVIDIA RTX3070 (8G) GPU. BiSwift improves 10%∼21% accuracy and presents 1.2∼ 9× throughput compared with the state-of-the-art video analytics pipelines.

AOC-IDS: Autonomous Online Framework with Contrastive Learning for Intrusion Detection

Xinchen Zhang, Running Zhao, Zhihan Jiang, Zhicong Sun, Yulong Ding, Edith C. H. Ngai, Shuang-Hua Yang

Expand abstract

The rapid expansion of the Internet of Things (IoT) has raised increasing concern about targeted cyber attacks. Previous research primarily focused on static Intrusion Detection Systems (IDSs), which employ offline training to safeguard IoT systems. However, such static IDSs struggle with real-world scenarios where IoT system behaviors and attack strategies can undergo rapid evolution, necessitating dynamic and adaptable IDSs. In response to this challenge, we propose AOC-IDS, a novel online IDS that features an autonomous anomaly detection module (ADM) and a labor-free online framework for continual adaptation. In order to enhance data comprehension, the ADM employs an Autoencoder (AE) with a tailored Cluster Repelling Contrastive (CRC) loss function to generate distinctive representation from limited or incrementally incoming data in the online setting. Moreover, to reduce the burden of manual labeling, our online framework leverages pseudo-labels automatically generated from the decision-making process in the ADM to facilitate periodic updates of the ADM. The elimination of human intervention for labeling and decision-making boosts the system’s compatibility and adaptability in the online setting to remain synchronized with dynamic environments. Experimental validation using the NSL-KDD and UNSW-NB15 datasets demonstrates the superior performance and adaptability of AOC-IDS, surpassing the state-of-the-art solutions. The code is released at https://github.com/xinchen930/AOC-IDS.

A Generic Blockchain-based Steganography Framework with High Capacity via Reversible GAN

Zhuo Chen, Liehuang Zhu, Peng Jiang, Jialing He, Zijian Zhang

Expand abstract

Blockchain-based steganography enables data hiding via encoding the covert data into a specific blockchain transaction field. However, previous works focus on the specific field-embedding methods while lacking a consideration on required field-generation embedding. In this paper, we propose GBSF, a generic framework for blockchain-based steganography. The sender generates the required fields, where the additional covert data is embedded to enhance the channel capacity. Based on GBSF, we design R-GAN that utilizes the generative adversarial network (GAN) with a reversible generator to generate the required fields and encode additional covert data into the input noise of the reversible generator. We then explore the performance flaw of R-GAN and introduce CCR-GAN as an improvement. CCR-GAN employs a counter-intuitive data preprocessing mechanism to reduce decoding errors in covert data. It incurs gradient explosion for model convergence and we design a custom activation function. We conduct experiments using the transaction amount of the Bitcoin mainnet as the required field. The results demonstrate that R-GAN and CCR-GAN allow to embed 11-bit (embedding rate of 17.2%) and 24-bit (embedding rate of 37.5%) covert data within a transaction amount, and enhance the channel capacity of state-of-the-art works by 4.30% to 91.67% and 9.38% to 200.00%, respectively.

Tolerating Disasters with Hierarchical Consensus

Wassim Yahyaoui, Joachim Bruneau-Queyreix, Marcus Völp, Jérémie Decouchant

Expand abstract

Geo-replication provides disaster recovery after catastrophic accidental failures or attacks, such as fires, blackouts or denial-of-service attacks to a data center or region. Naturally distributed data structures, such as Blockchains, when well designed, are immune against such disruptions, but they also benefit from leveraging locality. In this work, we consolidate the performance of geo-replicated consensus by leveraging novel insights about hierarchical consensus and a construction methodology that allows creating novel protocols from existing building blocks. In particular we show that cluster confirmation, paired with subgroup rotation, allows protocols to safely operate through situations where all members of the global consensus group are Byzantine. We demonstrate our compositional construction by combining the recent HotStuff and Damysus protocols into a hierarchical geo-replicated blockchain with global durability guarantees. We present a compositionality proof and demonstrate the correctness of our protocol, including its ability to tolerate cluster crashes. Our protocol — Orion1— achieves a 20% higher throughput than GeoBFT, the latest hierarchical Byzantine Fault-Tolerant (BFT) protocol.

Efficient and Effective In-Vehicle Intrusion Detection System using Binarized Convolutional Neural Network

Linxi Zhang, Xuke Yan, Di Ma

Expand abstract

Modern vehicles are equipped with multiple Electronic Control Units (ECUs) communicating over in-vehicle networks such as Controller Area Network (CAN). Inherent security limitations in CAN necessitate the use of Intrusion Detection Systems (IDSs) for protection against potential threats. While some IDSs leverage advanced deep learning to improve accuracy, issues such as long processing time and large memory size remain. Existing Binarized Neural Network (BNN)-based IDSs, proposed as a solution for efficiency, often compromise on accuracy. To this end, we introduce a novel Binarized Convolutional Neural Network (BCNN)-based IDS, designed to exploit the temporal and spatial characteristics of CAN messages to achieve both efficiency and detection accuracy. In particular, our approach includes a novel input generator capturing temporal and spatial correlations of messages, aiding model learning and ensuring high-accuracy performance. Experimental results suggest our IDS effectively reduces memory utilization and detection latency while maintaining high detection rates. Our IDS runs 4 times faster and utilizes only 3.3% of the memory space required by a full-precision CNN-based IDS. Meanwhile, our proposed system demonstrates a detection accuracy between 94.19% and 96.82% relative to the CNN-based IDS across different attack scenarios. This performance marks a noteworthy improvement over existing state-of-the-art BNN-based IDS designs.

Workflow Optimization for Parallel Split Learning

Joana Tirana, Dimitra Tsigkari, George Iosifidis, Dimitris Chatzopoulos

Expand abstract

Split learning (SL) has been recently proposed as a way to enable resource-constrained devices to train multi-parameter neural networks (NNs) and participate in federated learning (FL). In a nutshell, SL splits the NN model into parts, and allows clients (devices) to offload the largest part as a processing task to a computationally powerful helper. In parallel SL, multiple helpers can process model parts of one or more clients, thus, considerably reducing the maximum training time over all clients (makespan). In this paper, we focus on orchestrating the workflow of this operation, which is critical in highly heterogeneous systems, as our experiments show. In particular, we formulate the joint problem of client-helper assignments and scheduling decisions with the goal of minimizing the training makespan, and we prove that it is NPhard. We propose a solution method based on the decomposition of the problem by leveraging its inherent symmetry, and a second one that is fully scalable. A wealth of numerical evaluations using our testbed’s measurements allow us to build a solution strategy comprising these methods. Moreover, we show that this strategy finds a near-optimal solution, and achieves a shorter makespan than the baseline scheme by up to 52.3%.

MultiHGR: Multi-Task Hand Gesture Recognition with Cross-Modal Wrist-Worn Devices

Mengxia Lyu, Hao Zhou, Kaiwen Guo, Wangqiu Zhou, Xingfa Shen, Yu Gu

Expand abstract

Hand gesture recognition (HGR) is essential for human-machine interaction. Although the existing solutions achieve good performance in specific tasks, they still face challenges when users navigate through different application contexts, i.e., demanding multi-task ability to support newly arrived HGR tasks. In this paper, we propose the first IMU-vision based system hosted on wrist-worn devices to support multi-task HGR, denoted as MultiHGR. The system introduces a novel two-stage training strategy, i.e., task-agnostic stage to align cross-modal features from unlabeled arbitrary gesture through contrastive learning, and task-related stage to learn modality contributions with limited labeled data in specific tasks through self-attention mechanism. Since only the second task-related stage should be executed for each new task, MultiHGR could accommodate multiple tasks with significant reduced training cost and storage requirement. The evaluation results on three HGR tasks demonstrates that MultiHGR reduces 64.92% training time, and 24.04% storage as compared with traditional multimodal single-task models, and MultiHGR outperforms unimodal single-task models with 14.37%, 19.28%, and 31% improvements in these three tasks, respectively. As compared with state-of-the-art multimodal single-task model, MultiHGR achieves average 6.35% accuracy improvement, along with 65.74% training time reduction.

ConFuzz: Towards Large Scale Fuzz Testing of Smart Contracts in Ethereum

Taiyu Wong, Chao Zhang, Yuandong Ni, Mingsen Luo, Heying Chen, Yufei Yu, Weilin Li, Xiapu Luo, Haoyu Wang

Expand abstract

Fuzzing is effective at finding vulnerabilities in traditional applications and has been adapted to smart contracts. However, existing fuzzing solutions for smart contracts are not smart enough and can hardly be applied to large-scale testing since they heavily rely on source code or ABI. In this paper, we propose a fuzzing solution ConFuzz applicable to large-scale testing, especially for bytecode-only contracts. ConFuzz adopts Adaptive Interface Recovery (AIR) and Function Information Collection (FIC) algorithm to automatically recover the function interfaces and information, supporting fuzzing smart contracts without source code or ABI. Furthermore, ConFuzz employs a Dependence-based Transaction Sequence Generation (DTSG) algorithm to infer dependencies of transactions and generate high-quality sequences to trigger the vulnerabilities. Lastly, ConFuzz utilizes taint analysis and function information to help detect harmful vulnerabilities and reduce false positives. The experiment shows that ConFuzz can accurately recover over 99.7% of function interfaces and reports more vulnerabilities than state-of-the-art solutions with 98.89% precision and 93.69% accuracy. On all 1.4M unique contracts from Ethereum, ConFuzz found over 11.92% vulnerable contracts. To the best of our knowledge, ConFuzz is the first efficient and scalable solution to test all smart contracts deployed in Ethereum.

A Learning-only Method for Multi-Cell Multi-User MIMO Sum Rate Maximization

Qingyu Song, Juncheng Wang, Jingzong Li, Guochen Liu, Hong Xu

Expand abstract

Solving the sum rate maximization problem for interference reduction in multi-cell multi-user multiple-input multiple-output (MIMO) wireless communication systems has been investigated for a decade. Several machine learning-assisted methods have been proposed under conventional sum rate maximization frameworks, such as the Weighted Minimum Mean Square Error (WMMSE) framework. However, existing learning-assisted methods suffer from a deficiency in parallelization, and their performance is intrinsically bounded by WMMSE. In contrast, we propose a structural learning-only framework from the abstraction of WMMSE. Our proposed framework increases the solvability of the original MIMO sum rate maximization problem by dimension expansion via a unitary learnable parameter matrix to create an equivalent problem in a higher dimension. We then propose a structural solution updating method to solve the higher dimensional problem, utilizing neural networks to generate the learnable matrix-multiplication parameters. We show that the proposed structural learning framework achieves lower complexity than WMMSE thanks to its parallel implementation. Simulation results under practical communication network settings demonstrate that our proposed learning-only framework achieves up to 98% optimality over state-of-the-art algorithms while providing up to 47× acceleration in various scenarios.

Hybrid Zone: Bridging Acoustic and Wi-Fi for Enhanced Gesture Recognition

Mengning Li, Wenye Wang

Expand abstract

Gesture recognition has significant applications such as assisted living, e-health, and human-device interactions. Moving away from conventional computer vision techniques, recent studies are turning towards ubiquitous methods such as Wi-Fi and acoustic signals, which offer device deployment at minimal costs. In this paper, we explore these two ubiquitous techniques to enhance gesture recognition, addressing challenges related to multi-modal fusion. Due to the inherent differences in signal types, we employ a tailored method to harmonize information from these distinct signals. Traditional multi-modal fusion methods often lack theoretical models due to insufficient analysis of the foundational characteristics of different signals. In particular, we propose a concept, Hybrid Zone, which is a theoretical model illustrating the fusion of acoustic and Wi-Fi sensing. Hybrid zone offers a comprehensive perspective on the fusion of acoustic and Wi-Fi sensing areas. Moreover, it provides an intricate view of the synthesis of acoustic and Wi-Fi velocities at a granular level. Our experimental results have been promising, achieving a high accuracy rate of 93.75% in gesture recognition.

A Two Time-Scale Joint Optimization Approach for UAV-assisted MEC

Zemin Sun, Geng Sun, Long He, Fang Mei, Shuang Liang, Yanheng Liu

Expand abstract

Unmanned aerial vehicles (UAV)-assisted mobile edge computing (MEC) is emerging as a promising paradigm to provide aerial-terrestrial computing services close to mobile devices (MDs). However, meeting the demands of computation-intensive and delay-sensitive tasks for MDs poses several challenges, including the demand-supply contradiction between MDs and MEC servers, the demand-supply heterogeneity between MDs and MEC servers, the trajectory control requirements on energy efficiency and timeliness, and the different timescale dynamics of the network. To address these issues, we first present a hierarchical architecture by incorporating terrestrial-aerial computing capabilities and leveraging UAV flexibility. Furthermore, we formulate a joint computing resource allocation, computation offloading, and trajectory control problem to maximize the system utility. Since the problem is a non-convex mixed integer nonlinear programming (MINLP), we propose a two timescale joint computing resource allocation, computation offloading, and trajectory control (TJCCT) approach. In the short time scale, we propose a price-incentive method for on-demand computing resource allocation and a matching mechanism-based method for computation offloading. In the long time scale, we propose a convex optimization-based method for UAV trajectory control. Besides, we prove the stability, optimality, and polynomial complexity of TJCCT. Simulation results demonstrate that TJCCT outperforms the comparative algorithms in terms of the utility of the system, the QoE of MDs, and the revenue of MEC servers.

Parm: Efficient Training of Large Sparsely-Activated Models with Dedicated Schedules

Xinglin Pan, WenXiang Lin, Shaohuai Shi, Xiaowen Chu, Weinong Sun, Bo Li

Expand abstract

Sparsely-activated Mixture-of-Expert (MoE) layers have found practical applications in enlarging the model size of large-scale foundation models, with only a sub-linear increase in computation demands. Despite the wide adoption of hybrid parallel paradigms like model parallelism, expert parallelism, and expert-sharding parallelism (i.e., MP+EP+ESP) to support MoE model training on GPU clusters, the training efficiency is hindered by communication costs introduced by these parallel paradigms. To address this limitation, we propose Parm, a system that accelerates MP+EP+ESP training by designing two dedicated schedules for placing communication tasks. The proposed schedules eliminate redundant computations and communications and enable overlaps between intra-node and inter-node communications, ultimately reducing the overall training time. As the two schedules are not mutually exclusive, we provide comprehensive theoretical analyses and derive an automatic and accurate solution to determine which schedule should be applied in different scenarios. Experimental results on an 8-GPU server and a 32-GPU cluster demonstrate that Parm outperforms the state-of-the-art MoE training system, DeepSpeed-MoE, achieving 1.13× to 5.77× speedup on 1296 manually configured MoE layers and approximately 3× improvement on two real-world MoE models based on BERT and GPT-2.