$D^2$: Decentralized Training over Decentralized Data

Abstract

While training a machine learning model using multiple workers, each of which collects data from its own data source, it would be useful when the data collected from different workers are