Adaptive Segmented Decentralized Federated Learning on Tiny Devices

Abstract

We present tinyGist, an efficient decentralized federated learning framework designed for tiny microcontrollers. Instead of treating all model parameters equally, each device exchanges a portion of them, effectively sharing a “gist” of the information. Our major contributions are: (1) segment model parameters based on their importance, while identifying the most impactful updates, (2) share the gist probabilistically, maintaining model diversity while ensuring fast knowledge propagation across peers, (3) update models based on the information merit received from other neighbors, and (4) adaptive learning. We implement tinyGist across various simulation experiments, large-scale realistic emulation with hundreds of devices, and real deployment on homogeneous and heterogeneous physical microcontroller clusters. Evaluated on eight machine learning models and 15 tasks, tinyGist outperforms existing baselines in convergence speed by 1.35–3.3× and accuracy by up to 17%, while reducing communication overhead by ≈3× compared to the full-model baselines. We further validate our lightweight importance proxy against gradient-, Fisher-, and Hessian-based measures, and characterize how tinyGist interacts with differential privacy and secure aggregation.