Resource-Aware Decentralized Learning with Rate-Adaptive Quantization

摘要

In this paper, we propose Adriana, the first provable resource-aware decentralized learning framework that leverages rate-adaptive quantization to enhance both computational and communication efficiency. Theoretically, we present a rigorous analysis of the proposed Adriana algorithm and formally characterize how training and communication quantization errors, caused by heterogeneous and limited client resources, jointly affect the convergence of decentralized learning. When these errors are negligible, Adriana converges to a small constant with a rate of $\mathcal{O}\left( {\frac{1}{{\sqrt {nRT} }}} \right)$, which matches the state of the arts under resource-unconstrained scenarios. Then we formulate the quantization scale selection problems and propose rate-adaptive one-block and multi-block communication quantization schemes that enable efficient solutions, thereby accelerating convergence. Extensive experiments on diverse datasets demonstrate that Adriana reduces local computation by 62.63%–71.70% and communication costs by approximately 50%, without sacrificing accuracy compared to full-precision DPSGD.