CAIC: Congestion-Aware Intent Communication for Multi-Agent Reinforcement Learning

Abstract

In multi-agent reinforcement learning, communication is essential for effective cooperation. Most existing methods assume instantaneous message delivery, and the few delay-aware studies consider only external fixed or stochastic delays, neglecting queueing delays caused by competition for a shared channel. We formulate the shared channel as a queueing system with state-dependent service rates, and propose Congestion-Aware Intent Communication (CAIC). To ensure message validity under delay, CAIC employs a temporal masked autoencoder to predict each agent’s future trajectory and encode it into a delay-robust intent message, with attention-based fusion integrating asynchronous messages at the receiver. To maintain timeliness, CAIC dynamically adjusts communication frequency based on intent message changes and delay estimates, proactively mitigating congestion. Experiments on Hallway, MPE, and SMAC demonstrate that CAIC outperforms existing baselines on most evaluated scenarios in both no-delay and queueing-delay settings, and ablation studies reveal that stale messages under delay can be even more harmful than no communication.