CSTDFormer: Empowering Transformers to Learn Spatio-Temporal Delays via Contrastive Learning

Abstract

Urban spatio-temporal prediction is crucial for real-world applications such as traffic dispatching, environmental monitoring, and emergency response. Despite breakthroughs in deep learning that have advanced spatio-temporal pattern learning, the complex spatio-temporal dependencies inherent in urban dynamic systems continue to pose challenges for effective spatio-temporal modeling. Existing approaches often focus on modifying model architectures and assume that spatio-temporal information passes immediately, while neglecting the spatio-temporal delays caused by urban events, geographic factors, and sensor communication latency. To fill this gap, we propose CSTDFormer, which leverages contrastive learning to empower the pure Transformer architecture to capture both short-range and long-range spatio-temporal delays. Specifically, we introduce Spatio-Temporal Delay Contrastive Learning (STD-CL) combined with self-attention to model fine-grained short-range spatio-temporal delay effects, and then devise Local-Global Contrastive Learning (LG-CL) combined with cross-attention to model long-range spatio-temporal delay relationships. Extensive experiments demonstrate that our approach outperforms nine baseline methods on five real-world datasets from different urban spatio-temporal scenarios, highlighting the necessity of considering spatio-temporal delays in urban spatio-temporal prediction.