Plug-and-Play Traffic Element Awareness for End-to-End Autonomous Driving

Abstract

Traffic elements such as traffic lights and road signs play a fundamental role in human driving decisions and should naturally influence end-to-end driving performance. However, existing end-to-end driving research predominantly focuses on dynamic road participants (e.g., vehicles and pedestrians), while the role of traffic elements remains largely unexplored. To date, the community lacks a systematic study quantifying how traffic elements affect end-to-end driving models. This gap stems from two main challenges: first, existing public datasets rarely provide structured annotations for traffic elements; second, modern end-to-end driving systems vary widely in architectures and training paradigms, making conclusions drawn from a single method difficult to generalize. In this work, we present the first systematic investigation of traffic element awareness for end-to-end autonomous driving. We begin by constructing a unified research infrastructure by augmenting multiple public driving datasets with comprehensive traffic element annotations. To ensure broad applicability across diverse model families, we intentionally adopt a minimal and universal integration design, allowing traffic element signals to be incorporated into existing pipelines in a plug-and-play manner with negligible architectural modification. We evaluate this design across a wide spectrum of modern paradigms, including perception–prediction-planning pipelines, vision-language-action models (VLA), regression-based planners, diffusion-based policies, and trajectory scoring frameworks. Experiments are conducted on multiple widely used benchmarks, including nuScenes, NAVSIM-v1, NAVSIM-v2, and Bench2Drive. Across all paradigms and datasets, our simple integration consistently improves driving performance, demonstrating that traffic element awareness provides a robust and generalizable signal for end-to-end driving systems. Notably, on the challenging NAVSIM-v2 benchmark, our approach significantly boosts the performance of stateof-the-art architectures and data pipelines, establishing a new state of the art.