DeltaAR: Redundancy-Aware Object Detection for Mobile Augmented Reality

Abstract

Real-time object detection is essential for mobile augmented reality (AR) applications; however, modern detectors based on Vision Transformers (ViTs) incur significant computational overhead. Although edge offloading can alleviate the limited compute resources on mobile devices, the inference latency of ViT-based detection on edge servers remains a major bottleneck, causing detection results to arrive after the camera viewpoint has changed and degrading AR experiences. To address this problem, we propose DeltaAR, an edge-assisted AR object detection system that leverages AR geometry (depth maps and 6-DoF camera poses) to reduce the cost of ViT-based detection. Different from existing AR systems, DeltaAR uses sensory data available on modern AR devices to accurately identify newly exposed regions between consecutive frames. Based on this information, we design a region-aware ViT that reduces computation by processing tokens required for detecting objects in newly exposed regions and skipping redundant tokens from old regions. To preserve detection accuracy, we further introduce a lightweight token selector that retains a small set of old-region tokens needed for cross-boundary objects and contextual information. Experiments show that DeltaAR significantly reduces edge inference cost and latency while maintaining high detection accuracy for real-time AR applications.