PATHS: Plug-and-Play Tracking from Heterogeneous Streams to a Shared Token Interface

Abstract

Multimodal target tracking estimates a moving target's trajectory by fusing heterogeneous per-anchor feature streams from sensing modalities. Existing tracking pipelines are usually built for a fixed feature set and anchor layout. They become brittle when streams drop out, new feature families are introduced, or the deployment changes. We present PATHS, a deployment-aware token interface for plug-and-play tracking. PATHS converts per-anchor feature streams into a shared, fixed-width token interface, allowing a unified tracking model to operate as the available feature mix changes. To add a feature family, PATHS trains only a new encoder with a frozen reference decoder, aligning its tokens to the shared interface without paired cross-modal data. At runtime, set attention fuses any available tokens, and a shared decoder reconstructs the trajectory. A reference-frame mapping adapts PATHS to different spatial scales without retraining. Across seven feature families in simulation and three real deployments, PATHS achieves mean tracking errors of 0.13–0.49 m and reduces median error by up to 33% relative to fixed-input baselines under stream quality fluctuations.