EventSTU: Event-Guided Efficient Spatio-Temporal Understanding for Video-LLMs

Abstract

Video large language models have demonstrated strong video understanding capabilities, but suffer from high inference costs due to the massive number of tokens in long videos. Inspired by efficient biological vision systems, we propose EventSTU, a training-free framework guided by bio-inspired event cameras to significantly reduce redundant tokens, thereby enabling efficient spatio-temporal understanding. In the temporal domain, we design a coarse-to-fine keyframe sampling algorithm that exploits the change-triggered property of event cameras to eliminate redundant frames. In the spatial domain, we design an adaptive token pruning algorithm that leverages the visual saliency of events as a zero-cost prior to guide token reduction. From a holistic spatio-temporal perspective, we further integrate question relevance from keyframe sampling to adaptively allocate token retention budgets. To facilitate evaluation, we construct EventBench, the first event-inclusive, human-annotated multimodal benchmark that covers diverse real-world scenarios. Beyond physical event cameras, EventSTU also supports general video understanding through simulated events. Comprehensive experiments show that EventSTU achieves 2.87× FLOPs reduction and 3.10× prefilling speedup over the strongest baseline while still improving performance.