X-Stream: Benchmarking MLLMs as Multiplexers for Multi-Stream Understanding

Abstract

While video streaming understanding has made significantstrides, real-world applications, such as live sports broadcasting, au-tonomous driving, and multi-screen collaboration, inherently demandcontinuous, multi-stream interactions. However, existing benchmarks areconfined to single-stream paradigms, leaving a critical gap in evaluatingonline, cross-stream reasoning. To bridge this, we introduce X-Stream,the first benchmark dedicated to multi-stream streaming understand-ing. Comprising 4,220 rigorously curated QA pairs across 932 videos,X-Stream evaluates 11 subtasks across multi-window, multi-view, andmulti-device scenarios. Crucially, our dataset is constructed using a noveldual-verification pipeline that prevents over-reliance on a single stream.Furthermore, we pioneer the conceptualization of multi-modal large lan-guage models (MLLMs) as naive multiplexers, systematically evaluat-ing their performance through the lens of Signal Multiplexing Theory. Ourextensive online inference experiments reveal a stark reality: state-of-the-art MLLMs struggle significantly with concurrent streams, achieving only∼50% score and exhibiting a poor proactive ability. Ultimately, X-Streamexposes the trade-off of current multiplexing schemes, providing both apractical evaluation protocol and empirical guidance for next-generationmulti-stream agents. Code and data are released at Œhomepage.