E3VS-Bench: A Benchmark for Viewpoint-Dependent Active Perception in 3D Gaussian Splatting Scenes

摘要

Visual search in 3D environments requires embodied agentsto actively explore their surroundings and acquire task-relevant evidence.However, existing visual search and embodied AI benchmarks, includ-ing EQA, typically rely on static observations or constrained egocen-tric motion, and thus do not explicitly evaluate fine-grained viewpoint-dependent phenomena that arise under unrestricted 5-DoF viewpointcontrol, such as disambiguating object attributes observable only fromspecific angles. To address this limitation, we introduce E3VS-Bench,a benchmark for embodied 3D visual search where agents must controltheir viewpoints in 5-DoF to gather viewpoint-dependent evidence forquestion answering. E3VS-Bench consists of 99 high-fidelity 3D scenesreconstructed using 3D Gaussian Splatting and 2,014 question-drivenepisodes. 3D Gaussian Splatting enables photorealistic free-viewpointrendering that preserves fine-grained visual details (e.g., small text andsubtle attributes) often degraded in mesh-based simulators, thereby al-lowing the construction of questions that cannot be answered from asingle view and instead require active inspection across viewpoints in5-DoF. We evaluate multiple state-of-the-art VLMs and compare theirperformance with humans. Despite strong 2D reasoning ability, all mod-els exhibit a substantial gap from humans, highlighting limitations inactive perception and coherent viewpoint planning specifically under full5-DoF viewpoint changes. The benchmark, code, and dataset are publiclyavailable at https://k0uya.github.io/e3vs-proj/.