Streaming Dense Voxel Representations for 3D Occupancy Prediction

Abstract

In this paper, we explore dense voxel streaming for accurateand efficient 3D occupancy prediction. While dense voxel representa-tions offer fine-grained spatial details and streaming paradigm enablesefficient temporal processing, naively combining the two introduces keychallenges: (i) warping-induced distortions caused by interpolation usedfor temporal alignment, and (ii) degraded dynamic object representa-tions due to motion misalignment and detail loss in image-to-voxel pro-jection. To address these, we propose StreamOcc, a novel frameworkthat utilizes two aggregation strategies. Specifically, it first refines prop-agated voxel features to reduce warping artifacts before temporal accu-mulation, and then selectively injects instance-level query features en-coding dynamic-object semantics into the corresponding occupied voxelregions, preserving temporally consistent modeling while strengtheningdynamic object representations. Unlocking effective dense voxel stream-ing, StreamOcc achieves state-of-the-art performance on SurroundOcc-benchmark and Occ3D-nuScenes under real-time constraints, outper-forming the prior best methods by +1.3/2.5 and +1.5/2.0 in (over-all/dynamic object) mIoU, respectively, while running at 83.3 msper frame with only 2.8 GB of memory. The project page is available athttps://moonseokha.github.io/StreamOcc/.