SceneDiff: A Benchmark and Method for Multiview Object Change Detection

Abstract

We investigate the problem of identifying objects that havebeen added, removed, or moved between a pair of captures (imagesor videos) of the same scene at different times. Accurately identifyingverifiable changes is extremely challenging – some objects may appear tobe missing because they are occluded or out of frame, while others mayappear different due to large viewpoint changes. To study this problem, weintroduce the SceneDiff Benchmark, the first multiview change detectiondataset for scenes captured along different camera trajectories, comprising350 diverse video pairs with dense object instance-level annotations. Wealso introduce the SceneDiff algorithm, a training-free approach thatsolves for image poses, segments images into objects, and compares themusing semantic and geometric features. By building on pretrained models,SceneDiff generalizes across domains without retraining and naturallyimproves as the underlying models advance. Experiments on multiviewand two-view benchmarks demonstrate that our method outperformsexisting approaches by large margins (51.6% and 30.6% relative APimprovements). Project page: https://yuqunw.github.io/SceneDiff