Augmenting with NeRFs: Fast Relocalization on Densified Datasets

摘要

We reinterpret NeRFs as a resource for extreme data augmentation to advance the field of camera relocalization. Our approach lets us automatically render a massive, densified dataset of novel views, given only sparse ground-truth viewpoints. We introduce a filtering strategy that, compared to existing novel-view-synthesis-focused relocalizers, does not rely on custom or specific NeRF backbones. This filtering strategy allows for significant spatial extrapolation within the scene, without compromising novel view quality. As a result, training a lightweight off-the-shelf vision backbone as a pose regressor on our expanded datasets significantly improves accuracy, uniquely enables relocalization of very spatially-novel views, and performs well on portable-scale hardware.