ObjectCarver: Semi-automatic segmentation, reconstruction and separation of 3D objects

Abstract

Implicit neural fields have made remarkable progress in reconstructing 3D surfaces from multiple images; however, they encounter challenges when it comes to separating individual objects within a scene. Previous approaches to this problem require ground-truth segmentation masks and introduce floating artifacts in occluded parts of the scene. We address these challenges with ObjectCarver. ObjectCarver requires no ground-truth segmentation; all it needs is just a few user clicks in a single view. ObjectCarver also introduces a new loss function that prevents floaters and avoids inappropriate carving-out due to occlusion. Finally, ObjectCarver uses a simple initialization technique that significantly speeds up the process while preserving geometric details. We demonstrate qualitatively and quantitatively on multiple datasets (including a new dataset and benchmark with complete ground-truth) that ObjectCarver produces more accurate reconstructions of each object while minimizing artifacts