Gaussian Garments: Reconstructing Simulation-Ready Clothing with Photorealistic Appearance from Multi-View Video

摘要

We introduce Gaussian Garments, a novel approach for reconstructing realistic-looking, simulation-ready, garments from multi-view videos. Our method represents garments with a combination of a 3D mesh and a Gaussian texture that encodes both the color and high-frequency surface texture. With this, it is able to model rich textures and surface detail including complex materials such as fur. As part of our reconstruction process, we demonstrate how such a representation can be used to register a 3D mesh to multi-view videos. Our reconstructed Gaussian Garments can be animated on unseen body shapes and motions using physics-based simulation. We show how a learned GNN-based physical simulator can be used to optimize the garment's material parameters to match its real behavior. Finally, we devise a procedure for automatically ordering virtual garments, allowing us to combine the captured individual garments into multi-garment outfits.