We address the problem of clothed human reconstruction from a single image or uncalibrated multi-view images. Existing methods struggle with reconstructing detailed geometry of a clothed human and often require a calibrated setting for multi-view reconstruction. We propose a flexible framework which, by leveraging the parametric SMPL-X model, can take an arbitrary number of input images to reconstruct a clothed human model under an uncalibrated setting. At the core of our framework is our novel self-evolved signed distance field (SeSDF) module which allows the framework to learn to deform the signed distance field (SDF) derived from the fitted SMPL-X model, such that detailed geometry reflecting the actual clothed human can be encoded for better reconstruction. Besides, we propose a simple method for self-calibration of multi-view images via the fitted SMPL-X parameters. This lifts the requirement of tedious manual calibration and largely increases the flexibility of our method. Further, we introduce an effective occlusion-aware feature fusion strategy to account for the most useful features to reconstruct the human model. We thoroughly evaluate our framework on public benchmarks, and our framework establishes a new state-of-the-art.
Video
Method
Single-view Results
Multi-view Results
Paper
Y. Cao, K. Han, K.-Y. K. Wong. SeSDF: Self-evolved Signed Distance Field for Implicit 3D Clothed Human Reconstruction.
In CVPR, 2023. [pdf]
@inproceedings{cao2023sesdf,
author = {Yukang Cao and Kai Han and Kwan-Yee K. Wong},
title = {SeSDF: Self-evolved Signed Distance Field for Implicit 3D Clothed Human Reconstruction},
booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
year = {2023},
}
This work is supported by Hong Kong Research Grant Council - Early Career Scheme (Grant No. 27208022) and HKU Seed Fund for Basic Research.
We appreciate the very helpful discussions with Dr. Guanying Chen.
We also thank Yuliang Xiu, Yuanlu Xu for ARCH and ARCH++ results.
