SeSDF: Self-evolved Signed Distance Field for Implicit 3D Clothed Human Reconstruction
Yukang Cao
Kai Han
Kwan-Yee K. Wong
The University of Hong Kong
CVPR 2023 [Paper]


We address the problem of clothed human reconstruction from a single image or uncalibrated multi-view images. Existing methods struggle with reconstructing detailed geometry of a clothed human and often require a calibrated setting for multi-view reconstruction. We propose a flexible framework which, by leveraging the parametric SMPL-X model, can take an arbitrary number of input images to reconstruct a clothed human model under an uncalibrated setting. At the core of our framework is our novel self-evolved signed distance field (SeSDF) module which allows the framework to learn to deform the signed distance field (SDF) derived from the fitted SMPL-X model, such that detailed geometry reflecting the actual clothed human can be encoded for better reconstruction. Besides, we propose a simple method for self-calibration of multi-view images via the fitted SMPL-X parameters. This lifts the requirement of tedious manual calibration and largely increases the flexibility of our method. Further, we introduce an effective occlusion-aware feature fusion strategy to account for the most useful features to reconstruct the human model. We thoroughly evaluate our framework on public benchmarks, and our framework establishes a new state-of-the-art.



Single-view Results

Multi-view Results


Y. Cao, K. Han, K.-Y. K. Wong.
SeSDF: Self-evolved Signed Distance Field for Implicit 3D Clothed Human Reconstruction.
In CVPR, 2023. [pdf]

author    = {Yukang Cao and Kai Han and Kwan-Yee K. Wong},
title     = {SeSDF: Self-evolved Signed Distance Field for Implicit 3D Clothed Human Reconstruction},
booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
year      = {2023},

Code for SeSDF will be available at Github!


This work is supported by Hong Kong Research Grant Council - Early Career Scheme (Grant No. 27208022) and HKU Seed Fund for Basic Research. We appreciate the very helpful discussions with Dr. Guanying Chen. We also thank Yuliang Xiu, Yuanlu Xu for ARCH and ARCH++ results.

Webpage template borrowed from Split-Brain Autoencoders, CVPR 2017.