Learning-detailed 3D face reconstruction based on convolutional neural networks from a single image
- PDF / 2,530,634 Bytes
- 14 Pages / 595.276 x 790.866 pts Page_size
- 7 Downloads / 232 Views
(0123456789().,-volV)(0123456789().,-volV)
ORIGINAL ARTICLE
Learning-detailed 3D face reconstruction based on convolutional neural networks from a single image Asad Khan1 • Sakander Hayat2 • Muhammad Ahmad3 • Jinde Cao4 • Muhammad Faizan Tahir5 Asad Ullah6 • Muhammad Sufyan Javed7,8
•
Received: 21 August 2019 / Accepted: 18 September 2020 Ó Springer-Verlag London Ltd., part of Springer Nature 2020
Abstract The efficiency of convolutional neural networks (CNNs) facilitates 3D face reconstruction, which takes a single image as an input and demonstrates significant performance in generating a detailed face geometry. The dependence of the extensive scale of labelled data works as a key to making CNN-based techniques significantly successful. However, no such datasets are publicly available that provide an across-the-board quantity of face images with correspondingly explained 3D face geometry. State-of-the-art learning-based 3D face reconstruction methods synthesize the training data by using a coarse morphable model of a face having non-photo-realistic synthesized face images. In this article, by using a learning-based inverse face rendering, we propose a novel data-generation technique by rendering a large number of face images that are photo-realistic and possess distinct properties. Based on the real-time fine-scale textured 3D face reconstruction comprising decently constructed datasets, we can train two cascaded CNNs in a coarse-to-fine manner. The networks are trained for actual detailed 3D face reconstruction from a single image. Experimental results demonstrate that the reconstruction of 3D face shapes with geometry details from only one input image can efficiently be performed by our method. Furthermore, the results demonstrate the efficiency of our technique to pose, expression and lighting dynamics. Keywords 3D face reconstruction Single images Lighting Inverse rendering Image synthesis Convolutional neural networks
1 Introduction
Asad Khan and Sakander Hayat contributed equally to this work.
Applications related to computer vision have been extensively studied in the last decade. One of these applications includes three-dimensional (3D) face reconstruction. 3D
& Asad Khan [email protected]
5
School of Electric Power, South China University of Technology, Guangzhou 510640, China
& Sakander Hayat [email protected]
6
Department of Mathematical Sciences, Karakoram International University (KIU), Gilgit-Baltistan, Pakistan
7
Department of Physics, COMSATS University Islamabad, Lahore Campus, Punjab 54000, Pakistan
8
Siyuan Laboratory, Department of Physics, Jinan University, Guangzhou 510632, People’s Republic of China
1
School of Computer Science and Software Engineering, Guangzhou University, Guangzhou 510006, People’s Republic of China
2
Faculty of Engineering Sciences, GIK Institute of Engineering Sciences and Technology, Topi, Pakistan
3
Department of Computer Science, National University of Computer and Emerging Sciences (NUCES-FAST), Faisalabad Campus, Pakistan
Data Loading...