Joint Face Alignment and 3D Face Reconstruction
We present an approach to simultaneously solve the two problems of face alignment and 3D face reconstruction from an input 2D face image of arbitrary poses and expressions. The proposed method iteratively and alternately applies two sets of cascaded regre
- PDF / 13,006,522 Bytes
- 16 Pages / 439.37 x 666.142 pts Page_size
- 24 Downloads / 248 Views
2
College of Computer Science, Sichuan University, Chengdu, China [email protected] Department of Computer Science and Engineering, Michigan State University, East Lansing, MI, USA
Abstract. We present an approach to simultaneously solve the two problems of face alignment and 3D face reconstruction from an input 2D face image of arbitrary poses and expressions. The proposed method iteratively and alternately applies two sets of cascaded regressors, one for updating 2D landmarks and the other for updating reconstructed poseexpression-normalized (PEN) 3D face shape. The 3D face shape and the landmarks are correlated via a 3D-to-2D mapping matrix. In each iteration, adjustment to the landmarks is firstly estimated via a landmark regressor, and this landmark adjustment is also used to estimate 3D face shape adjustment via a shape regressor. The 3D-to-2D mapping is then computed based on the adjusted 3D face shape and 2D landmarks, and it further refines the 2D landmarks. An effective algorithm is devised to learn these regressors based on a training dataset of pairing annotated 3D face shapes and 2D face images. Compared with existing methods, the proposed method can fully automatically generate PEN 3D face shapes in real time from a single 2D face image and locate both visible and invisible 2D landmarks. Extensive experiments show that the proposed method can achieve the state-of-the-art accuracy in both face alignment and 3D face reconstruction, and benefit face recognition owing to its reconstructed PEN 3D face shapes. Keywords: Face alignment regression
1
·
3D face reconstruction
·
Cascaded
Introduction
Three-dimensional (3D) face models have recently been employed to assist pose or expression invariant face recognition [3,14,42], and the state-of-the-art performance has been achieved. A crucial step in these 3D face-assisted face recognition methods is to reconstruct the 3D face model from a two-dimensional (2D) face image. Besides its applications in face recognition, 3D face reconstruction is also Electronic supplementary material The online version of this chapter (doi:10. 1007/978-3-319-46454-1 33) contains supplementary material, which is available to authorized users. c Springer International Publishing AG 2016 B. Leibe et al. (Eds.): ECCV 2016, Part V, LNCS 9909, pp. 545–560, 2016. DOI: 10.1007/978-3-319-46454-1 33
546
F. Liu et al.
Fig. 1. We view 2D landmarks are generated from a 3D face through 3D expression (fE ) and pose (fP ) deformation, and camera projection (fC ) (top row). While conventional face alignment and 3D face reconstruction are two separate tasks and the latter requires the former as the input, this paper performs these two tasks jointly, i.e., reconstructing a pose-expression-normalized (PEN) 3D face and estimating visible/invisible landmarks (green/red points) from a 2D face image with arbitrary poses and expressions. (Color figure online)
useful in other face-related tasks, such as facial expression analysis [7,36] and facial animation [4,5]. While many 3D face reconstructio
Data Loading...
