The First 3D Face Alignment in the Wild (3DFAW) Challenge
2D alignment of face images works well provided images are frontal or nearly so and pitch and yaw remain modest. In spontaneous facial behavior, these constraints often are violated by moderate to large head rotation. 3D alignment from 2D video has been p
- PDF / 1,917,653 Bytes
- 10 Pages / 439.37 x 666.142 pts Page_size
- 99 Downloads / 200 Views
4
Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, USA laszlojeni@cmu.edu 2 DISI, University of Trento, Trento, Italy Department of Computer Science, State University of New York at Binghamton, Binghamton, USA Department of Psychology, The University of Pittsburgh, Pittsburgh, PA, USA
Abstract. 2D alignment of face images works well provided images are frontal or nearly so and pitch and yaw remain modest. In spontaneous facial behavior, these constraints often are violated by moderate to large head rotation. 3D alignment from 2D video has been proposed as a solution. A number of approaches have been explored, but comparisons among them have been hampered by the lack of common test data. To enable comparisons among alternative methods, The 3D Face Alignment in the Wild (3DFAW) Challenge, presented for the first time, created an annotated corpus of over 23,000 multi-view images from four sources together with 3D annotation, made training and validation sets available to investigators, and invited them to test their algorithms on an independent test-set. Eight teams accepted the challenge and submitted test results. We report results for four that provided necessary technical descriptions of their methods. The leading approach achieved prediction consistency error of 3.48 %. Corresponding result for the lowest ranked approach was 5.9 %. The results suggest that 3D alignment from 2D video is feasible on a wide range of face orientations. Differences among methods are considered and suggest directions for further research. Keywords: 3D alignment from 2D video consistency error · Faces in-the-wild
1
· Head rotation · Prediction
Introduction
Face alignment – the problem of automatically locating detailed facial landmarks across different subjects, illuminations, and viewpoints – is critical to face analysis applications, such as identification, facial expression analysis, robot-human interaction, affective computing, and multimedia. Previous methods can be divided into two broad categories: 2D approaches and 3D approaches. 2D approaches treat the face as a 2D object. This assumption holds as long as the face is frontal and planar. As face orientation varies c Springer International Publishing Switzerland 2016 G. Hua and H. J´ egou (Eds.): ECCV 2016 Workshops, Part II, LNCS 9914, pp. 511–520, 2016. DOI: 10.1007/978-3-319-48881-3 35
512
L.A. Jeni et al.
from frontal, 2D annotated points lose correspondence. Pose variation results in self-occlusion that confounds landmark annotation. 2D approaches include Active Appearance Models [5,16], Constrained Local Models [6,21] and shaperegression-based methods [4,8,18,24]). These approaches train a set of 2D models, each of which is intended to cope with shape or appearance variation within a small range of viewpoints. 3D approaches have strong advantages over 2D with respect to representational power and robustness to illumination and pose. 3D approaches [2,7,12,27] accommodate a wide range of views. Depending on the 3D model, they easily can accommodate a full
