Model-Based Real-Time Head Tracking
- PDF / 3,245,851 Bytes
- 14 Pages / 612 x 792 pts (letter) Page_size
- 17 Downloads / 204 Views
Model-Based Real-Time Head Tracking ¨ Jacob Strom Multimedia Technologies Department, Ericsson Research, Torshamnsgatan 23, 164 86 Stockholm, Sweden Email: [email protected] Received 31 August 2001 and in revised form 29 May 2002 This paper treats real-time tracking of a human head using an analysis by synthesis approach. The work is based on the Structure from Motion (SfM) algorithm from Azarbayejani and Pentland (1995). We will analyze the convergence properties of the SfM algorithm for planar objects, and extend it to handle new points. The extended algorithm is then used for head tracking. The system tracks feature points in the image using a texture mapped three-dimensional model of the head. The texture is updated adaptively so that points in the ear region can be tracked when the user’s head is rotated far, allowing out-of-plane rotation of up to 90◦ without losing track. The covariance of the x- and the y-coordinates are estimated and forwarded to the Kalman filter, making the tracker robust to occlusion. The system automatically detects tracking failure and reinitializes the algorithm using information gathered in the original initialization process. Keywords and phrases: face tracking, modeling, real-time, EKF, structure from motion, planar objects, analysis by synthesis.
1.
INTRODUCTION
Automatic tracking and modeling of human faces from image sequences is an important and challenging task in computer vision. Applications include face recognition, modelbased coding for video conferencing, avatar control, and computer graphics. The main goal of this paper is to present a tracking system built on the extended Kalman filter based SfM algorithm in [1], thus extending the foundations of Azarbayejani and Pentland and Jebara and Pentland [2] to achieve robust performance. Since points on the face might lie in a nearly planar constellation, the stability of the SfM algorithm is investigated for planar surfaces. The theory of Triggs for SfM of planar objects will be used as a starting point, and simulations on both noise free and noisy data are carried out to investigate how often the algorithm converges. The results are compared to general three-dimensional objects. The algorithm is also extended to handle new points, that is, points that are not visible in the first frame. This poses a problem since the first frame is used as a reference frame in the error function that the Kalman filter is minimizing. Three ways to handle these new points are investigated, and the resulting solution is to keep the old reference frame for the old points and use the new reference frame for the new ones. The results will be applied to a face tracking system. The core idea is to select a dense set of feature points (essentially, optical flow at all the most information bearing points). Figure 1 illustrates how the system works. Patches around the feature points taken from the rendered threedimensional model (lower left corner) are matched against the incoming video, and the two-dimensional trajectories
of these feature
Data Loading...
