Face-Off: A Face Reconstruction Technique for Virtual Reality (VR) Scenarios
Virtual Reality (VR) headsets occlude a significant portion of human face. The real human face is required in many VR applications, for example, video teleconferencing. This paper proposes a wearable camera setup-based solution to reconstruct the real fac
- PDF / 1,806,321 Bytes
- 14 Pages / 439.37 x 666.142 pts Page_size
- 86 Downloads / 192 Views
tment of Applied Physics and Electronics, Ume˚ a University, 90187 Ume˚ a, Sweden {muhammad.sikandar.lal.khan,shafiq.urrehman}@umu.se 2 KTH Royal Institute of Technology, 100 44 Stockholm, Sweden 3 University of East London, London, England
Abstract. Virtual Reality (VR) headsets occlude a significant portion of human face. The real human face is required in many VR applications, for example, video teleconferencing. This paper proposes a wearable camera setup-based solution to reconstruct the real face of a person wearing VR headset. Our solution lies in the core of asymmetrical principal component analysis (aPCA). A user-specific training model is built using aPCA with full face, lips and eye region information. During testing phase, lower face region and partial eye information is used to reconstruct the wearer face. Online testing session consists of two phases, (i) calibration phase and (ii) reconstruction phase. In former, a small calibration step is performed to align test information with training data, while the later uses half face information to reconstruct the full face using aPCAbased trained-data. The proposed approach is validated with qualitative and quantitative analysis. Keywords: Virtual reality Wearable setup · Oculus
1
· VR headset · Face reconstruction · PCA ·
Introduction
Visualization is an important pillar for multimedia computing. The devices used for visualizing multimedia contents can be broadly categorized into (i) nonwearable computer devices and (ii) wearable computer devices. Non-wearable computer technologies employ two-dimensional (2-D) and/or three dimensional (3-D) displays/technologies for multimedia content rendering [13]. Three dimensional (3-D) technologies offer one more dimension for visualizing the data flow and the interplay of programs in complex multimedia applications. However, in past few years, this trend is shifting from non-wearable visualization to wearable visualization experience [12,15]. The wearable devices used for watching multimedia content are commonly known as virtual reality (VR) headsets and/or head mounted displays (HMD). These virtual reality (VR) headsets are becoming popular because of cheaper prices, more immersive experience, high quality, c Springer International Publishing Switzerland 2016 G. Hua and H. J´ egou (Eds.): ECCV 2016 Workshops, Part I, LNCS 9913, pp. 490–503, 2016. DOI: 10.1007/978-3-319-46604-0 35
Face Reconstruction Technique for VR
Fig. 1. Virtual reality headsets; (a) Oculus rift, (b) Gear VR, (c) HTC vive, (d) Google cardboard.
491
Fig. 2. Our wearable virtual reality setup: it consists of two RGB cameras along with a VR headset. Front camera is used for capturing lower face region and a side camera is used for capturing profile of an eye region.
better screen resolution, low latency and better control. According to statistics, around 6.7 million people have used VR headsets in year 2015 and it is expected to grow to 43 million users in year 2016 and 171 million users in year 2018 [31]. These headsets, for instance, Oculus
Data Loading...
