The effects of spatial auditory and visual cues on mixed reality remote collaboration
- PDF / 1,546,501 Bytes
- 16 Pages / 595.276 x 790.866 pts Page_size
- 98 Downloads / 195 Views
ORIGINAL PAPER
The effects of spatial auditory and visual cues on mixed reality remote collaboration Jing Yang1
· Prasanth Sasikumar2 · Huidong Bai2 · Amit Barde2
· Gábor Sörös3 · Mark Billinghurst2
Received: 2 February 2020 / Accepted: 6 June 2020 © Springer Nature Switzerland AG 2020
Abstract Collaborative Mixed Reality (MR) technologies enable remote people to work together by sharing communication cues intrinsic to face-to-face conversations, such as eye gaze and hand gestures. While the role of visual cues has been investigated in many collaborative MR systems, the use of spatial auditory cues remains underexplored. In this paper, we present an MR remote collaboration system that shares both spatial auditory and visual cues between collaborators to help them complete a search task. Through two user studies in a large office, we found that compared to non-spatialized audio, the spatialized remote expert’s voice and auditory beacons enabled local workers to find small occluded objects with significantly stronger spatial perception. We also found that while the spatial auditory cues could indicate the spatial layout and a general direction to search for the target object, visual head frustum and hand gestures intuitively demonstrated the remote expert’s movements and the position of the target. Integrating visual cues (especially the head frustum) with the spatial auditory cues significantly improved the local worker’s task performance, social presence, and spatial perception of the environment. Keywords Mixed reality · Augmented reality · Virtual reality · Remote collaboration · Spatial audio · Hand gesture
1 Introduction This paper explores the impact of using spatial auditory and visual cues in a Mixed Reality (MR) interface for remote collaboration. Remote collaboration enables spatially distant people to work together, which can increase collaborators’
B
Jing Yang [email protected] Prasanth Sasikumar [email protected] Huidong Bai [email protected] Amit Barde [email protected] Gábor Sörös [email protected] Mark Billinghurst [email protected]
1
Department of Computer Science, ETH Zurich, Zurich, Switzerland
2
Auckland Bioengineering Institute, University of Auckland, Auckland, New Zealand
3
Nokia Bell Labs, Budapest, Hungary
productivity and is a cost-effective alternative to business travel and expert visits. However, most people still prefer direct face-to-face communication over current audio-video conferencing solutions, partly because the latter usually fail to convey the implicit non-verbal cues that play an important role in face-to-face collaborations. The limitations of video conferencing can be addressed by using MR technologies, which seamlessly combine virtual contents with the real environment. MR remote collaboration systems with Head-Mounted Displays (HMDs) can transfer various spatial cues used in face-to-face communications. For example, besides talking like in a phone call [12], users can also share their eye gaze and ha
Data Loading...
