Implementation of escape room system based on augmented reality involving deep convolutional neural network
- PDF / 3,119,472 Bytes
- 12 Pages / 595.276 x 790.866 pts Page_size
- 100 Downloads / 272 Views
ORIGINAL ARTICLE
Implementation of escape room system based on augmented reality involving deep convolutional neural network Hong Zeng1 · Xingxi He1 · Honghu Pan1 Received: 19 July 2019 / Accepted: 28 September 2020 © Springer-Verlag London Ltd., part of Springer Nature 2020
Abstract Escape room is a live-action adventure game, where the players search clues, solve puzzles and achieve the assigned tasks. This paper proposed a novel escape room system combining augmented reality and deep learning technology. The system adopts a client–server architecture and can be divided into the server module, the smart glasses module and the player– hardware interaction module. The player–hardware interaction module consists of subsystems each of which includes a Raspberry Pi 3. HoloLens is used as the smart glasses in the experiment of the paper. The server communicates with all the Raspberry Pis and HoloLens through TCP/IP protocol and manages all the devices to achieve the game flow by following the process timeline. The smart glasses module provides two display modes, i.e., the AR 3D models display and the 2D text clues display. In the first mode, the SDK Vuforia is used for detection and tracking of markers. In the second mode, the scene images captured by HoloLens camera are sent to the pre-trained image classifier based on deep convolutional neural network. Considering both the image category and the game status value, the server decides the text clue image to be displayed on HoloLens. The accuracy of the image classification model reaches 94.9%, which can be correctly classified for a certain rotation angle and partial occlusion. The integration of AR, deep learning, electronics and escape room games opens up exciting new directions for the development of escape room. Finally, a built mini-escape room is analyzed to prove that the proposed system can support more complicated narratives showing the potential of achieving immersion. Keywords Augmented reality · Convolutional neural network · Image classification · Escape room game
1 Introduction With the development of technology and the advancement of society, offline games such as live Counter-Strike and live Escape Room, luring the players away from their living room, have become more and more popular in recent years. Compared with online games, the offline games have the advantages of good interactivity and strong immersion (Sun et al. 2015). The escape room game is essentially a live-action adventure game, where team-based players look for clues in a themed room, solve all the puzzles and achieve some specific tasks, usually including getting out of a locked room, in normally 60 min. The teamwork, creativity, patience and speed are required to win an escape room. * Xingxi He [email protected] 1
Department of Mechatronics, College of Mechanical Engineering, Chongqing University, Chongqing 400044, China
As of the end of 2018, there were over 5000 escape rooms worldwide (Miller 2019). The traditional escape rooms focus primarily on locks, puzzles and automatio
Data Loading...
