Integrating Eye-Tracking to Augmented Reality System for Surgical Training
- PDF / 2,106,730 Bytes
- 7 Pages / 595.276 x 790.866 pts Page_size
- 87 Downloads / 266 Views
EDUCATION & TRAINING
Integrating Eye-Tracking to Augmented Reality System for Surgical Training Shang Lu 1 & Yerly Paola Sanchez Perdomo 2 & Xianta Jiang 2 & Bin Zheng 2 Received: 15 January 2019 / Accepted: 16 September 2020 # Springer Science+Business Media, LLC, part of Springer Nature 2020
Abstract Augmented Reality has been utilized for surgical training. During the implementation, displaying instructional information at the right moment is critical for skill acquisition. We built a new surgical training platform combining augmented reality system (HoloLens, Microsoft) with an eye-tracker (Pupil labs, Germany). Our goal is to detect the moments of performance difficulty using the integrated eye-tracker so that the system could display instructions at the precise moment when the user is seeking instructional information during a surgical skill practice in simulation. In the paper, we describe the system design, system calibration and data transferring between these devices. Keywords Augmented reality . Eye tracking . Calibration . Simulation . Surgical training
Introduction Augmented Reality (AR), which adds texts, graphic images or 3D contents into the user’s direct vision of the real world creating a mixed reality, has been widely used for guiding surgical procedures [2, 14, 20] and building the platforms for skill training [1–3, 22]. Many researchers [1, 2] agree that AR can provide graphic guidance to surgeons in real-time, help surgeons match images into the surgical scene, display cutting trajectories or margins, and minimize anatomical ambiguities. All these functions of the AR create substantial ways to enhance the precision of the operation for maximizing patient safety inside the operating room.
* Bin Zheng [email protected] Shang Lu [email protected] Yerly Paola Sanchez Perdomo [email protected] Xianta Jiang [email protected] 1
Multimedia Research Center, Department of Computing Science, University of Alberta, Edmonton, AB, Canada
2
Surgical Simulation Research Lab, Department of Surgery, University of Alberta, Edmonton, AB, Canada
When applied to skill training, AR can guide learners to practice a complex procedure without frequent pauses in the middle of practice to search for instruction. The needed instruction can be displayed to the learners on their side vision. This feature of AR is truly helpful for healthcare learners when they are practicing a procedure composed of multiple steps, such as delivering a baby or inserting a tube to the chest for saving a life. The crucial question in using AR for skill training is how to detect the proper moments for displaying the instructional information to maximize the learning outcome without causing a distraction. A constant display of instructional messages may take away the self-learning process of a learner. To protect the natural learning process, some systems required the learners to signal their moments of performance difficulty by giving a gesture in hand or demand by voice [4]. Both ways are not ideal as the learners need to stop the sk
Data Loading...
