Real-time virtual mouse system using RGB-D images and fingertip detection
- PDF / 1,709,399 Bytes
- 18 Pages / 439.37 x 666.142 pts Page_size
- 57 Downloads / 229 Views
Real-time virtual mouse system using RGB-D images and fingertip detection Dinh-Son Tran 1 & Ngoc-Huynh Ho 1 & Hyung-Jeong Yang 1 Guee Sang Lee 1
1
& Soo-Hyung Kim &
Received: 2 March 2019 / Revised: 3 August 2020 / Accepted: 10 November 2020 # The Author(s) 2020
Abstract
A real-time fingertip-gesture-based interface is still challenging for human–computer interactions, due to sensor noise, changing light levels, and the complexity of tracking a fingertip across a variety of subjects. Using fingertip tracking as a virtual mouse is a popular method of interacting with computers without a mouse device. In this work, we propose a novel virtualmouse method using RGB-D images and fingertip detection. The hand region of interest and the center of the palm are first extracted using in-depth skeleton-joint information images from a Microsoft Kinect Sensor version 2, and then converted into a binary image. Then, the contours of the hands are extracted and described by a border-tracing algorithm. The Kcosine algorithm is used to detect the fingertip location, based on the hand-contour coordinates. Finally, the fingertip location is mapped to RGB images to control the mouse cursor based on a virtual screen. The system tracks fingertips in real-time at 30 FPS on a desktop computer using a single CPU and Kinect V2. The experimental results showed a high accuracy level; the system can work well in real-world environments with a single CPU. This fingertip-gesture-based interface allows humans to easily interact with computers by hand. Keywords Virtual mouse . RGB-D images . Fingertip detection and tracking . Fingertip-gesturebased interface . Kinect . Human–computer interaction (HCI)
1 Introduction With the development of augmented-reality technology, researchers are working to reduce people’s workload while increasing their productivity by studying human–computer interactions (HCI). The Natural User Interface (NUI) of hand-gesture recognition is an important topic in HCI. Hand-gesture-based interfaces allow humans to interact with a computer in the most natural way, typically by using fingertip movements.
* Hyung-Jeong Yang [email protected] Extended author information available on the last page of the article
Multimedia Tools and Applications
Fingertip detection is broadly applied in practical applications, e.g., virtual mice, remote controls, sign-language recognition, or immersive gaming technology. Therefore, virtual mouse control by fingertip detection from images has been one of the main goals of visionbased technology in the last decades, especially with traditional red-green-blue (RGB) cameras [1, 17, 19, 25, 31]. However, even with RGB cameras, most existing algorithms [1, 17, 19, 25, 31] tend to fail when faced with changing light levels, complex backgrounds, multiple people, or background or foreground movements during the hand tracking. Microsoft’s Kinect RGB with depth (RGB-D) camera [8] has extended depth-sensing technology and interfaces for humanmotion analysis applications [4, 14, 15]. Some systems use d
Data Loading...
