Benchmarking deep neural network approaches for Indian Sign Language recognition
- PDF / 1,554,742 Bytes
- 12 Pages / 595.276 x 790.866 pts Page_size
- 39 Downloads / 266 Views
(0123456789().,-volV)(0123456789(). ,- volV)
ORIGINAL ARTICLE
Benchmarking deep neural network approaches for Indian Sign Language recognition Ashish Sharma1 • Nikita Sharma1 • Yatharth Saxena1 • Anuraj Singh1 • Debanjan Sadhya1 Received: 23 April 2020 / Accepted: 14 October 2020 Springer-Verlag London Ltd., part of Springer Nature 2020
Abstract Sign language is the language of the deaf and mute. However, this particular population of the world is unfortunately overlooked as the majority of the hearing population does not understand sign language. In this paper, an extensive comparative analysis of various gesture recognition techniques involving convolutional neural networks and machine learning algorithms has been discussed and tested for real-time accuracy. Three models: a pre-trained VGG16 with finetuning, VGG16 with transfer learning and a hierarchical neural network were analyzed based on a number of trainable parameters. These models were trained on a self-developed dataset consisting images of Indian Sign Language (ISL) representation of all 26 English alphabets. The performance evaluation was based on the practical application of these models, which was simulated by varying lighting and background environments. Out of the three, the hierarchical model outperformed the other two models to give the best accuracy of 98.52% for one-hand and 97% for two-hand gestures. Thereafter, a conversation interface was built in Django using this model for the real-time gesture to speech conversion and vice versa. This publicly accessible interface can be used by anyone who wishes to learn or converse in ISL. Keywords Indian Sign Language recognition Convolutional neural networks Hierarchical network
1 Introduction In this fast-growing world, there is an urgent need to uplift the challenged sections of the society. People suffering from speech disabilities communicate in sign language and therefore face trouble in connecting with the able-bodies. Thus, there exists a communication gap that is difficult to mitigate into the mainstream society. According to the estimation of the Indian National Association of the deaf, & Debanjan Sadhya [email protected] Ashish Sharma [email protected] Nikita Sharma [email protected] Yatharth Saxena [email protected] Anuraj Singh [email protected] 1
ABV-Indian Institute of Information Technology and Management Gwalior, Gwalior, India
around a million people suffer from some form of functional hearing loss1. This research work targets the hearing impaired population of India. It attempts to develop a machine learning-based conversation interface that uses both one-hand and two-hand gestures for communication. This work can be used for assisting differently-abled people in conversing with others by using Indian Sign Language (ISL). ISL is a two-hand gesture language. It is a language that is spoken with the facilitation of hand movements, facial expressions and body language. There exist many languages and dialects in India since it is a diverse country. Howev
Data Loading...
