Brazilian Sign Language Recognition Using Kinect
The simultaneous-sequential nature of sign language production, which employs hand gestures and body motions combined with facial expressions, still challenges sign language recognition algorithms. This paper presents a method to recognize Brazilian Sign
- PDF / 3,034,569 Bytes
- 12 Pages / 439.37 x 666.142 pts Page_size
- 60 Downloads / 204 Views
tract. The simultaneous-sequential nature of sign language production, which employs hand gestures and body motions combined with facial expressions, still challenges sign language recognition algorithms. This paper presents a method to recognize Brazilian Sign Language (Libras) using Kinect. Skeleton information is used to segment sign gestures from a continuous stream, while depth information is used to provide distinctive features. The method was assessed in a new data-set of 107 medical signs selected from common dialogues in health-care centers. The dynamic time warping–nearest neighbor (DTW-kNN) classifier using the leave-one-out cross-validation strategy reported outstanding results. Keywords: Sign language · Isolated sign language recognition · Brazilian Sign Language · Libras · Dynamic time warping · k–Nearest Neighbor
1
Introduction
In daily life, deaf and hearing impaired people use sign language as a communication system [1]. Sign language combines hand gestures, body postures, and facial expressions to convey meaning. The richness of sign language lexicon allows, as any other language, the expression of concepts, ideas, feelings, mood, or thoughts. Contrary to popular belief, sign language is not a universal language. There are many different sign languages around the world, for instance, the American Sign Language (ASL) in United States, British Sign Language (BSL) in England, Brazilian Sign Language (Libras) in Brazil. Furthermore, different countries that have the same spoken language may have their own sign language, e.g., although United States and England share the English as common oral language, ASL differs from BSL. Despite sign language capabilities to communicate messages, there is a strong barrier between deaf and hearing people. This language barrier arises because deaf people usually do not master spoken and written language and only few hearing people can communicate using sign language. Aiming to reduce this language barrier, research efforts have been conducted in sign language recognition (SLR) [2–4]. Automatic SLR systems translate sign language into text and c Springer International Publishing Switzerland 2016 G. Hua and H. J´ egou (Eds.): ECCV 2016 Workshops, Part II, LNCS 9914, pp. 391–402, 2016. DOI: 10.1007/978-3-319-48881-3 27
392
J.E. Yauri Vidal´ on and J.M. De Martino
can improve the interaction between deaf and hearing people. Critical situations where the communication is decisive, such as the access to emergency health services, may greatly benefit from automatic sign language technologies. Currently, powered by new sensing technologies, new promising SLR approaches are being developed. The advent of depth cameras [5], also known as RGB-D cameras, has been an important milestone in the computer vision community because they can provide multimodal data, such as RGB or color images, depth range images, body skeleton, and user silhouettes, that can help to overcome the traditional restrictions of illumination changes and cluttered background of SLR systems based on traditi
Data Loading...
