Sensor fusion based manipulative action recognition
- PDF / 4,573,260 Bytes
- 13 Pages / 595.276 x 790.866 pts Page_size
- 17 Downloads / 206 Views
Sensor fusion based manipulative action recognition Ye Gu1
· Meiqin Liu2 · Weihua Sheng3 · Yongsheng Ou4 · Yongqiang Li5
Received: 4 January 2019 / Accepted: 18 August 2020 © Springer Science+Business Media, LLC, part of Springer Nature 2020
Abstract Manipulative action recognition is one of the most important and challenging topic in the fields of image processing. In this paper, three kinds of sensor modules are used for motion, force and object information capture in the manipulative actions. Two fusion methods are proposed. Further, the recognition accuracy can be improved by using object as context. For the feature-level fusion method, significant features are chosen first. Then the Hidden Markov Models are built with these selected features to characterize the temporal sequence. For the decision-level fusion method, HMMs are built for each feature group. Then the decisions are fused. On top of these two fusion methods, the object/action context is modeled using Bayesian network. Assembly tasks are used for algorithm evaluation. The experimental results prove that the proposed approach is effective on manipulative action recognition task. The recognition accuracy of the decision-level, feature-level fusion methods and the Bayesian model are 72%, 80% and 90% respectively.
1 Introduction Action recognition is a task to infer human actions based on spatial and temporal information. This task has become a popular topic given its wide real-world applications, such as autonomous driving vehicle, video surveillance/retrieval, robot skill learning and etc. The potential application of the proposed action recognition approach is learning from demonstration. In order to teaching a robot delicate manipulative skills through human demonstration, one of the fundamental problems is to understand complex human manipulative actions. A manipulative action usually contains
B
Ye Gu [email protected] Meiqin Liu [email protected] Weihua Sheng [email protected]
1
Shenzhen Technology University, Shenzhen, Guangdong, China
2
College of Electrical Engineering, Zhejiang University, Hangzhou 310027, China
3
Shenzhen Academy of Robotics, Shenzhen, Guangdong, China
4
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, Guangdong, China
5
Harbin Institute of Technology, No. 92, Xidazhi Street, Harbin 150001, China
finger and arm motions and force applied to the manipulated object. For example, wrenching is mainly about wrist and elbow movement while involves minor finger motion. Screwdriving involves finger force and motion applied to screwdriver. Actions using electrical device, such as drilling is mainly about device operation using fingers. Meanwhile, the manipulated object type may also offer useful clues to the assembly actions. Motion or vision sensors are usually used to capture these information in order to recognize manipulative actions (Chen et al. 2017; Kumar and Sivaprakash 2013). Visual sensors can be used to track human body parts and capture object information howeve
Data Loading...
