Proposing novel methods for gynecologic surgical action recognition on laparoscopic videos
- PDF / 1,481,697 Bytes
- 23 Pages / 439.37 x 666.142 pts Page_size
- 16 Downloads / 164 Views
Proposing novel methods for gynecologic surgical action recognition on laparoscopic videos Toktam Khatibi 1
& Parastoo Dezyani
1
Received: 8 February 2020 / Revised: 15 July 2020 / Accepted: 4 August 2020 # Springer Science+Business Media, LLC, part of Springer Nature 2020
Abstract
Laparoscopy or minimally-invasive surgery (MIS) is performed by inserting a camera called endoscope inside the body to display the surgical actions online with the ability to record and archive the video. Recognizing the surgical actions automatically from the laparoscopic videos have many applications such as surgical skill assessment, teaching purposes, and workflow recognition but is a challenging task. The main aim of this study is proposing novel automatic methods for surgical action recognition from the laparoscopic video frames. For this purpose, three different scenarios are designed, evaluated and compared using 5-fold cross validation strategy. The first and the second scenarios are based on deep neural networks and combination of pre-trained CNNs and conventional machine learning models, respectively. The last scenario combines handcraft feature extraction, pre-trained CNNs, feature engineering based on complex networks and conventional classifiers. Dataset analyzed in this study is ITEC LapGyn4 Gynecologic Laparoscopy Image dataset. Experimental results show that the second and the third scenarios have highly desirable performance for multi-instance surgical action recognition with the average accuracy of 99.20 and AUC of 99.12. On the other hand, for singleinstance surgical action recognition, the third scenario outperforms the compared ones with the average accuracy of 99.05 and AUC of 96.41. Moreover, different feature sets in the third scenario are ranked and assigned the importance score based on “Mean Decrease of Accuracy” measure. The first-ranked features are the deep features extracted from our proposed CNNs in the first scenario and the second-ranked ones are the features engineered from the complex networks. Keywords Minimally-invasive surgery (MIS) . Medical image processing . Multi-instance classification . Deep neural networks . Wrapper feature selection . Feature engineering
* Toktam Khatibi [email protected]
1
School of Industrial and Systems Engineering, Tarbiat Modares University, Tehran, Iran
Multimedia Tools and Applications
1 Introduction Laparoscopy or minimally-invasive surgery (MIS) is performed by inserting a camera called endoscope or telescope inside the body to display the tissues, surgical instruments and the surgical actions online via a monitor with tunable magnification [20]. It has therapeutic benefits and many advantages compared to open surgeries [6]. Moreover, the camera can record the video of the surgery to be used as retrospective data for many purposes [20]. Laparoscopic surgery provides the ability to record and archive the video during the surgical operation, which can be used for education of junior surgeons and medical experts, providing a patient copy for fur
Data Loading...
