Temporal variability of surgical technical skill perception in real robotic surgery
- PDF / 555,860 Bytes
- 7 Pages / 595.276 x 790.866 pts Page_size
- 98 Downloads / 112 Views
ORIGINAL ARTICLE
Temporal variability of surgical technical skill perception in real robotic surgery Jason D. Kelly1
· Michael Nash2 · Nicholas Heller3 · Thomas S. Lendvay4 · Timothy M. Kowalewski1
Received: 29 April 2020 / Accepted: 19 August 2020 © CARS 2020
Abstract Purpose Summary score metrics, either from crowds of non-experts, faculty surgeons or from automated performance metrics, have been trusted as the prevailing method of reporting surgeon technical skill. The aim of this paper is to learn whether there exist significant fluctuations in the technical skill assessments of a surgeon throughout long durations of surgical footage. Methods A set of 12 videos of robotic surgery cases from common human patient robotic surgeries were used to evaluate the perceived technical skill at each individual minute of the surgical videos, which were originally 12–15 min in length. A linear mixed-effects model for each video was used to compare the ratings of each minute to those from every other minute in order to learn whether a change in scores over time can be detected and reliably measured apart from inter- and intrarater variation. Results Modeling the change over time of the global evaluative assessment of robotic skills scores significantly contributed to the prediction models for 11 of the 12 surgeons. This demonstrates that measurable changes in technical skill occur over time during robotic surgery. Conclusion The findings from this research raise questions about the optimal duration of footage needed to be evaluated to arrive at an accurate rating of surgical technical skill for longer procedures. This may imply non-negligible label noise for supervised machine learning approaches. In the future, it may be necessary to report a surgeon’s skill variability in addition to their mean score to have proper knowledge of a surgeon’s overall skill level. Keywords Crowd sourcing · Surgical technical skill · Video segmentation · Bias
Introduction Methods for assessing the technical skills of surgeons is paramount to ensuring to the public that surgeons are safe and effective. For years, summary scores or metrics have been used as the main method to report surgical skill. The most popular of these has been to use a Likert-scale scoring metric,
B
Jason D. Kelly [email protected]
1
Department of Mechanical Engineering, University of Minnesota, Minneapolis, MN, USA
2
Department of Biostatistics, University of Washington, Seattle, WA, USA
3
Department of Computer Science and Engineering, University of Minnesota, Minneapolis, MN, USA
4
Department of Urology, University of Washington, Seattle, WA, USA
from either non-expert crowds or from faculty surgeons. Past research has shown that crowds of non-experts concord with surgeon raters in evaluating technical skill [1]. Automated performance metrics have also been used, in which surgical events or streaming kinematic data have been used for computation of various metrics, suggesting superior objectivity [2]. As it is known that surgical skill is related to patient
Data Loading...
