Using conditional generative adversarial networks to reduce the effects of latency in robotic telesurgery
- PDF / 2,278,338 Bytes
- 7 Pages / 595.276 x 790.866 pts Page_size
- 90 Downloads / 182 Views
ORIGINAL ARTICLE
Using conditional generative adversarial networks to reduce the effects of latency in robotic telesurgery Neil Sachdeva1,2 · Misha Klopukh1 · Rachel St. Clair1 · William Edward Hahn1 Received: 22 July 2020 / Accepted: 23 September 2020 © Springer-Verlag London Ltd., part of Springer Nature 2020
Abstract The introduction of surgical robots brought about advancements in surgical procedures. The applications of remote telesurgery range from building medical clinics in underprivileged areas, to placing robots abroad in military hot-spots where accessibility and diversity of medical experience may be limited. Poor wireless connectivity may result in a prolonged delay, referred to as latency, between a surgeon’s input and action which a robot takes. In surgery, any micro-delay can injure a patient severely and, in some cases, result in fatality. One way to increase safety is to mitigate the effects of latency using deep learning aided computer vision. While the current surgical robots use calibrated sensors to measure the position of the arms and tools, in this work, we present a purely optical approach that provides a measurement of the tool position in relation to the patient’s tissues. This research aimed to produce a neural network that allowed a robot to detect its own mechanical manipulator arms. A conditional generative adversarial network (cGAN) was trained on 1107 frames of a mock gastrointestinal robotic surgery from the 2015 EndoVis Instrument Challenge and corresponding hand-drawn labels for each frame. When run on new testing data, the network generated near-perfect labels of the input images which were visually consistent with the hand-drawn labels and was able to do this in 299 ms. These accurately generated labels can then be used as simplified identifiers for the robot to track its own controlled tools. These results show potential for conditional GANs as a reaction mechanism, such that the robot can detect when its arms move outside the operating area in a patient. This system allows for more accurate monitoring of the position of surgical instruments in relation to the patient’s tissue, increasing safety measures that are integral to successful telesurgery systems. Keywords Conditional generative adversarial networks · Robotic surgery · Medical image · Image segmentation · Remote surgery
Introduction Surgical robots, such as the da Vinci Surgical System, allow for surgeons to perform minimally invasive surgeries with pinpoint accuracy and complete maneuverability. In a typical * Neil Sachdeva [email protected] Misha Klopukh [email protected] Rachel St. Clair [email protected] William Edward Hahn [email protected] 1
Florida Atlantic University, Boca Raton, FL, USA
Pine Crest School, Science Research, Fort Lauderdale, USA
2
robotic surgery system, the surgeon’s console is directly wired to the robot and a screen that shows a live feed of the robotic arms inside the patient. For surgical robots to have full reliability in a remote setup far from the op
Data Loading...
