FetNet: a recurrent convolutional network for occlusion identification in fetoscopic videos
- PDF / 2,068,471 Bytes
- 11 Pages / 595.276 x 790.866 pts Page_size
- 87 Downloads / 160 Views
ORIGINAL ARTICLE
FetNet: a recurrent convolutional network for occlusion identification in fetoscopic videos Sophia Bano1 · Francisco Vasconcelos1 · Emmanuel Vander Poorten2 · Tom Vercauteren3 · Sebastien Ourselin3 · Jan Deprest4 · Danail Stoyanov1 Received: 16 November 2019 / Accepted: 10 April 2020 © The Author(s) 2020
Abstract Purpose Fetoscopic laser photocoagulation is a minimally invasive surgery for the treatment of twin-to-twin transfusion syndrome (TTTS). By using a lens/fibre-optic scope, inserted into the amniotic cavity, the abnormal placental vascular anastomoses are identified and ablated to regulate blood flow to both fetuses. Limited field-of-view, occlusions due to fetus presence and low visibility make it difficult to identify all vascular anastomoses. Automatic computer-assisted techniques may provide better understanding of the anatomical structure during surgery for risk-free laser photocoagulation and may facilitate in improving mosaics from fetoscopic videos. Methods We propose FetNet, a combined convolutional neural network (CNN) and long short-term memory (LSTM) recurrent neural network architecture for the spatio-temporal identification of fetoscopic events. We adapt an existing CNN architecture for spatial feature extraction and integrated it with the LSTM network for end-to-end spatio-temporal inference. We introduce differential learning rates during the model training to effectively utilising the pre-trained CNN weights. This may support computer-assisted interventions (CAI) during fetoscopic laser photocoagulation. Results We perform quantitative evaluation of our method using 7 in vivo fetoscopic videos captured from different human TTTS cases. The total duration of these videos was 5551 s (138,780 frames). To test the robustness of the proposed approach, we perform 7-fold cross-validation where each video is treated as a hold-out or test set and training is performed using the remaining videos. Conclusion FetNet achieved superior performance compared to the existing CNN-based methods and provided improved inference because of the spatio-temporal information modelling. Online testing of FetNet, using a Tesla V100-DGXS-32GB GPU, achieved a frame rate of 114 fps. These results show that our method could potentially provide a real-time solution for CAI and automating occlusion and photocoagulation identification during fetoscopic procedures. Keywords Deep learning · Surgical vision · Twin-to-twin transfusion syndrome (TTTS) · Fetoscopy · Video segmentation · Computer assisted interventions (CAI)
Introduction Twin-to-twin transfusion syndrome (TTTS) is a fetal anomaly affecting 10−15% of identical twins sharing a monochori-
onic placenta [3]. It is caused by the presence of abnormal placental vessels that disproportionately transfuse blood from one twin to the other. The recipient of excessive blood is
Electronic supplementary material The online version of this article (https://doi.org/10.1007/s11548-020-02169-0) contains supplementary material, which is available to authorized use
Data Loading...
