Learned optical flow for intra-operative tracking of the retinal fundus
- PDF / 1,201,936 Bytes
- 10 Pages / 595.276 x 790.866 pts Page_size
- 112 Downloads / 170 Views
ORIGINAL ARTICLE
Learned optical flow for intra-operative tracking of the retinal fundus Claudio S. Ravasio1,2 · Theodoros Pissas1,2 · Edward Bloch3 · Blanca Flores3 · Sepehr Jalali1 · Danail Stoyanov1 · Jorge M. Cardoso2 · Lyndon Da Cruz3 · Christos Bergeles2 Received: 16 November 2019 / Accepted: 3 April 2020 / Published online: 22 April 2020 © The Author(s) 2020
Abstract Purpose Sustained delivery of regenerative retinal therapies by robotic systems requires intra-operative tracking of the retinal fundus. We propose a supervised deep convolutional neural network to densely predict semantic segmentation and optical flow of the retina as mutually supportive tasks, implicitly inpainting retinal flow information missing due to occlusion by surgical tools. Methods As manual annotation of optical flow is infeasible, we propose a flexible algorithm for generation of large synthetic training datasets on the basis of given intra-operative retinal images. We evaluate optical flow estimation by tracking a grid and sparsely annotated ground truth points on a benchmark of challenging real intra-operative clips obtained from an extensive internally acquired dataset encompassing representative vitreoretinal surgical cases. Results The U-Net-based network trained on the synthetic dataset is shown to generalise well to the benchmark of real surgical videos. When used to track retinal points of interest, our flow estimation outperforms variational baseline methods on clips containing tool motions which occlude the points of interest, as is routinely observed in intra-operatively recorded surgery videos. Conclusions The results indicate that complex synthetic training datasets can be used to specifically guide optical flow estimation. Our proposed algorithm therefore lays the foundation for a robust system which can assist with intra-operative tracking of moving surgical targets even when occluded. Keywords Synthetic data · Optical flow · Retinal tracking · Deep learning
Introduction Vitreoretinal surgery takes place within the gel-like vitreous humour of the eye, on top of the retinal surface, using a variety of tools of less than 0.7 mm diameter. The tools are inserted through trocar ports placed on the sclera, the white part of the eye (see Fig. 1). The surgeon operates without force perception, relying primarily on visual cues and feedback from stereo biomicroscopy providing a high-resolution view of the retinal surface. Recently, this en-face 2D view has been coupled with intra-operative optical coherence tomography Lyndon Da Cruz and Christos Bergeles have contributed equally to this work.
B
Claudio S. Ravasio [email protected]
1
University College London, London, UK
2
King’s College London, London, UK
3
Moorfields Eye Hospital NHS Foundation Trust, London, UK
(iOCT) as a complimentary imaging modality providing cross-sectional information of retinal layers. Common vitreoretinal surgical interventions like vitrectomy, epiretinal membrane peeling, laser delivery, etc. are routinely performed with very high s
Data Loading...
