Transparent and Shaped Stiffness Reflection for Telesurgery

status: publishe

ToolNet: Holistically-Nested Real-Time Segmentation of Robotic Surgical Tools

Real-time tool segmentation from endoscopic videos is an essential part of many computer-assisted robotic surgical systems and of critical importance in robotic surgical data science. We propose two novel deep learning architectures for automatic segmentation of non-rigid surgical instruments. Both methods take advantage of automated deep-learning-based multi-scale feature extraction while trying to maintain an accurate segmentation quality at all resolutions. The two proposed methods encode the multi-scale constraint inside the network architecture. The first proposed architecture enforces it by cascaded aggregation of predictions and the second proposed network does it by means of a holistically-nested architecture where the loss at each scale is taken into account for the optimization process. As the proposed methods are for real-time semantic labeling, both present a reduced number of parameters. We propose the use of parametric rectified linear units for semantic labeling in these small architectures to increase the regularization ability of the design and maintain the segmentation accuracy without overfitting the training sets. We compare the proposed architectures against state-of-the-art fully convolutional networks. We validate our methods using existing benchmark datasets, including ex vivo cases with phantom tissue and different robotic surgical instruments present in the scene. Our results show a statistically significant improved Dice Similarity Coefficient over previous instrument segmentation methods. We analyze our design choices and discuss the key drivers for improving accuracy.Comment: Paper accepted at IROS 201

State-of-the-Art of Non-Radiative, Non-Visual Spine Sensing with a Focus on Sensing Forces, Vibrations and Bioelectrical Properties: A Systematic Review

In the research field of robotic spine surgery, there is a big upcoming momentum for surgeon-like autonomous behaviour and surgical accuracy in robotics which goes beyond the standard engineering notions such as geometric precision. The objective of this review is to present an overview of the state of the art in non-visual, non-radiative spine sensing for the enhancement of surgical techniques in robotic automation. It provides a vantage point that facilitates experimentation and guides new research projects to what has not been investigated or integrated in surgical robotics. Studies were identified, selected and processed according to the PRISMA guidelines. Relevant study characteristics that were searched for include the sensor type and measured feature, the surgical action, the tested sample, the method for data analysis and the system's accuracy of state identification. The 6DOF f/t sensor, the microphone and the electromyography probe were the most commonly used sensors in each category, respectively. The performance of the electromyography probe is unsatisfactory in terms of preventing nerve damage as it can only signal after the nerve is disturbed. Feature thresholding and artificial neural networks were the most common decision algorithms for state identification. The fusion of different sensor data in the decision algorithm improved the accuracy of state identification

Deep Sequential Mosaicking of Fetoscopic Videos

Twin-to-twin transfusion syndrome treatment requires fetoscopic laser photocoagulation of placental vascular anastomoses to regulate blood flow to both fetuses. Limited field-of-view (FoV) and low visual quality during fetoscopy make it challenging to identify all vascular connections. Mosaicking can align multiple overlapping images to generate an image with increased FoV, however, existing techniques apply poorly to fetoscopy due to the low visual quality, texture paucity, and hence fail in longer sequences due to the drift accumulated over time. Deep learning techniques can facilitate in overcoming these challenges. Therefore, we present a new generalized Deep Sequential Mosaicking (DSM) framework for fetoscopic videos captured from different settings such as simulation, phantom, and real environments. DSM extends an existing deep image-based homography model to sequential data by proposing controlled data augmentation and outlier rejection methods. Unlike existing methods, DSM can handle visual variations due to specular highlights and reflection across adjacent frames, hence reducing the accumulated drift. We perform experimental validation and comparison using 5 diverse fetoscopic videos to demonstrate the robustness of our framework.Comment: Accepted at MICCAI 201

High-Fidelity Low-Cost Synthetic Training Model for Fetoscopic Spina Bifida Repair

BACKGROUND: Fetoscopic Spina Bifida repair (fSB-repair) is increasingly being practiced, but limited skill acquisition poses a barrier to widespread adoption. Extensive training in relevant models, including both ex- and in-vivo models may help. To address this, a synthetic training model that is affordable, realistic and allows skill analysis would be useful.OBJECTIVE: To create a high-fidelity model for training the essential neurosurgical steps of fetoscopic spina bifida repair using synthetic materials. Additionally, we aimed to obtain a cheap and easily reproducible model.STUDY DESIGN: We developed a three-layered silicon-based model resembling the anatomical layers of a typical myelomeningocele lesion. It allows for filling the cyst with fluid and conducting a water tightness test post-repair. A compliant silicon ball mimics the uterine cavity, and is fixed to a solid 3D printed base. The fetal back with the lesion (single-use) is placed inside the uterine ball, which is reusable and repairable to allow practicing port insertion and fixation multiple times. Following cannula insertion, the uterus is insufflated, and clinical fetoscopic, robotic or prototype instruments can be used. Three skilled endoscopic surgeons each did six simulated fetoscopic repairs following the surgical steps of an open repair. The primary outcome was surgical success, based on water tightness of the repair, operation time &lt;180 minutes and an Objective-Structured-Assessment-of-Technical-Skills (OSATS)-score of ≥ 18/25. Skill retention was measured using a competence commulative sum (C-CUSUM) analysis on composite binary outcome for surgical success. Secondary outcomes were cost and fabrication time of the model.RESULTS: We made a model for simulating spina bifida repair neurosurgical steps with anatomical details, port insertion, placode release and descent, undermining of skin and muscular layer, and endoscopic suturing. The model is made with reusable 3D-printed molds with easily accessible materials. The one-time startup cost was 211€, and each single-use simulated MMC-lesion costs 9.5€ in materials and 50 min working hours. Two skilled endoscopic surgeons performed six simulated three-port fetoscopic repairs, while a third used a Da-Vinci surgical robot. Operation times decreased over 30% from the first to last trial. Six experiments per surgeon did not show an obvious OSATS-score improvement. C-CUSUM analysis confirmed competency for each surgeon.CONCLUSION: This high-fidelity low-cost spina bifida model allows simulated dissection and closure of a myelomeningocele lesion.</p