Uncertainty-Aware Organ Classification for Surgical Data Science Applications in Laparoscopy

Objective: Surgical data science is evolving into a research field that aims to observe everything occurring within and around the treatment process to provide situation-aware data-driven assistance. In the context of endoscopic video analysis, the accurate classification of organs in the field of view of the camera proffers a technical challenge. Herein, we propose a new approach to anatomical structure classification and image tagging that features an intrinsic measure of confidence to estimate its own performance with high reliability and which can be applied to both RGB and multispectral imaging (MI) data. Methods: Organ recognition is performed using a superpixel classification strategy based on textural and reflectance information. Classification confidence is estimated by analyzing the dispersion of class probabilities. Assessment of the proposed technology is performed through a comprehensive in vivo study with seven pigs. Results: When applied to image tagging, mean accuracy in our experiments increased from 65% (RGB) and 80% (MI) to 90% (RGB) and 96% (MI) with the confidence measure. Conclusion: Results showed that the confidence measure had a significant influence on the classification accuracy, and MI data are better suited for anatomical structure labeling than RGB data. Significance: This work significantly enhances the state of art in automatic labeling of endoscopic videos by introducing the use of the confidence metric, and by being the first study to use MI data for in vivo laparoscopic tissue classification. The data of our experiments will be released as the first in vivo MI dataset upon publication of this paper.Comment: 7 pages, 6 images, 2 table

Clopidogrel and proton pump inhibitor (PPI) interaction: separate intake and a non-omeprazole PPI the solution?

<p>Abstract</p> <p>Background</p> <p>Dual therapy with aspirin and clopidogrel increases the risk of gastrointestinal bleeding. Therefore, co-therapy with a proton pump inhibitor (PPI) is recommended by most guidelines. However, there are warnings against combining PPIs with clopidogrel because of their interactions with cytochrome P450 isoenzyme 2C19 (<it>CYP2C19</it>).</p> <p>Methods</p> <p>The effects of the combined or separate intake of 20 mg of omeprazole and 75 mg of clopidogrel on the clopidogrel-induced inhibition of platelet aggregation were measured in four healthy subjects whose <it>CYP2C19 </it>exon sequences were determined. The effects of co-therapy with 10 mg of rabeprazole were also examined.</p> <p>Results</p> <p>Two subjects showed the wild-type <it>CYP2C19 </it>sequence. The concurrent intake of omeprazole had no effect on clopidogrel-induced platelet inhibition in these subjects. Two subjects were heterozygous for the *2 allele, with predicted reduced <it>CYP2C19 </it>activity. One of them was a clopidogrel non-responder. In the second heterozygous subject, omeprazole co-therapy reduced the clopidogrel anti-platelet effect when taken simultaneously or separately. However, the simultaneous intake of rabeprazole did not reduce the clopidogrel effect.</p> <p>Conclusion</p> <p>The clopidogrel-PPI interaction does not seem to be a PPI class effect. Rabeprazole did not affect the clopidogrel effect in a subject with a clear omeprazole-clopidogrel interaction. The separate intake of PPI and clopidogrel may not be sufficient to prevent their interaction.</p

Tissue classification for laparoscopic image understanding based on multispectral texture analysis.

Intraoperative tissue classification is one of the prerequisites for providing context-aware visualization in computer-assisted minimally invasive surgeries. As many anatomical structures are difficult to differentiate in conventional RGB medical images, we propose a classification method based on multispectral image patches. In a comprehensive ex vivo study through statistical analysis, we show that (1) multispectral imaging data are superior to RGB data for organ tissue classification when used in conjunction with widely applied feature descriptors and (2) combining the tissue texture with the reflectance spectrum improves the classification performance. The classifier reaches an accuracy of 98.4% on our dataset. Multispectral tissue analysis could thus evolve as a key enabling technique in computer-assisted laparoscopy

Physiological parameter estimation from multispectral images unleashed

Multispectral imaging in laparoscopy can provide tissue reflectance measurements for each point in the image at multiple wavelengths of light. These reflectances encode information on important physiological parameters not visible to the naked eye. Fast decoding of the data during surgery, however, remains challenging. While model-based methods suffer from inaccurate base assumptions, a major bottleneck related to competing machine learning-based solutions is the lack of labelled training data. In this paper, we address this issue with the first transfer learning-based method to physiological parameter estimation from multispectral images. It relies on a highly generic tissue model that aims to capture the full range of optical tissue parameters that can potentially be observed in vivo. Adaptation of the model to a specific clinical application based on unlabelled in vivo data is achieved using a new concept of domain adaptation that explicitly addresses the high variance often introduced by conventional covariance-shift correction methods. According to comprehensive in silico and in vivo experiments our approach enables accurate parameter estimation for various tissue types without the need for incorporating specific prior knowledge on optical properties and could thus pave the way for many exciting applications in multispectral laparoscopy

A learning robot for cognitive camera control in minimally invasive surgery

Background!#!We demonstrate the first self-learning, context-sensitive, autonomous camera-guiding robot applicable to minimally invasive surgery. The majority of surgical robots nowadays are telemanipulators without autonomous capabilities. Autonomous systems have been developed for laparoscopic camera guidance, however following simple rules and not adapting their behavior to specific tasks, procedures, or surgeons.!##!Methods!#!The herein presented methodology allows different robot kinematics to perceive their environment, interpret it according to a knowledge base and perform context-aware actions. For training, twenty operations were conducted with human camera guidance by a single surgeon. Subsequently, we experimentally evaluated the cognitive robotic camera control. A VIKY EP system and a KUKA LWR 4 robot were trained on data from manual camera guidance after completion of the surgeon's learning curve. Second, only data from VIKY EP were used to train the LWR and finally data from training with the LWR were used to re-train the LWR.!##!Results!#!The duration of each operation decreased with the robot's increasing experience from 1704 s ± 244 s to 1406 s ± 112 s, and 1197 s. Camera guidance quality (good/neutral/poor) improved from 38.6/53.4/7.9 to 49.4/46.3/4.1% and 56.2/41.0/2.8%.!##!Conclusions!#!The cognitive camera robot improved its performance with experience, laying the foundation for a new generation of cognitive surgical robots that adapt to a surgeon's needs

Comparative validation of single-shot optical techniques for laparoscopic 3-D surface reconstruction

Intra-operative imaging techniques for obtaining the shape and morphology of soft-tissue surfaces in vivo are a key enabling technology for advanced surgical systems. Different optical techniques for 3-D surface reconstruction in laparoscopy have been proposed, however, so far no quantitative and comparative validation has been performed. Furthermore, robustness of the methods to clinically important factors like smoke or bleeding has not yet been assessed. To address these issues, we have formed a joint international initiative with the aim of validating different state-of-the-art passive and active reconstruction methods in a comparative manner. In this comprehensive in vitro study, we investigated reconstruction accuracy using different organs with various shape and texture and also tested reconstruction robustness with respect to a number of factors like the pose of the endoscope as well as the amount of blood or smoke present in the scene. The study suggests complementary advantages of the different techniques with respect to accuracy, robustness, point density, hardware complexity and computation time. While reconstruction accuracy under ideal conditions was generally high, robustness is a remaining issue to be addressed. Future work should include sensor fusion and in vivo validation studies in a specific clinical context. To trigger further research in surface reconstruction, stereoscopic data of the study will be made publically available at www.open-CAS.com upon publication of the paper

A reporting and analysis framework for structured evaluation of COVID-19 clinical and imaging data

The COVID-19 pandemic has worldwide individual and socioeconomic consequences. Chest computed tomography has been found to support diagnostics and disease monitoring. A standardized approach to generate, collect, analyze, and share clinical and imaging information in the highest quality possible is urgently needed. We developed systematic, computer-assisted and context-guided electronic data capture on the FDA-approved mint LesionTM software platform to enable cloud-based data collection and real-time analysis. The acquisition and annotation include radiological findings and radiomics performed directly on primary imaging data together with information from the patient history and clinical data. As proof of concept, anonymized data of 283 patients with either suspected or confirmed SARS-CoV-2 infection from eight European medical centers were aggregated in data analysis dashboards. Aggregated data were compared to key findings of landmark research literature. This concept has been chosen for use in the national COVID-19 response of the radiological departments of all university hospitals in Germany

Surgical Data Science - from Concepts toward Clinical Translation

Recent developments in data science in general and machine learning in particular have transformed the way experts envision the future of surgery. Surgical Data Science (SDS) is a new research field that aims to improve the quality of interventional healthcare through the capture, organization, analysis and modeling of data. While an increasing number of data-driven approaches and clinical applications have been studied in the fields of radiological and clinical data science, translational success stories are still lacking in surgery. In this publication, we shed light on the underlying reasons and provide a roadmap for future advances in the field. Based on an international workshop involving leading researchers in the field of SDS, we review current practice, key achievements and initiatives as well as available standards and tools for a number of topics relevant to the field, namely (1) infrastructure for data acquisition, storage and access in the presence of regulatory constraints, (2) data annotation and sharing and (3) data analytics. We further complement this technical perspective with (4) a review of currently available SDS products and the translational progress from academia and (5) a roadmap for faster clinical translation and exploitation of the full potential of SDS, based on an international multi-round Delphi process