Confocal Laser Endomicroscopy Image Analysis with Deep Convolutional Neural Networks

abstract: Rapid intraoperative diagnosis of brain tumors is of great importance for planning treatment and guiding the surgeon about the extent of resection. Currently, the standard for the preliminary intraoperative tissue analysis is frozen section biopsy that has major limitations such as tissue freezing and cutting artifacts, sampling errors, lack of immediate interaction between the pathologist and the surgeon, and time consuming. Handheld, portable confocal laser endomicroscopy (CLE) is being explored in neurosurgery for its ability to image histopathological features of tissue at cellular resolution in real time during brain tumor surgery. Over the course of examination of the surgical tumor resection, hundreds to thousands of images may be collected. The high number of images requires significant time and storage load for subsequent reviewing, which motivated several research groups to employ deep convolutional neural networks (DCNNs) to improve its utility during surgery. DCNNs have proven to be useful in natural and medical image analysis tasks such as classification, object detection, and image segmentation. This thesis proposes using DCNNs for analyzing CLE images of brain tumors. Particularly, it explores the practicality of DCNNs in three main tasks. First, off-the shelf DCNNs were used to classify images into diagnostic and non-diagnostic. Further experiments showed that both ensemble modeling and transfer learning improved the classifier’s accuracy in evaluating the diagnostic quality of new images at test stage. Second, a weakly-supervised learning pipeline was developed for localizing key features of diagnostic CLE images from gliomas. Third, image style transfer was used to improve the diagnostic quality of CLE images from glioma tumors by transforming the histology patterns in CLE images of fluorescein sodium-stained tissue into the ones in conventional hematoxylin and eosin-stained tissue slides. These studies suggest that DCNNs are opted for analysis of CLE images. They may assist surgeons in sorting out the non-diagnostic images, highlighting the key regions and enhancing their appearance through pattern transformation in real time. With recent advances in deep learning such as generative adversarial networks and semi-supervised learning, new research directions need to be followed to discover more promises of DCNNs in CLE image analysis.Dissertation/ThesisDoctoral Dissertation Neuroscience 201

Surgical Phase Recognition of Short Video Shots Based on Temporal Modeling of Deep Features

Recognizing the phases of a laparoscopic surgery (LS) operation form its video constitutes a fundamental step for efficient content representation, indexing and retrieval in surgical video databases. In the literature, most techniques focus on phase segmentation of the entire LS video using hand-crafted visual features, instrument usage signals, and recently convolutional neural networks (CNNs). In this paper we address the problem of phase recognition of short video shots (10s) of the operation, without utilizing information about the preceding/forthcoming video frames, their phase labels or the instruments used. We investigate four state-of-the-art CNN architectures (Alexnet, VGG19, GoogleNet, and ResNet101), for feature extraction via transfer learning. Visual saliency was employed for selecting the most informative region of the image as input to the CNN. Video shot representation was based on two temporal pooling mechanisms. Most importantly, we investigate the role of 'elapsed time' (from the beginning of the operation), and we show that inclusion of this feature can increase performance dramatically (69% vs. 75% mean accuracy). Finally, a long short-term memory (LSTM) network was trained for video shot classification based on the fusion of CNN features with 'elapsed time', increasing the accuracy to 86%. Our results highlight the prominent role of visual saliency, long-range temporal recursion and 'elapsed time' (a feature so far ignored), for surgical phase recognition.Comment: 6 pages, 4 figures, 6 table

The real-time molecular characterisation of human brain tumours during surgery using Rapid Evaporative Ionization Mass Spectrometry [REIMS] and Raman spectroscopy: a platform for precision medicine in neurosurgery

Aim: To investigate new methods for the chemical detection of tumour tissue during neurosurgery. Rationale: Surgeons operating on brain tumours currently lack the ability to directly and immediately assess the presence of tumour tissue to help guide resection. Through developing a first in human application of new technology we hope to demonstrate the proof of concept that chemical detection of tumour tissue is possible. It will be further demonstrated that information can be obtained to potentially aid treatment decisions. This new technology could, therefore, become a platform for more effective surgery and introducing precision medicine to Neurosurgery. Methods: Molecular analysis was performed using Raman spectroscopy and Rapid Evaporative Ionization Mass Spectrometry (REIMS). These systems were first developed for use in brain surgery. A single centre prospective observational study of both modalities was designed involving a total of 75 patients undergoing craniotomy and resection of a range of brain tumours. A neuronavigation system was used to register spectral readings in 3D space. Precise intraoperative readings from different tumour zones were taken and compared to matched core biopsy samples verified by routine histopathology. Results: Multivariate statistics including PCA/LDA analysis was used to analyse the spectra obtained and compare these to the histological data. The systems identified normal versus tumour tissue, tumour grade, tumour type, tumour density and tissue status of key markers of gliomagenesis. Conclusions: The work in this thesis provides proof of concept that useful real time intraoperative spectroscopy is possible. It can integrate well with the current operating room setup to provide key information which could potentially enhance surgical safety and effectiveness in increasing extent of resection. The ability to group tissue samples with respect to genomic data opens up the possibility of using this information during surgery to speed up treatment, escalate/deescalate surgery in specific phenotypic groups to introduce precision medicine to Neurosurgery.Open Acces

Identifying risk patterns for suicide attempts in individuals with diabetes : a data-driven approach using LASSO regression

Diabetes is a major health concern in the United States, with 34.2 million Americans affected in 2020. Unfortunately, the risk of suicide is also elevated in individuals with diabetes, with around 90,000 people with diabetes committing suicide each year. People with type 1 diabetes are three to four times more likely to attempt suicide, and those with newly diagnosed type 2 diabetes are twice as likely to attempt suicide compared to the general population. However, poor mental health comorbidity is still neglected, and more recommendations are needed to support for people with diabetes. It is widely acknowledged that the comorbidity of depression with diabetes is considered a higher risk factor for suicide attempts Previous studies have used logistic regression to identify risk factors for suicide attempts in individuals with diabetes. However, this technique can be prone to overfitting when the number of variables is high. To address this issue, we used the LASSO (Least Absolute Shrinkage and Selection Operator), a regularization technique, to reduce overfitting in a logistic regression model. It works by adding a penalty term ([lambda]) to the log-likelihood function, which shrinks the estimates of the coefficients. This process allows LASSO to act as a feature selection method, effectively setting coefficients that contribute most to the error to zero. Because few studies have focused on un derstanding the relationship between suicide attempts and diabetes, we used association rule mining ARM an explainable rule based machine learning technique, for knowledge discovery to reveal previously unknown relationships between suicide attempts and diabetes. This approach has already proved useful in the medical field, where it has been applied to electronic health record (EHR) data to discover associations such as disease co-occurrences, drug-disease associations, and symptomatic patterns of disease. However, no previous studies have used ARM to determine risk factors and predict suicide attempts in people with diabetes. The aim of this dissertation is to identify patterns of risk factors for suicide attempts in individuals with diabetes, with the long term goal of developing a clinical decision support system that can be integrated into EHRs. This system would allow healthcare providers to identify patients with diabetes at high risk of suicide attempts and provide appropriate preventive measures during outpatient clinic visits. To achieve this goal, we have three specific aims: (1) to identify potential risk factors for suicide attempts in individuals with diabetes through a literature review; (2) to investigate risk factors for suicide attempts in individuals with diabetes using LASSO regression; (3) to identify risk patterns for suicide attempts in individuals with diabetes using association rule mining. In this dissertation, we have reviewed the literature and compiled a list of data elements for suicide attempts in people with diabetes. We then retrieved data on patients with diabetes from Cerner Real-World Data [trade mark]. LASSO regression was used for feature selection, and ARM was used for investigating the risk patterns. We discovered risk patterns that are understandable and practical for healthcare providers. The findings of this research can inform suicide prevention efforts for people with diabetes and contribute to improved mental health outcomes.Includes bibliographical references

DATA-DRIVEN APPROACH TO IMAGE CLASSIFICATION

Image classification has been a core topic in the computer vision community. Its recent success with convolutional neural network (CNN) algorithm has led to various real world applications such as large scale management of photos/videos on cloud/social-media, image based search for online retailers, self-driving cars, building robots and healthcare. Image classification can be broadly categorized into binary, multi-class and multi-label classification problems. Binary classification involves assigning one of the two class labels to an instance. In multi-class classification problem, an instance should be categorized into one of more than two classes. Multi-label classification is a generalized version of the multi-class classification problem where each image is assigned multiple labels as opposed to a single label. In this work, we first present various methods that take advantage of deep representations (fully connected layer of pre-trained CNN on the ImageNet dataset) and yield better performance on multi-label classification when compared to methods that use over a dozen conventional visual features. Following the success of deep representations, we intend to build a generic end-to-end deep learning framework to address all three problem categories of image classification. However, there are still no well established guidelines (in terms of choosing the number of layers to go deeper, the number of kernels and the size, the type of regularizer, the choice of non-linear function, etc.) to build an efficient deep neural network and often network architecture design is specific to a problem/dataset. Hence, we present some initial efforts in building a computational framework called Deep Decision Network (DDN) which is completely data-driven. DDN is a tree-like structured built stage-wise. During the learning phase, starting from the root network node, DDN automatically builds a network that splits the data into disjoint clusters of classes which would be handled by the subsequent expert networks. This results in a tree-like structured network driven by the data. The proposed approach provides an insight into the data by identifying the group of classes that are hard to classify and require more attention when compared to others. This feature is crucial for people trying to solve the problem with little or no domain knowledge, especially for applications in medical domain. Initially, we evaluate DDN on a binary classification problem and later extend it to more challenging multi-class and multi-label classification problems. The extension of DDN to multi-class and multi-label involves some changes but they still operate under the same underlying principle. In all the three cases, the proposed approach is tested for its recognition performance and scalability on publicly available datasets providing comparison to other methods

Context aware decision support in neurosurgical oncology based on an efficient classification of endomicroscopic data

Purpose: Probe-based confocal laser endomicroscopy (pCLE) enables in vivo, in situ tissue characterisation without changes in the surgical setting and simplifies the oncological surgical workflow. The potential of this technique in identifying residual cancer tissue and improving resection rates of brain tumours has been recently verified in pilot studies. The interpretation of endomicroscopic information is challenging, particularly for surgeons who do not themselves routinely review histopathology. Also, the diagnosis can be examiner-dependent, leading to considerable inter-observer variability. Therefore, automatic tissue characterisation with pCLE would support the surgeon in establishing diagnosis as well as guide robot-assisted intervention procedures. Methods: The aim of this work is to propose a deep learning-based framework for brain tissue characterisation for context aware diagnosis support in neurosurgical oncology. An efficient representation of the context information of pCLE data is presented by exploring state-of-the-art CNN models with different tuning configurations. A novel video classification framework based on the combination of convolutional layers with long-range temporal recursion has been proposed to estimate the probability of each tumour class. The video classification accuracy is compared for different network architectures and data representation and video segmentation methods. Results: We demonstrate the application of the proposed deep learning framework to classify Glioblastoma and Meningioma brain tumours based on endomicroscopic data. Results show significant improvement of our proposed image classification framework over state-of-the-art feature-based methods. The use of video data further improves the classification performance, achieving accuracy equal to 99.49%. Conclusions: This work demonstrates that deep learning can provide an efficient representation of pCLE data and accurately classify Glioblastoma and Meningioma tumours. The performance evaluation analysis shows the potential clinical value of the technique

Efficacy and pharmacokinetics of intravenous paracetamol in the critically ill patient

Introduction: Paracetamol (PCM) is a drug with analgesic and antipyretic properties. Despite its frequent use, little is known about its efficacy and pharmacokinetics (PK) when intravenously administered in the critically ill patient. A previous study suggests that therapeutic concentrations are not always reached [1]. The primary aim of this open-label, multiple-dose study was to evaluate intravenous PCM therapy in critically ill, secondary aim was to study the PK of intravenous PCM. Methods: Ventilated patients needing PCM treatment according to our ICU protocol (1 g PCM intravenously four times daily) were eligible for inclusion. Excluded were those with severe liver failure and those treated with PCM on the time of admission to the ICU. Blood samples were collected at 0, 30, 60, 180 and 300 minutes after the first and, if possible, the fifth and 21st doses. A computerized model was used to estimate population PK. Results: Nineteen patients were included of which 13 were male, with a mean APACHE IV score of 94.8. No antipyretic effect could be measured in any of the patients. PK parameters have been calculated for all patients after the first PCM dose. The half-life was 2.2 hours, the volume of distribution was 1.03 l/kg, and the clearance was 0.33 l/kg/hour. Data from 15 patients could be analysed after the fifth dose and from five patients after the 21st dose. The PK of intravenous PCM in our population show a biphasic profile (Figure 1). One hour after the dose, the mean serum concentration level was below the therapeutic level. In 18 out of 19 patients serum concentration dropped below 5 mg/ml before the next dose, resulting in a lack of build-up of a suitable therapeutic level of PCM after multiple dosages. Conclusions: The recommended dose of 1 g intravenous PCM four times daily is not sufficient to achieve a therapeutic effect in critically ill patients. This can be explained by the low serum levels reached. These results warrant the development of an adequate dosing scheme for intravenous PCM followed by a large clinical trial studying the effects and safety of this regimen in critically ill patients