Development and clinical translation of optical and software methods for endomicroscopic imaging

Endomicroscopy is an emerging technology that aims to improve clinical diagnostics by allowing for in vivo microscopy in difficult to reach areas of the body. This is most commonly achieved by using coherent fibre bundles to relay light for illumination and imaging to and from the area under investigation. Endomicroscopy’s attraction for researchers and clinicians is two-fold: on the one hand, its use can reduce the invasiveness of a diagnostic procedure by removing the need for biopsies; On the other hand, it allows for structural and functional in vivo imaging. Endomicroscopic images acquired through optical fibre bundles exhibit artefacts that deteriorate image quality and contrast. This thesis aims to improve an existing endomicroscopy imaging system by exploring two methods that mitigate these artefacts. The first, software-based method takes several processing steps from literature and implements them in an existing endomicroscopy device with a focus on real-time application to enable clinical use, after image quality was found to be inadequate without further processing. A contribution to the field is that two different approaches are implemented and compared in quantitative and qualitative means that have not been compared directly in this manner before. This first attempt at improving endomicroscopy image quality relies solely on digital image processing methods and is developed with a strong focus on real-time applicability in clinical use. Both approaches are compared on pre-clinical and clinical human imaging data. The second method targets the effect of inter-core coupling, which reduces contrast in fibre images. A parallelised confocal imaging method is developed in which a sequence of images is acquired while selectively illuminating groups of fibre cores through the use of a spatial light modulator. A bespoke algorithm creates a composite image in a final processing step. In doing so, unwanted light is detected and removed from the final image. This method is shown to reduce the negative impact of inter-core coupling on image contrast on small imaging targets, while no benefit was found in large, scattering samples

Transferability of Deep Learning Algorithms for Malignancy Detection in Confocal Laser Endomicroscopy Images from Different Anatomical Locations of the Upper Gastrointestinal Tract

Squamous Cell Carcinoma (SCC) is the most common cancer type of the epithelium and is often detected at a late stage. Besides invasive diagnosis of SCC by means of biopsy and histo-pathologic assessment, Confocal Laser Endomicroscopy (CLE) has emerged as noninvasive method that was successfully used to diagnose SCC in vivo. For interpretation of CLE images, however, extensive training is required, which limits its applicability and use in clinical practice of the method. To aid diagnosis of SCC in a broader scope, automatic detection methods have been proposed. This work compares two methods with regard to their applicability in a transfer learning sense, i.e. training on one tissue type (from one clinical team) and applying the learnt classification system to another entity (different anatomy, different clinical team). Besides a previously proposed, patch-based method based on convolutional neural networks, a novel classification method on image level (based on a pre-trained Inception V.3 network with dedicated preprocessing and interpretation of class activation maps) is proposed and evaluated. The newly presented approach improves recognition performance, yielding accuracies of 91.63% on the first data set (oral cavity) and 92.63% on a joint data set. The generalization from oral cavity to the second data set (vocal folds) lead to similar area-under-the-ROC curve values than a direct training on the vocal folds data set, indicating good generalization.Comment: Erratum for version 1, correcting the number of CLE image sequences used in one data se

Evaluating technologies in the assessment of pulmonary disease to aid lung transplantation: from ex vivo to in vivo models

INTRODUCTION: Despite technological advances, the rate of utility of potential lung donors for transplantation remains low. As a consequence, more than a third of patients listed for lung transplant will not receive one. Application of emerging technologies to select suitable donor organs may help to increase the donor organ pool. Those who receive a lung transplant remain at risk of significant morbidity and mortality and there is a lack of useful biomarkers to identify those at risk of rejection. In this thesis I aim to explore the use of new technologies currently not part of standard practice in the UK to improve the potential donor pool for transplant by (i) evaluating an ex vivo lung perfusion (EVLP) platform to physiologically assess lungs declined for transplant; (ii) developing a novel method for assessing pulmonary vascular leak using EVLP and optical fibre- based endomicroscopy; and (iii) using the optical fibre-based endomicroscopy platform in a first in human study to assess in situ enzyme activity which may be implicated in graft rejection. METHODS: Retrospective study of a national database of multiorgan donors. Prospective assessment of human lungs declined for transplant using a custom built EVLP platform. Fibered endomicroscopy in ex vivo and in vivo animal models of acute lung injury. First in human study of fibered endomicroscopic molecular imaging of in situ matrix metalloprotease (MMP) activity. RESULTS: Lung transplantation from multiorgan donors remains low in Scotland, usually due to suboptimal functional assessment. Our results suggest that further isolated ex vivo assessment may offer additional confidence in graft function. I demonstrate that it is possible to image leaked intravascular protein in situ using an ex vivo perfused acute lung injury model, however we were unable to translate this into an in vivo porcine acute lung injury model. I show that it is possible to optically image, in real-time, pulmonary enzyme activity using a bespoke molecular smartprobe and to observe drug-target engagement with an enzyme inhibitor. DISCUSSION: For those with life limiting pulmonary disease, lung transplantation may offer the only opportunity for survival. Despite increasing operative rates world-wide the vast majority of donated lungs are deemed unsuitable and as a result more than one third of those listed never receive a transplant. Those that do remain at risk of subsequent morbidity and mortality. In order to improve the rate of transplant as a lifesaving intervention and reduce the post-operative risk to recipients a rapid adoption of technological innovation is required to better characterise the lung both pre and post transplant

Assessing the utility of autofluorescence-based pulmonary optical endomicroscopy to predict the malignant potential of solitary pulmonary nodules in humans

Solitary pulmonary nodules are common, often incidental findings on chest CT scans. The investigation of pulmonary nodules is time-consuming and often leads to protracted follow-up with ongoing radiological surveillance, however, clinical calculators that assess the risk of the nodule being malignant exist to help in the stratification of patients. Furthermore recent advances in interventional pulmonology include the ability to both navigate to nodules and also to perform autofluorescence endomicroscopy. In this study we assessed the efficacy of incorporating additional information from label-free fibre-based optical endomicrosopy of the nodule on assessing risk of malignancy. Using image analysis and machine learning approaches, we find that this information does not yield any gain in predictive performance in a cohort of patients. Further advances with pulmonary endomicroscopy will require the addition of molecular tracers to improve information from this procedure

Bayesian image restoration and bacteria detection in optical endomicroscopy

Optical microscopy systems can be used to obtain high-resolution microscopic images of tissue cultures and ex vivo tissue samples. This imaging technique can be translated for in vivo, in situ applications by using optical fibres and miniature optics. Fibred optical endomicroscopy (OEM) can enable optical biopsy in organs inaccessible by any other imaging systems, and hence can provide rapid and accurate diagnosis in a short time. The raw data the system produce is difficult to interpret as it is modulated by a fibre bundle pattern, producing what is called the “honeycomb effect”. Moreover, the data is further degraded due to the fibre core cross coupling problem. On the other hand, there is an unmet clinical need for automatic tools that can help the clinicians to detect fluorescently labelled bacteria in distal lung images. The aim of this thesis is to develop advanced image processing algorithms that can address the above mentioned problems. First, we provide a statistical model for the fibre core cross coupling problem and the sparse sampling by imaging fibre bundles (honeycomb artefact), which are formulated here as a restoration problem for the first time in the literature. We then introduce a non-linear interpolation method, based on Gaussian processes regression, in order to recover an interpretable scene from the deconvolved data. Second, we develop two bacteria detection algorithms, each of which provides different characteristics. The first approach considers joint formulation to the sparse coding and anomaly detection problems. The anomalies here are considered as candidate bacteria, which are annotated with the help of a trained clinician. Although this approach provides good detection performance and outperforms existing methods in the literature, the user has to carefully tune some crucial model parameters. Hence, we propose a more adaptive approach, for which a Bayesian framework is adopted. This approach not only outperforms the proposed supervised approach and existing methods in the literature but also provides computation time that competes with optimization-based methods

Human neutrophil elastase phenotyping: classifying neutrophils by function with novel imaging agents

This thesis describes approaches taken to classify the functionally diverse neutrophil by its different functions in a live-cell imaging tool. Human neutrophil elastase (HNE) cleaves over 30 substrates, its release is controlled with varying extents of degranulation and its activity is subject to complex modulation. HNE regulation is necessary for the maintenance of health via HNE’s multiplicity of functions. Although neutrophils are routinely quantified in assessments of chronic inflammation, we do not have a clinically translatable, integrated tool for measuring HNE in human tissues. Two novel probes have been characterised and multiplexed to form the basis of an imaging technique for understanding the functional implications of neutrophil activity in human tissues, in real-time. Neutrophil Activation Probe (NAP) and VE200 are novel probes for HNE activity and presence. These probes are characterised in vitro as sensitive and HNE-specific imaging agents for live-cell imaging and image cytometry. Multiplexed NAP and HNE are detect neutrophil activation, apoptosis and necrosis. These imaging agents can inform deep profiling techniques to separate neutrophils into untreated, primed, activated and primed-activated states and endogenous vs. exogenous stimulus sub-states. Finally, sections of adenocarcinomatous human lung and whole human lungs, ventilated ex vivo, demonstrate the applicability of HNEbased, multiparametric profiling and neutrophil activation detection to clinically relevant platforms