Deconvolution and Restoration of Optical Endomicroscopy Images

Optical endomicroscopy (OEM) is an emerging technology platform with preclinical and clinical imaging applications. Pulmonary OEM via fibre bundles has the potential to provide in vivo, in situ molecular signatures of disease such as infection and inflammation. However, enhancing the quality of data acquired by this technique for better visualization and subsequent analysis remains a challenging problem. Cross coupling between fiber cores and sparse sampling by imaging fiber bundles are the main reasons for image degradation, and poor detection performance (i.e., inflammation, bacteria, etc.). In this work, we address the problem of deconvolution and restoration of OEM data. We propose a hierarchical Bayesian model to solve this problem and compare three estimation algorithms to exploit the resulting joint posterior distribution. The first method is based on Markov chain Monte Carlo (MCMC) methods, however, it exhibits a relatively long computational time. The second and third algorithms deal with this issue and are based on a variational Bayes (VB) approach and an alternating direction method of multipliers (ADMM) algorithm respectively. Results on both synthetic and real datasets illustrate the effectiveness of the proposed methods for restoration of OEM images

Bayesian image restoration and bacteria detection in optical endomicroscopy

Optical microscopy systems can be used to obtain high-resolution microscopic images of tissue cultures and ex vivo tissue samples. This imaging technique can be translated for in vivo, in situ applications by using optical fibres and miniature optics. Fibred optical endomicroscopy (OEM) can enable optical biopsy in organs inaccessible by any other imaging systems, and hence can provide rapid and accurate diagnosis in a short time. The raw data the system produce is difficult to interpret as it is modulated by a fibre bundle pattern, producing what is called the “honeycomb effect”. Moreover, the data is further degraded due to the fibre core cross coupling problem. On the other hand, there is an unmet clinical need for automatic tools that can help the clinicians to detect fluorescently labelled bacteria in distal lung images. The aim of this thesis is to develop advanced image processing algorithms that can address the above mentioned problems. First, we provide a statistical model for the fibre core cross coupling problem and the sparse sampling by imaging fibre bundles (honeycomb artefact), which are formulated here as a restoration problem for the first time in the literature. We then introduce a non-linear interpolation method, based on Gaussian processes regression, in order to recover an interpretable scene from the deconvolved data. Second, we develop two bacteria detection algorithms, each of which provides different characteristics. The first approach considers joint formulation to the sparse coding and anomaly detection problems. The anomalies here are considered as candidate bacteria, which are annotated with the help of a trained clinician. Although this approach provides good detection performance and outperforms existing methods in the literature, the user has to carefully tune some crucial model parameters. Hence, we propose a more adaptive approach, for which a Bayesian framework is adopted. This approach not only outperforms the proposed supervised approach and existing methods in the literature but also provides computation time that competes with optimization-based methods

Effective deep learning training for single-image super-resolution in endomicroscopy exploiting video-registration-based reconstruction

PURPOSE: Probe-based confocal laser endomicroscopy (pCLE) is a recent imaging modality that allows performing in vivo optical biopsies. The design of pCLE hardware, and its reliance on an optical fibre bundle, fundamentally limits the image quality with a few tens of thousands fibres, each acting as the equivalent of a single-pixel detector, assembled into a single fibre bundle. Video registration techniques can be used to estimate high-resolution (HR) images by exploiting the temporal information contained in a sequence of low-resolution (LR) images. However, the alignment of LR frames, required for the fusion, is computationally demanding and prone to artefacts. METHODS: In this work, we propose a novel synthetic data generation approach to train exemplar-based Deep Neural Networks (DNNs). HR pCLE images with enhanced quality are recovered by the models trained on pairs of estimated HR images (generated by the video registration algorithm) and realistic synthetic LR images. Performance of three different state-of-the-art DNNs techniques were analysed on a Smart Atlas database of 8806 images from 238 pCLE video sequences. The results were validated through an extensive image quality assessment that takes into account different quality scores, including a Mean Opinion Score (MOS). RESULTS: Results indicate that the proposed solution produces an effective improvement in the quality of the obtained reconstructed image. CONCLUSION: The proposed training strategy and associated DNNs allows us to perform convincing super-resolution of pCLE images

Online Super-Resolution For Fibre-Bundle-Based Confocal Laser Endomicroscopy

Probe-based Confocal Laser Endomicroscopy (pCLE) produces microscopic images enabling real-time in vivo optical biopsy. However, the miniaturisation of the optical hardware, specifically the reliance on an optical fibre bundle as an imaging guide, fundamentally limits image quality by producing artefacts, noise, and relatively low contrast and resolution. The reconstruction approaches in clinical pCLE products do not fully alleviate these problems. Consequently, image quality remains a barrier that curbs the full potential of pCLE. Enhancing the image quality of pCLE in real-time remains a challenge. The research in this thesis is a response to this need. I have developed dedicated online super-resolution methods that account for the physics of the image acquisition process. These methods have the potential to replace existing reconstruction algorithms without interfering with the fibre design or the hardware of the device. In this thesis, novel processing pipelines are proposed for enhancing the image quality of pCLE. First, I explored a learning-based super-resolution method that relies on mapping from the low to the high-resolution space. Due to the lack of high-resolution pCLE, I proposed to simulate high-resolution data and use it as a ground truth model that is based on the pCLE acquisition physics. However, pCLE images are reconstructed from irregularly distributed fibre signals, and grid-based Convolutional Neural Networks are not designed to take irregular data as input. To alleviate this problem, I designed a new trainable layer that embeds Nadaraya- Watson regression. Finally, I proposed a novel blind super-resolution approach by deploying unsupervised zero-shot learning accompanied by a down-sampling kernel crafted for pCLE. I evaluated these new methods in two ways: a robust image quality assessment and a perceptual quality test assessed by clinical experts. The results demonstrate that the proposed super-resolution pipelines are superior to the current reconstruction algorithm in terms of image quality and clinician preference

Bayesian Bacterial Detection Using Irregularly Sampled Optical Endomicroscopy Images

Pneumonia is a major cause of morbidity and mortality of patients in intensive care. Rapid determination of the presence and gram status of the pathogenic bacteria in the distal lung may enable a more tailored treatment regime. Optical Endomicroscopy (OEM) is an emerging medical imaging platform with preclinical and clinical utility. Pulmonary OEM via multi-core fibre bundles has the potential to provide in vivo, in situ, fluorescent molecular signatures of the causes of infection and inflammation. This paper presents a Bayesian approach for bacterial detection in OEM images. The model considered assumes that the observed pixel fluorescence is a linear combination of the actual intensity value associated with tissues or background, corrupted by additive Gaussian noise and potentially by an additional sparse outlier term modelling anomalies (bacteria). The bacteria detection problem is formulated in a Bayesian framework and prior distributions are assigned to the unknown model parameters. A Markov chain Monte Carlo algorithm based on a partially collapsed Gibbs sampler is used to sample the posterior distribution of the unknown parameters. The proposed algorithm is first validated by simulations conducted using synthetic datasets for which good performance is obtained. Analysis is then conducted using two ex vivo lung datasets in which fluorescently labelled bacteria are present in the distal lung. A good correlation between bacteria counts identified by a trained clinician and those of the proposed method, which detects most of the manually annotated regions, is observed

High-resolution fluorescence endomicroscopy for rapid evaluation of breast cancer margins

Breast cancer is a major public health problem world-wide and the second leading cause of cancer-related female deaths. Breast conserving surgery (BCS), in the form of wide local excision (WLE), allows complete tumour resection while maintaining acceptable cosmesis. It is the recommended treatment for a large number of patients with early stage disease or, in more advanced cases, following neoadjuvant chemotherapy. About 30% of patients undergoing BCS require one or more re-operative interventions, mainly due to the presence of positive margins. The standard of care for surgical margin assessment is post-operative examination of histopathological tissue sections. However, this process is invasive, introduces sampling errors and does not provide real-time assessment of the tumour status of radial margins. The objective of this thesis is to improve intra-operative assessment of margin status by performing optical biopsy in breast tissue. This thesis presents several technical and clinical developments related to confocal fluorescence endomicroscopy systems for real-time characterisation of different breast morphologies. The imaging systems discussed employ flexible fibre-bundle based imaging probes coupled to high-speed line-scan confocal microscope set-up. A preliminary study on 43 unfixed breast specimens describes the development and testing of line-scan confocal laser endomicroscope (LS-CLE) to image and classify different breast pathologies. LS-CLE is also demonstrated to assess the intra-operative tumour status of whole WLE specimens and surgical excisions with high diagnostic accuracy. A third study demonstrates the development and testing of a bespoke LS-CLE system with methylene blue (MB), an US Food and Drug Administration (FDA) approved fluorescent agent, and integration with robotic scanner to enable large-area in vivo imaging of breast cancer. The work also addresses three technical issues which limit existing fibre-bundle based fluorescence endomicroscopy systems: i) Restriction to use single fluorescence agent due to low-speed, single excitation and single fluorescence spectral band imaging systems; ii) Limited Field of view (FOV) of fibre-bundle endomicroscopes due to small size of the fibre tip and iii) Limited spatial resolution of fibre-bundle endomicroscopes due to the spacing between the individual fibres leading to fibre-pixelation effects. Details of design and development of a high-speed dual-wavelength LS-CLE system suitable for high-resolution multiplexed imaging are presented. Dual-wavelength imaging is achieved by sequentially switching between 488 nm and 660 nm laser sources for alternate frames, avoiding spectral bleed-through, and providing an effective frame rate of 60 Hz. A combination of hand-held or robotic scanning with real-time video mosaicking, is demonstrated to enable large-area imaging while still maintaining microscopic resolution. Finally, a miniaturised piezoelectric transducer-based fibre-shifting endomicroscope is developed to enhance the resolution over conventional fibre-bundle based imaging systems. The fibre-shifting endomicroscope provides a two-fold improvement in resolution and coupled to a high-speed LS-CLE scanning system, provides real-time imaging of biological samples at 30 fps. These investigations furthered the utility and applications of the fibre-bundle based fluorescence systems for rapid imaging and diagnosis of cancer margins.Open Acces

Review of photoacoustic imaging plus X

Photoacoustic imaging (PAI) is a novel modality in biomedical imaging technology that combines the rich optical contrast with the deep penetration of ultrasound. To date, PAI technology has found applications in various biomedical fields. In this review, we present an overview of the emerging research frontiers on PAI plus other advanced technologies, named as PAI plus X, which includes but not limited to PAI plus treatment, PAI plus new circuits design, PAI plus accurate positioning system, PAI plus fast scanning systems, PAI plus novel ultrasound sensors, PAI plus advanced laser sources, PAI plus deep learning, and PAI plus other imaging modalities. We will discuss each technology's current state, technical advantages, and prospects for application, reported mostly in recent three years. Lastly, we discuss and summarize the challenges and potential future work in PAI plus X area

Bayesian Activity Estimation and Uncertainty Quantification of Spent Nuclear Fuel Using Passive Gamma Emission Tomography

In this paper, we address the problem of activity estimation in passive gamma emission tomography (PGET) of spent nuclear fuel. Two different noise models are considered and compared, namely, the isotropic Gaussian and the Poisson noise models. The problem is formulated within a Bayesian framework as a linear inverse problem and prior distributions are assigned to the unknown model parameters. In particular, a Bernoulli-truncated Gaussian prior model is considered to promote sparse pin configurations. A Markov chain Monte Carlo (MCMC) method, based on a split and augmented Gibbs sampler, is then used to sample the posterior distribution of the unknown parameters. The proposed algorithm is first validated by simulations conducted using synthetic data, generated using the nominal models. We then consider more realistic data simulated using a bespoke simulator, whose forward model is non-linear and not available analytically. In that case, the linear models used are mis-specified and we analyse their robustness for activity estimation. The results demonstrate superior performance of the proposed approach in estimating the pin activities in different assembly patterns, in addition to being able to quantify their uncertainty measures, in comparison with existing methods