Transferability of Deep Learning Algorithms for Malignancy Detection in Confocal Laser Endomicroscopy Images from Different Anatomical Locations of the Upper Gastrointestinal Tract

Squamous Cell Carcinoma (SCC) is the most common cancer type of the epithelium and is often detected at a late stage. Besides invasive diagnosis of SCC by means of biopsy and histo-pathologic assessment, Confocal Laser Endomicroscopy (CLE) has emerged as noninvasive method that was successfully used to diagnose SCC in vivo. For interpretation of CLE images, however, extensive training is required, which limits its applicability and use in clinical practice of the method. To aid diagnosis of SCC in a broader scope, automatic detection methods have been proposed. This work compares two methods with regard to their applicability in a transfer learning sense, i.e. training on one tissue type (from one clinical team) and applying the learnt classification system to another entity (different anatomy, different clinical team). Besides a previously proposed, patch-based method based on convolutional neural networks, a novel classification method on image level (based on a pre-trained Inception V.3 network with dedicated preprocessing and interpretation of class activation maps) is proposed and evaluated. The newly presented approach improves recognition performance, yielding accuracies of 91.63% on the first data set (oral cavity) and 92.63% on a joint data set. The generalization from oral cavity to the second data set (vocal folds) lead to similar area-under-the-ROC curve values than a direct training on the vocal folds data set, indicating good generalization.Comment: Erratum for version 1, correcting the number of CLE image sequences used in one data se

Development, Implementation and Application of Confocal Laser Endomicroscopy in Brain, Head and Neck Surgery : A Review

When we talk about visualization methods in surgery, it is important to mention that the diagnosis of tumors and how we define tumor borders intraoperatively in a correct way are two main things that would not be possible to achieve without this grand variety of visualization methods we have at our disposal nowadays. In addition, histopathology also plays a very important role, and its importance cannot be neglected either. Some biopsy specimens, e.g., frozen sections, are examined by a histopathologist and lead to tumor diagnosis and the definition of its borders. Furthermore, surgical resection is a very important point when it comes to prognosis and life survival. Confocal laser endomicroscopy (CLE) is an imaging technique that provides microscopic information on the tissue in real time. CLE of disorders, such as head, neck and brain tumors, has only recently been suggested to contribute to both immediate tumor characterization and detection. It can be used as an additional tool for surgical biopsies during biopsy or surgical procedures and for inspection of resection margins during surgery. In this review, we analyze the development, implementation, advantages and disadvantages as well as the future directions of this technique in neurosurgical and otorhinolaryngological disciplines

High-resolution fluorescence endomicroscopy for rapid evaluation of breast cancer margins

Breast cancer is a major public health problem world-wide and the second leading cause of cancer-related female deaths. Breast conserving surgery (BCS), in the form of wide local excision (WLE), allows complete tumour resection while maintaining acceptable cosmesis. It is the recommended treatment for a large number of patients with early stage disease or, in more advanced cases, following neoadjuvant chemotherapy. About 30% of patients undergoing BCS require one or more re-operative interventions, mainly due to the presence of positive margins. The standard of care for surgical margin assessment is post-operative examination of histopathological tissue sections. However, this process is invasive, introduces sampling errors and does not provide real-time assessment of the tumour status of radial margins. The objective of this thesis is to improve intra-operative assessment of margin status by performing optical biopsy in breast tissue. This thesis presents several technical and clinical developments related to confocal fluorescence endomicroscopy systems for real-time characterisation of different breast morphologies. The imaging systems discussed employ flexible fibre-bundle based imaging probes coupled to high-speed line-scan confocal microscope set-up. A preliminary study on 43 unfixed breast specimens describes the development and testing of line-scan confocal laser endomicroscope (LS-CLE) to image and classify different breast pathologies. LS-CLE is also demonstrated to assess the intra-operative tumour status of whole WLE specimens and surgical excisions with high diagnostic accuracy. A third study demonstrates the development and testing of a bespoke LS-CLE system with methylene blue (MB), an US Food and Drug Administration (FDA) approved fluorescent agent, and integration with robotic scanner to enable large-area in vivo imaging of breast cancer. The work also addresses three technical issues which limit existing fibre-bundle based fluorescence endomicroscopy systems: i) Restriction to use single fluorescence agent due to low-speed, single excitation and single fluorescence spectral band imaging systems; ii) Limited Field of view (FOV) of fibre-bundle endomicroscopes due to small size of the fibre tip and iii) Limited spatial resolution of fibre-bundle endomicroscopes due to the spacing between the individual fibres leading to fibre-pixelation effects. Details of design and development of a high-speed dual-wavelength LS-CLE system suitable for high-resolution multiplexed imaging are presented. Dual-wavelength imaging is achieved by sequentially switching between 488 nm and 660 nm laser sources for alternate frames, avoiding spectral bleed-through, and providing an effective frame rate of 60 Hz. A combination of hand-held or robotic scanning with real-time video mosaicking, is demonstrated to enable large-area imaging while still maintaining microscopic resolution. Finally, a miniaturised piezoelectric transducer-based fibre-shifting endomicroscope is developed to enhance the resolution over conventional fibre-bundle based imaging systems. The fibre-shifting endomicroscope provides a two-fold improvement in resolution and coupled to a high-speed LS-CLE scanning system, provides real-time imaging of biological samples at 30 fps. These investigations furthered the utility and applications of the fibre-bundle based fluorescence systems for rapid imaging and diagnosis of cancer margins.Open Acces

Confocal Microscopy and Nuclear Segmentation Algorithm for Quantitative Imaging of Epithelial Tissue

Carcinomas, cancers that originate in the epithelium, account for more than 80% of all cancers. When detected early, the 5-year survival rate is greatly increased. Biopsy and histopathology is the current gold standard for diagnosis of epithelial carcinomas which is an invasive, time-intensive, and stressful procedure. In vivo confocal microscopy has the potential to non-invasively image epithelial tissue in near-real time. This dissertation describes the development of a confocal microscope for imaging epithelial tissues and an image processing algorithm for segmentation of epithelial nuclei. A rapid beam and stage scanning combination was used to acquire fluorescence confocal images of cellular and tissue features along the length of excised mouse colon. A single 1 × 60 mm2 field of view is acquired in 10 seconds. Disruption of crypt structure such as size, shape, and distribution is visualized in images of inflamed colon tissue, while the normal mouse colon exhibited uniform crypt structure and distribution. An automated pulse coupled neural network segmentation algorithm was developed for epithelial nuclei segmentation. An increase in nuclear size and the nuclear-to-cytoplasmic ratio is a potential precursor to pre-cancer development. The spiking cortical model algorithm was evaluated using a developed confocal image model of epithelial tissues with varying contrast. It was further validated on reflectance confocal images of porcine and human oral tissue from two separate confocal imaging systems. Biopsies of human oral mucosa are used to determine the tissue and system effects on measurements of nuclear-to-cytoplasmic ratio

Semantic segmentation of non-linear multimodal images for disease grading of inflammatory bowel disease: A segnet-based application

Non-linear multimodal imaging, the combination of coherent anti-stokes Raman scattering (CARS), two-photon excited fluorescence (TPEF) and second harmonic generation (SHG), has shown its potential to assist the diagnosis of different inflammatory bowel diseases (IBDs). This label-free imaging technique can support the ‘gold-standard’ techniques such as colonoscopy and histopathology to ensure an IBD diagnosis in clinical environment. Moreover, non-linear multimodal imaging can measure biomolecular changes in different tissue regions such as crypt and mucosa region, which serve as a predictive marker for IBD severity. To achieve a real-time assessment of IBD severity, an automatic segmentation of the crypt and mucosa regions is needed. In this paper, we semantically segment the crypt and mucosa region using a deep neural network. We utilized the SegNet architecture (Badrinarayanan et al., 2015) and compared its results with a classical machine learning approach. Our trained SegNet mod el achieved an overall F1 score of 0.75. This model outperformed the classical machine learning approach for the segmentation of the crypt and mucosa region in our study

Using Fluorescence – Polarization Endoscopy in Detection of Precancerous and Cancerous Lesions in Colon and Pancreatic Cancer

Colitis-associated cancer (CAC) arises from premalignant flat lesions of the colon, which are difficult to detect with current endoscopic screening approaches. We have developed a complementary fluorescence and polarization reporting strategy that combines the unique biochemical and physical properties of dysplasia and cancer for real time detection of these lesions. Utilizing a new thermoresponsive sol-gel formulation with targeted molecular probe allowed topical application and detection of precancerous and cancerous lesions during endoscopy. Incorporation of nanowire-filtered polarization imaging into NIR fluorescence endoscopy served as a validation strategy prior to obtaining biopsies. In order to reduce repeat surgeries arising from incomplete tumor resection, we demonstrated the efficacy of the targeted molecular probe towards margins of sporadic colorectal cancer (SCC). Fluorescence-polarization microscopy using circular polarized (CP) light served as a rapid, supplementary tool for assessment and validation of excised tissue to ensure complete tumor resection for examining tumor margins prior to H&E-based pathological diagnosis. We extended our platform towards non-invasive directed detection of pancreatic cancer utilizing fluorescence molecular tomography (FMT) and NIR laparoscopy using identified targeted molecular probe. We were able to non-invasively distinguished between pancreatitis and pancreatic cancer and guide pancreatic tumor resection using NIR laparoscopy

Development and clinical translation of optical and software methods for endomicroscopic imaging

Endomicroscopy is an emerging technology that aims to improve clinical diagnostics by allowing for in vivo microscopy in difficult to reach areas of the body. This is most commonly achieved by using coherent fibre bundles to relay light for illumination and imaging to and from the area under investigation. Endomicroscopy’s attraction for researchers and clinicians is two-fold: on the one hand, its use can reduce the invasiveness of a diagnostic procedure by removing the need for biopsies; On the other hand, it allows for structural and functional in vivo imaging. Endomicroscopic images acquired through optical fibre bundles exhibit artefacts that deteriorate image quality and contrast. This thesis aims to improve an existing endomicroscopy imaging system by exploring two methods that mitigate these artefacts. The first, software-based method takes several processing steps from literature and implements them in an existing endomicroscopy device with a focus on real-time application to enable clinical use, after image quality was found to be inadequate without further processing. A contribution to the field is that two different approaches are implemented and compared in quantitative and qualitative means that have not been compared directly in this manner before. This first attempt at improving endomicroscopy image quality relies solely on digital image processing methods and is developed with a strong focus on real-time applicability in clinical use. Both approaches are compared on pre-clinical and clinical human imaging data. The second method targets the effect of inter-core coupling, which reduces contrast in fibre images. A parallelised confocal imaging method is developed in which a sequence of images is acquired while selectively illuminating groups of fibre cores through the use of a spatial light modulator. A bespoke algorithm creates a composite image in a final processing step. In doing so, unwanted light is detected and removed from the final image. This method is shown to reduce the negative impact of inter-core coupling on image contrast on small imaging targets, while no benefit was found in large, scattering samples