Prospects for Theranostics in Neurosurgical Imaging: Empowering Confocal Laser Endomicroscopy Diagnostics via Deep Learning

Confocal laser endomicroscopy (CLE) is an advanced optical fluorescence imaging technology that has the potential to increase intraoperative precision, extend resection, and tailor surgery for malignant invasive brain tumors because of its subcellular dimension resolution. Despite its promising diagnostic potential, interpreting the gray tone fluorescence images can be difficult for untrained users. In this review, we provide a detailed description of bioinformatical analysis methodology of CLE images that begins to assist the neurosurgeon and pathologist to rapidly connect on-the-fly intraoperative imaging, pathology, and surgical observation into a conclusionary system within the concept of theranostics. We present an overview and discuss deep learning models for automatic detection of the diagnostic CLE images and discuss various training regimes and ensemble modeling effect on the power of deep learning predictive models. Two major approaches reviewed in this paper include the models that can automatically classify CLE images into diagnostic/nondiagnostic, glioma/nonglioma, tumor/injury/normal categories and models that can localize histological features on the CLE images using weakly supervised methods. We also briefly review advances in the deep learning approaches used for CLE image analysis in other organs. Significant advances in speed and precision of automated diagnostic frame selection would augment the diagnostic potential of CLE, improve operative workflow and integration into brain tumor surgery. Such technology and bioinformatics analytics lend themselves to improved precision, personalization, and theranostics in brain tumor treatment.Comment: See the final version published in Frontiers in Oncology here: https://www.frontiersin.org/articles/10.3389/fonc.2018.00240/ful

Confocal laser endomicroscopy in head and neck cancer: steps forward?

Purpose of review Confocal laser endomicroscopy (CLE) is a novel, noninvasive technique used to obtain microanatomical images of the inner lining of hollow organs. It has been used in a variety of clinical specialties to aid in the diagnosis and treatment planning of inflammatory and neoplastic processes. Our intent is to provide an up-to-date review of the literature in the setting of head and neck diseases as well as describing our own initial results and areas of future research. Recent findings With increasing experience using CLE in the upper aerodigestive tract (UADT), evidence is mounting that this method can be a useful adjunct to standard endoscopy and other diagnostic techniques. Recent publications have shown that by using CLE, microanatomical structures of healthy and diseased mucosa can easily be identified, allowing for a differentiation of dysplastic/neoplastic and benign mucosal lesions. Standardized diagnostic protocols as well as clinically relevant classification systems for the UADT have not yet been described. Summary CLE is an imaging modality that allows real-time visualization of mucosal cellular architecture and other histologic characteristics. First reports on its use in the UADT have yielded promising results, but the true value of this method is yet to be determined

New Frontier in Hypericin-Mediated Diagnosis of Cancer with Current Optical Technologies

Photosensitizers (PSs) have shown great potentials as molecular contrast agents in photodynamic diagnosis (PDD) of cancer. While the diagnostic values of PSs have been proven previously, little efforts have been put into developing optical imaging and diagnostic algorithms. In this article, we review the recent development of optical probes that have been used in conjunction with a potent PS, hypericin (HY). Various fluorescence techniques such as laser confocal microscopy, fluorescence urine cytology, endoscopy and endomicroscopy are covered. We will also discuss about image processing and classification approaches employed for accurate PDD. We anticipate that continual efforts in these developments could lead to an objective PDD and complete surgical clearance of tumors. Recent advancements in nanotechnology have also opened new horizons for PSs. The use of biocompatible gold nanoparticles as carrier for enhanced targeted delivery of HY has been attained. In addition, plasmonic properties of nanoparticles were harnessed to induce localized hyperthermia and to manage the release of PS molecules, enabling a better therapeutic outcome of a combined photodynamic and photothermal therapy. Finally, we discuss how nanoparticles can be used as contrast agents for other optical techniques such as optical coherence tomography and surface-enhanced Raman scattering imaging

Identification of liver metastases with probe-based confocal laser endomicroscopy at two excitation wavelengths.

BACKGROUND: Metastasis of colorectal cancer to the liver is the most common indication for hepatic resection in a western population. Incomplete excision of malignancy due to residual microscopic disease normally results in worse patient outcome. Therefore, a method aiding in the real time discrimination of normal and malignant tissue on a microscopic level would be of benefit. MATERIAL AND METHODS: The ability of fluorescent probe-based confocal laser endomicroscopy (pCLE) to identify normal and malignant liver tissue was evaluated in an orthotopic murine model of colorectal cancer liver metastasis (CRLM). To maximise information yield, two clinical fluorophores, fluorescein and indocyanine green (ICG) were injected and imaged in a dual wavelength approach (488 and 660 nm, respectively). Visual tissue characteristics on pCLE examination were compared with histological features. Fluorescence intensity in both tissues was statistically analysed to elucidate if this can be used to differentiate between normal and malignant tissue. RESULTS: Fluorescein (488 nm) enabled good visualisation of normal and CRLM tissue, whereas ICG (660 nm) visualisation was limited to normal liver tissue only. Fluorescence intensity in areas of CRLM was typically 53-100% lower than normal hepatic parenchyma. Using general linear mixed modelling and receiver operating characteristic analysis, high fluorescence intensity was found to be statistically more likely in normal hepatic tissue. CONCLUSION: Real time discrimination between normal liver parenchyma and metastatic tissue with pCLE examination of fluorescein and ICG is feasible. Employing two (rather than a single) fluorophores allows a combination of qualitative and quantitative characteristics to be used to distinguish between hepatic parenchyma and CRLM. Lasers Surg. Med. © 2016 Wiley Periodicals, Inc

Fast widefield techniques for fluorescence and phase endomicroscopy

Thesis (Ph.D.)--Boston UniversityEndomicroscopy is a recent development in biomedical optics which gives researchers and physicians microscope-resolution views of intact tissue to complement macroscopic visualization during endoscopy screening. This thesis presents HiLo endomicroscopy and oblique back-illumination endomicroscopy, fast widefield imaging techniques with fluorescence and phase contrast, respectively. Fluorescence imaging in thick tissue is often hampered by strong out-of-focus background signal. Laser scanning confocal endomicroscopy has been developed for optically-sectioned imaging free from background, but reliance on mechanical scanning fundamentally limits the frame rate and represents significant complexity and expense. HiLo is a fast, simple, widefield fluorescence imaging technique which rejects out-of-focus background signal without the need for scanning. It works by acquiring two images of the sample under uniform and structured illumination and synthesizing an optically sectioned result with real-time image processing. Oblique back-illumination microscopy (OBM) is a label-free technique which allows, for the first time, phase gradient imaging of sub-surface morphology in thick scattering tissue with a reflection geometry. OBM works by back-illuminating the sample with the oblique diffuse reflectance from light delivered via off-axis optical fibers. The use of two diametrically opposed illumination fibers allows simultaneous and independent measurement of phase gradients and absorption contrast. Video-rate single-exposure operation using wavelength multiplexing is demonstrated

Multimodal Multispectral Optical Endoscopic Imaging for Biomedical Applications

Optical imaging is an emerging field of clinical diagnostics that can address the growing medical need for early cancer detection and diagnosis. Various human cancers are amenable to better prognosis and patient survival if found and treated during early disease onset. Besides providing wide-field, macroscopic diagnostic information similar to existing clinical imaging techniques, optical imaging modalities have the added advantage of microscopic, high resolution cellular-level imaging from in vivo tissues in real time. This comprehensive imaging approach to cancer detection and the possibility of performing an ‘optical biopsy’ without tissue removal has led to growing interest in the field with numerous techniques under investigation. Three optical techniques are discussed in this thesis, namely multispectral fluorescence imaging (MFI), hyperspectral reflectance imaging (HRI) and fluorescence confocal endomicroscopy (FCE). MFI and HRI are novel endoscopic imaging-based extensions of single point detection techniques, such as laser induced fluorescence spectroscopy and diffuse reflectance spectroscopy. This results in the acquisition of spectral data in an intuitive imaging format that allows for quantitative evaluation of tissue disease states. We demonstrate MFI and HRI on fluorophores, tissue phantoms and ex vivo tissues and present the results as an RGB colour image for more intuitive assessment. This follows dimensionality reduction of the acquired spectral data with a fixed-reference isomap diagnostic algorithm to extract only the most meaningful data parameters. FCE is a probe-based point imaging technique offering confocal detection in vivo with almost histology-grade images. We perform FCE imaging on chemotherapy-treated in vitro human ovarian cancer cells, ex vivo human cancer tissues and photosensitiser-treated in vivo murine tumours to show the enhanced detection capabilities of the technique. Finally, the three modalities are applied in combination to demonstrate an optical viewfinder approach as a possible minimally-invasive imaging method for early cancer detection and diagnosis

Confocal Laser Endomicroscopy Image Analysis with Deep Convolutional Neural Networks

abstract: Rapid intraoperative diagnosis of brain tumors is of great importance for planning treatment and guiding the surgeon about the extent of resection. Currently, the standard for the preliminary intraoperative tissue analysis is frozen section biopsy that has major limitations such as tissue freezing and cutting artifacts, sampling errors, lack of immediate interaction between the pathologist and the surgeon, and time consuming. Handheld, portable confocal laser endomicroscopy (CLE) is being explored in neurosurgery for its ability to image histopathological features of tissue at cellular resolution in real time during brain tumor surgery. Over the course of examination of the surgical tumor resection, hundreds to thousands of images may be collected. The high number of images requires significant time and storage load for subsequent reviewing, which motivated several research groups to employ deep convolutional neural networks (DCNNs) to improve its utility during surgery. DCNNs have proven to be useful in natural and medical image analysis tasks such as classification, object detection, and image segmentation. This thesis proposes using DCNNs for analyzing CLE images of brain tumors. Particularly, it explores the practicality of DCNNs in three main tasks. First, off-the shelf DCNNs were used to classify images into diagnostic and non-diagnostic. Further experiments showed that both ensemble modeling and transfer learning improved the classifier’s accuracy in evaluating the diagnostic quality of new images at test stage. Second, a weakly-supervised learning pipeline was developed for localizing key features of diagnostic CLE images from gliomas. Third, image style transfer was used to improve the diagnostic quality of CLE images from glioma tumors by transforming the histology patterns in CLE images of fluorescein sodium-stained tissue into the ones in conventional hematoxylin and eosin-stained tissue slides. These studies suggest that DCNNs are opted for analysis of CLE images. They may assist surgeons in sorting out the non-diagnostic images, highlighting the key regions and enhancing their appearance through pattern transformation in real time. With recent advances in deep learning such as generative adversarial networks and semi-supervised learning, new research directions need to be followed to discover more promises of DCNNs in CLE image analysis.Dissertation/ThesisDoctoral Dissertation Neuroscience 201

High-resolution fluorescence endomicroscopy for rapid evaluation of breast cancer margins

Breast cancer is a major public health problem world-wide and the second leading cause of cancer-related female deaths. Breast conserving surgery (BCS), in the form of wide local excision (WLE), allows complete tumour resection while maintaining acceptable cosmesis. It is the recommended treatment for a large number of patients with early stage disease or, in more advanced cases, following neoadjuvant chemotherapy. About 30% of patients undergoing BCS require one or more re-operative interventions, mainly due to the presence of positive margins. The standard of care for surgical margin assessment is post-operative examination of histopathological tissue sections. However, this process is invasive, introduces sampling errors and does not provide real-time assessment of the tumour status of radial margins. The objective of this thesis is to improve intra-operative assessment of margin status by performing optical biopsy in breast tissue. This thesis presents several technical and clinical developments related to confocal fluorescence endomicroscopy systems for real-time characterisation of different breast morphologies. The imaging systems discussed employ flexible fibre-bundle based imaging probes coupled to high-speed line-scan confocal microscope set-up. A preliminary study on 43 unfixed breast specimens describes the development and testing of line-scan confocal laser endomicroscope (LS-CLE) to image and classify different breast pathologies. LS-CLE is also demonstrated to assess the intra-operative tumour status of whole WLE specimens and surgical excisions with high diagnostic accuracy. A third study demonstrates the development and testing of a bespoke LS-CLE system with methylene blue (MB), an US Food and Drug Administration (FDA) approved fluorescent agent, and integration with robotic scanner to enable large-area in vivo imaging of breast cancer. The work also addresses three technical issues which limit existing fibre-bundle based fluorescence endomicroscopy systems: i) Restriction to use single fluorescence agent due to low-speed, single excitation and single fluorescence spectral band imaging systems; ii) Limited Field of view (FOV) of fibre-bundle endomicroscopes due to small size of the fibre tip and iii) Limited spatial resolution of fibre-bundle endomicroscopes due to the spacing between the individual fibres leading to fibre-pixelation effects. Details of design and development of a high-speed dual-wavelength LS-CLE system suitable for high-resolution multiplexed imaging are presented. Dual-wavelength imaging is achieved by sequentially switching between 488 nm and 660 nm laser sources for alternate frames, avoiding spectral bleed-through, and providing an effective frame rate of 60 Hz. A combination of hand-held or robotic scanning with real-time video mosaicking, is demonstrated to enable large-area imaging while still maintaining microscopic resolution. Finally, a miniaturised piezoelectric transducer-based fibre-shifting endomicroscope is developed to enhance the resolution over conventional fibre-bundle based imaging systems. The fibre-shifting endomicroscope provides a two-fold improvement in resolution and coupled to a high-speed LS-CLE scanning system, provides real-time imaging of biological samples at 30 fps. These investigations furthered the utility and applications of the fibre-bundle based fluorescence systems for rapid imaging and diagnosis of cancer margins.Open Acces