Prospects for Theranostics in Neurosurgical Imaging: Empowering Confocal Laser Endomicroscopy Diagnostics via Deep Learning

Confocal laser endomicroscopy (CLE) is an advanced optical fluorescence imaging technology that has the potential to increase intraoperative precision, extend resection, and tailor surgery for malignant invasive brain tumors because of its subcellular dimension resolution. Despite its promising diagnostic potential, interpreting the gray tone fluorescence images can be difficult for untrained users. In this review, we provide a detailed description of bioinformatical analysis methodology of CLE images that begins to assist the neurosurgeon and pathologist to rapidly connect on-the-fly intraoperative imaging, pathology, and surgical observation into a conclusionary system within the concept of theranostics. We present an overview and discuss deep learning models for automatic detection of the diagnostic CLE images and discuss various training regimes and ensemble modeling effect on the power of deep learning predictive models. Two major approaches reviewed in this paper include the models that can automatically classify CLE images into diagnostic/nondiagnostic, glioma/nonglioma, tumor/injury/normal categories and models that can localize histological features on the CLE images using weakly supervised methods. We also briefly review advances in the deep learning approaches used for CLE image analysis in other organs. Significant advances in speed and precision of automated diagnostic frame selection would augment the diagnostic potential of CLE, improve operative workflow and integration into brain tumor surgery. Such technology and bioinformatics analytics lend themselves to improved precision, personalization, and theranostics in brain tumor treatment.Comment: See the final version published in Frontiers in Oncology here: https://www.frontiersin.org/articles/10.3389/fonc.2018.00240/ful

Validation of vessel size imaging (VSI) in high-grade human gliomas using magnetic resonance imaging, image-guided biopsies, and quantitative immunohistochemistry.

To evaluate the association between a vessel size index (VSIMRI) derived from dynamic susceptibility contrast (DSC) perfusion imaging using a custom spin-and-gradient echo echoplanar imaging (SAGE-EPI) sequence and quantitative estimates of vessel morphometry based on immunohistochemistry from image-guided biopsy samples. The current study evaluated both relative cerebral blood volume (rCBV) and VSIMRI in eleven patients with high-grade glioma (7 WHO grade III and 4 WHO grade IV). Following 26 MRI-guided glioma biopsies in these 11 patients, we evaluated tissue morphometry, including vessel density and average radius, using an automated procedure based on the endothelial cell marker CD31 to highlight tumor vasculature. Measures of rCBV and VSIMRI were then compared to histological measures. We demonstrate good agreement between VSI measured by MRI and histology; VSIMRI = 13.67 μm and VSIHistology = 12.60 μm, with slight overestimation of VSIMRI in grade III patients compared to histology. rCBV showed a moderate but significant correlation with vessel density (r = 0.42, p = 0.03), and a correlation was also observed between VSIMRI and VSIHistology (r = 0.49, p = 0.01). The current study supports the hypothesis that vessel size measures using MRI accurately reflect vessel caliber within high-grade gliomas, while traditional measures of rCBV are correlated with vessel density and not vessel caliber

Glioma on Chips Analysis of glioma cell guidance and interaction in microfluidic-controlled microenvironment enabled by machine learning

In biosystems, chemical and physical fields established by gradients guide cell migration, which is a fundamental phenomenon underlying physiological and pathophysiological processes such as development, morphogenesis, wound healing, and cancer metastasis. Cells in the supportive tissue of the brain, glia, are electrically stimulated by the local field potentials from neuronal activities. How the electric field may influence glial cells is yet fully understood. Furthermore, the cancer of glia, glioma, is not only the most common type of brain cancer, but the high-grade form of it (glioblastoma) is particularly aggressive with cells migrating into the surrounding tissues (infiltration) and contribute to poor prognosis. In this thesis, I investigate how electric fields in the microenvironment can affect the migration of glioblastoma cells using a versatile microsystem I have developed. I employ a hybrid microfluidic design to combine poly(methylmethacrylate) (PMMA) and poly(dimethylsiloxane) (PDMS), two of the most common materials for microfluidic fabrication. The advantages of the two materials can be complemented while disadvantages can be mitigated. The hybrid microfluidics have advantages such as versatile 3D layouts in PMMA, high dimensional accuracy in PDMS, and rapid prototype turnaround by facile bonding between PMMA and PDMS using a dual-energy double sided tape. To accurately analyze label-free cell migration, a machine learning software, Usiigaci, is developed to automatically segment, track, and analyze single cell movement and morphological changes under phase contrast microscopy. The hybrid microfluidic chip is then used to study the migration of glioblastoma cell models, T98G and U-251MG, in electric field (electrotaxis). The influence of extracellular matrix and chemical ligands on glioblastoma electrotaxis are investigated. I further test if voltage-gated calcium channels are involved in glioblastoma electrotaxis. The electrotaxes of glioblastoma cells are found to require optimal laminin extracellular matrices and depend on different types of voltage-gated calcium channels, voltage-gated potassium channels, and sodium transporters. A reversiblysealed hybrid microfluidic chip is developed to study how electric field and laminar shear can condition confluent endothelial cells and if the biomimetic conditions affect glioma cell adhesion to them. It is found that glioma/endothelial adhesion is mediated by the Ang1/Tie2 signaling axis and adhesion of glioma is slightly increased to endothelial cells conditioned with shear flow and moderate electric field. In conclusion, robust and versatile hybrid microsystems are employed for studying glioma biology with emphasis on cell migration. The hybrid microfluidic tools can enable us to elucidate fundamental mechanisms in the field of the tumor biology and regenerative medicine.Okinawa Institute of Science and Technology Graduate Universit

Deep Learning for Detection and Segmentation in High-Content Microscopy Images

High-content microscopy led to many advances in biology and medicine. This fast emerging technology is transforming cell biology into a big data driven science. Computer vision methods are used to automate the analysis of microscopy image data. In recent years, deep learning became popular and had major success in computer vision. Most of the available methods are developed to process natural images. Compared to natural images, microscopy images pose domain specific challenges such as small training datasets, clustered objects, and class imbalance. In this thesis, new deep learning methods for object detection and cell segmentation in microscopy images are introduced. For particle detection in fluorescence microscopy images, a deep learning method based on a domain-adapted Deconvolution Network is presented. In addition, a method for mitotic cell detection in heterogeneous histopathology images is proposed, which combines a deep residual network with Hough voting. The method is used for grading of whole-slide histology images of breast carcinoma. Moreover, a method for both particle detection and cell detection based on object centroids is introduced, which is trainable end-to-end. It comprises a novel Centroid Proposal Network, a layer for ensembling detection hypotheses over image scales and anchors, an anchor regularization scheme which favours prior anchors over regressed locations, and an improved algorithm for Non-Maximum Suppression. Furthermore, a novel loss function based on Normalized Mutual Information is proposed which can cope with strong class imbalance and is derived within a Bayesian framework. For cell segmentation, a deep neural network with increased receptive field to capture rich semantic information is introduced. Moreover, a deep neural network which combines both paradigms of multi-scale feature aggregation of Convolutional Neural Networks and iterative refinement of Recurrent Neural Networks is proposed. To increase the robustness of the training and improve segmentation, a novel focal loss function is presented. In addition, a framework for black-box hyperparameter optimization for biomedical image analysis pipelines is proposed. The framework has a modular architecture that separates hyperparameter sampling and hyperparameter optimization. A visualization of the loss function based on infimum projections is suggested to obtain further insights into the optimization problem. Also, a transfer learning approach is presented, which uses only one color channel for pre-training and performs fine-tuning on more color channels. Furthermore, an approach for unsupervised domain adaptation for histopathological slides is presented. Finally, Galaxy Image Analysis is presented, a platform for web-based microscopy image analysis. Galaxy Image Analysis workflows for cell segmentation in cell cultures, particle detection in mice brain tissue, and MALDI/H&E image registration have been developed. The proposed methods were applied to challenging synthetic as well as real microscopy image data from various microscopy modalities. It turned out that the proposed methods yield state-of-the-art or improved results. The methods were benchmarked in international image analysis challenges and used in various cooperation projects with biomedical researchers

Automatic quantification of the microvascular density on whole slide images, applied to paediatric brain tumours

Angiogenesis is a key phenomenon for tumour progression, diagnosis and treatment in brain tumours and more generally in oncology. Presently, its precise, direct quantitative assessment can only be done on whole tissue sections immunostained to reveal vascular endothelial cells. But this is a tremendous task for the pathologist and a challenge for the computer since digitised whole tissue sections, whole slide images (WSI), contain typically around ten gigapixels. We define and implement an algorithm that determines automatically, on a WSI at objective magnification $40\times$, the regions of tissue, the regions without blur and the regions of large puddles of red blood cells, and constructs the mask of blur-free, significant tissue on the WSI. Then it calibrates automatically the optical density ratios of the immunostaining of the vessel walls and of the counterstaining, performs a colour deconvolution inside the regions of blur-free tissue, and finds the vessel walls inside these regions by selecting, on the image resulting from the colour deconvolution, zones which satisfy a double-threshold criterion. A mask of vessel wall regions on the WSI is produced. The density of microvessels is finally computed as the fraction of the area of significant tissue which is occupied by vessel walls. We apply this algorithm to a set of 186 WSI of paediatric brain tumours from World Health Organisation grades I to IV. The segmentations are of very good quality although the set of slides is very heterogeneous. The computation time is of the order of a fraction of an hour for each WSI on a modest computer. The computed microvascular density is found to be robust and strongly correlates with the tumour grade. This method requires no training and can easily be applied to other tumour types and other stainings

Quantitative histopathologic assessment of perfusion MRI as a marker of glioblastoma cell infiltration in and beyond the peritumoral edema region

Background: Conventional MRI fails to detect regions of glioblastoma cell infiltration beyond the contrast‐enhanced T1 solid tumor region, with infiltrating tumor cells often migrating along host blood vessels. Purpose: To quantitatively and qualitatively analyze the correlation between perfusion MRI signal and tumor cell density in order to assess whether local perfusion perturbation could provide a useful biomarker of glioblastoma cell infiltration. Study Type: Animal model. Subjects: Mice bearing orthotopic glioblastoma xenografts generated from a patient‐derived glioblastoma cell line. Field Strength/Sequences: 7T perfusion images acquired using a high signal‐to‐noise ratio (SNR) multiple boli arterial spin labeling sequence were compared with conventional MRI (T1/T2 weighted, contrast‐enhanced T1, diffusion‐weighted, and apparent diffusion coefficient). Assessment: Immunohistochemistry sections were stained for human leukocyte antigen (probing human‐derived tumor cells). To achieve quantitative MRI‐tissue comparison, multiple histological slices cut in the MRI plane were stacked to produce tumor cell density maps acting as a “ground truth.” Statistical Tests: Sensitivity, specificity, accuracy, and Dice similarity indices were calculated and a two‐tailed, paired t‐test used for statistical analysis. Results: High comparison test results (Dice 0.62–0.72, Accuracy 0.86–0.88, Sensitivity 0.51–0.7, and Specificity 0.92–0.97) indicate a good segmentation for all imaging modalities and highlight the quality of the MRI tissue assessment protocol. Perfusion imaging exhibits higher sensitivity (0.7) than conventional MRI (0.51–0.61). MRI/histology voxel‐to‐voxel comparison revealed a negative correlation between tumor cell infiltration and perfusion at the tumor margins (P = 0.0004). Data Conclusion: These results demonstrate the ability of perfusion imaging to probe regions of low tumor cell infiltration while confirming the sensitivity limitations of conventional imaging modalities. The quantitative relationship between tumor cell density and perfusion identified in and beyond the edematous T2 hyperintensity region surrounding macroscopic tumor could be used to detect marginal tumor cell infiltration with greater accuracy

Advances in establishment and analysis of three-dimensional tumor spheroid-based functional assays for target validation and drug evaluation

There is overwhelming evidence that in vitro three-dimensional tumor cell cultures more accurately reflect the complex in vivo microenvironment than simple two-dimensional cell monolayers, not least with respect to gene expression profiles, signaling pathway activity and drug sensitivity. However, most currently available threedimensional techniques are time consuming and/or lack reproducibility; thus standardized and rapid protocols are urgently needed. To address this requirement, we have developed a versatile toolkit of reproducible three-dimensional tumor spheroid models for dynamic, automated, quantitative imaging and analysis that are compatible with routine high-throughput preclinical studies. Not only do these microplate methods measure three-dimensional tumor growth, but they have also been significantly enhanced to facilitate a range of functional assays exemplifying additional key hallmarks of cancer, namely cell motility and matrix invasion. Moreover, mutual tissue invasion and angiogenesis is accommodated by coculturing tumor spheroids with murine embryoid bodies within which angiogenic differentiation occurs. Highly malignant human tumor cells were selected to exemplify therapeutic effects of three specific molecularly-targeted agents: PI-103 (phosphatidylinositol-3-kinase (PI3K)- mammalian target of rapamycin (mTOR) inhibitor), 17-N-allylamino-17-demethoxygeldanamycin (17-AAG) (heat shock protein 90 (HSP90) inhibitor) and CCT130234 (in-house phospholipase C (PLC)g inhibitor). Fully automated analysis using a Celigo cytometer was validated for tumor spheroid growth and invasion against standard image analysis techniques, with excellent reproducibility and significantly increased throughput. In addition, we discovered key differential sensitivities to targeted agents between two-dimensional and three-dimensional cultures, and also demonstrated enhanced potency of some agents against cell migration/invasion compared with proliferation, suggesting their preferential utility in metastatic disease.: We have established and validated a suite of highly reproducible tumor microplate threedimensional functional assays to enhance the biological relevance of early preclinical cancer studies. We believe these assays will increase the translational predictive value of in vitro drug evaluation studies and reduce the need for in vivo studies by more effective triaging of compounds.This work was funded by The National Centre for the Replacement, Refinement and Reduction of Animals in Research (G1000121 ID no. 94513), Cancer Research UK (grant number C309/A8274), and by Red Tematica de Investigación Cooperativa en Cancer (RD06/0020/1022). We acknowledge NHS funding to the NIHR Biomedical Research Centre. MM is supported by a postdoctoral research contract (FIS, Program ‘Sara Borrell’, Instituto de Salud Carlos III), Ministerio de Ciencia e Innovación, Spain