SEGMENTATION AND INFORMATICS IN MULTIDIMENSIONAL FLUORESCENCE OPTICAL MICROSCOPY IMAGES

Recent advances in the field of optical microscopy have enabled scientists to observe and image complex biological processes across a wide range of spatial and temporal resolution, resulting in an exponential increase in optical microscopy data. Manual analysis of such large volumes of data is extremely time consuming and often impossible if the changes cannot be detected by the human eye. Naturally it is essential to design robust, accurate and high performance image processing and analysis tools to extract biologically significant results. Furthermore, the presentation of the results to the end-user, post analysis, is also an equally challenging issue, especially when the data (and/or the hypothesis) involves several spatial/hierarchical scales (e.g., tissues, cells, (sub)-nuclear components). This dissertation concentrates on a subset of such problems such as robust edge detection, automatic nuclear segmentation and selection in multi-dimensional tissue images, spatial analysis of gene localization within the cell nucleus, information visualization and the development of a computational framework for efficient and high-throughput processing of large datasets. Initially, we have developed 2D nuclear segmentation and selection algorithms which help in the development of an integrated approach for determining the preferential spatial localization of certain genes within the cell nuclei which is emerging as a promising technique for the diagnosis of breast cancer. Quantification requires accurate segmentation of 100 to 200 cell nuclei in each patient tissue sample in order to draw a statistically significant result. Thus, for large scale analysis involving hundreds of patients, manual processing is too time consuming and subjective. We have developed an integrated workflow that selects, following 2D automatic segmentation, a sub-population of accurately delineated nuclei for positioning of fluorescence in situ hybridization labeled genes of interest in tissue samples. Application of the method was demonstrated for discriminating normal and cancerous breast tissue sections based on the differential positioning of the HES5 gene. Automatic results agreed with manual analysis in 11 out of 14 cancers, all 4 normal cases and all 5 non-cancerous breast disease cases, thus showing the accuracy and robustness of the proposed approach. As a natural progression from the 2D analysis algorithms to 3D, we first developed a robust and accurate probabilistic edge detection method for 3D tissue samples since several down stream analysis procedures such as segmentation and tracking rely on the performance of edge detection. The method based on multiscale and multi-orientation steps surpasses several other conventional edge detectors in terms of its performance. Subsequently, given an appropriate edge measure, we developed an optimal graphcut-based 3D nuclear segmentation technique for samples where the cell nuclei are volume or surface labeled. It poses the problem as one of finding minimal closure in a directed graph and solves it efficiently using the maxflow-mincut algorithm. Both interactive and automatic versions of the algorithm are developed. The algorithm outperforms, in terms of three metrics that are commonly used to evaluate segmentation algorithms, a recently reported geodesic distance transform-based 3D nuclear segmentation method which in turns was reported to outperform several other popular tools that segment 3D nuclei in tissue samples. Finally, to apply some of the aforementioned methods to large microscopic datasets, we have developed a user friendly computing environment called MiPipeline which supports high throughput data analysis, data and process provenance, visual programming and seamlessly integrated information visualization of hierarchical biological data. The computational part of the environment is based on LONI Pipeline distributed computing server and the interactive information visualization makes use of several javascript based libraries to visualize an XML-based backbone file populated with essential meta-data and results

DeepSynth: Three-dimensional nuclear segmentation of biological images using neural networks trained with synthetic data

The scale of biological microscopy has increased dramatically over the past ten years, with the development of new modalities supporting collection of high-resolution fluorescence image volumes spanning hundreds of microns if not millimeters. The size and complexity of these volumes is such that quantitative analysis requires automated methods of image processing to identify and characterize individual cells. For many workflows, this process starts with segmentation of nuclei that, due to their ubiquity, ease-of-labeling and relatively simple structure, make them appealing targets for automated detection of individual cells. However, in the context of large, three-dimensional image volumes, nuclei present many challenges to automated segmentation, such that conventional approaches are seldom effective and/or robust. Techniques based upon deep-learning have shown great promise, but enthusiasm for applying these techniques is tempered by the need to generate training data, an arduous task, particularly in three dimensions. Here we present results of a new technique of nuclear segmentation using neural networks trained on synthetic data. Comparisons with results obtained using commonly-used image processing packages demonstrate that DeepSynth provides the superior results associated with deep-learning techniques without the need for manual annotation

Computational Methods for Delineating Multiple Nuclear Phenotypes from Different Imaging Modalities

Characterizing histopathology or organoid models of breast cancer can provide fundamental knowledge that will lead to a better understanding of tumors, response to therapeutic agents, and discovery of new targeted therapies. To this aim, the delineation of nuclei is significantly interesting since it provides rich information about the aberrant microanatomy or colony formation. For example, (i) cancer cells tend to be larger and, if coupled with high chromatin content, may indicate aneuploidy; (ii) cellular density can be the result of rapid proliferation; (iii) nuclear micro-texture can be a surrogate for fluctuation of heterochromatin patterns, where epigenetic aberrations in cancers are sometimes correlated with alterations in heterochromatin distribution; and (iv) normalized colony formation of cancer cells, in 3D culture, can serve as a surrogate metric for tumor suppression. These evidences suggest that nuclear segmentation and profiling is a major step for subsequent bioinformatics analysis. However, there are two barriers which include technical variations during the sample preparation step and biological heterogeneity since no two patients/samples are alike. As a result of these complexities, extension of deep learning methodologies will have a significant impact on the robust characterization and profiling of pathology sections or organoid models. In this presentation, we demonstrate that integration of regional and contextual representations, within the framework of a deep encoder-decoder architecture, contribute to robust delineation of various nuclear phenotypes from both bright field and confocal microscopy. The deep encoder-decoder architecture can infer perceptual boundaries that are necessary to decompose clumps of nuclei. The method has been validated on pathology section and organoid models of human mammary epithelial cells

AI in Medical Imaging Informatics: Current Challenges and Future Directions

This paper reviews state-of-the-art research solutions across the spectrum of medical imaging informatics, discusses clinical translation, and provides future directions for advancing clinical practice. More specifically, it summarizes advances in medical imaging acquisition technologies for different modalities, highlighting the necessity for efficient medical data management strategies in the context of AI in big healthcare data analytics. It then provides a synopsis of contemporary and emerging algorithmic methods for disease classification and organ/ tissue segmentation, focusing on AI and deep learning architectures that have already become the de facto approach. The clinical benefits of in-silico modelling advances linked with evolving 3D reconstruction and visualization applications are further documented. Concluding, integrative analytics approaches driven by associate research branches highlighted in this study promise to revolutionize imaging informatics as known today across the healthcare continuum for both radiology and digital pathology applications. The latter, is projected to enable informed, more accurate diagnosis, timely prognosis, and effective treatment planning, underpinning precision medicine

New marked point process models for microscopy images

In developing new materials, the characterization of microstructures is one of the key steps. To characterize the microstructure, many microscope modalities have been devised and improved over decades. With the increase in image resolution in the spatial and time domains, the amount of image data keeps increasing in the fields such as materials science and biomedical engineering. As a result, image processing plays a critical role in this era of science and technology. In materials image analysis, image segmentation and feature detection are considered very important. The first part of this research aims to resolve the segmentation problem caused by blurring artifacts in scanning electron microscopy (SEM) images. This blurring issue can lead to a bridged channel problem, which becomes an obstacle in analyzing the microstructures. To tackle the problem, we propose a joint deconvolution and segmentation (JDS) method. As a segmentation method, we use the expectation-maximization/maximization of the posterior marginals (EM/MPM) method, using the Markov random field (MRF) prior model. Experiments show the proposed method improves the segmentation result at object boundaries. The next phase of the image segmentation is detecting image features. In the second part of this research, we detect channel configurations in materials images. We propose a new approach of channel identification, based on the marked point process (MPP) framework, to effectively detect channels in materials images. To describe a higher level of structures in an image, the MPP framework is more effective than the MRF prior model. The reversible-jump Markov chain Monte Carlo (RJMCMC) algorithm embedded with simulated annealing is used as an optimization method, and a new switching kernel in an RJMCMC is used to reduce computational time. The channel configuration is useful in characterizing materials images. In addition, this information can be used to reduce the bridged channel problem more effectively. In materials image processing, one of the most important goals of feature detection is identifying the 3D structure of objects from 3D microscope datasets. The final part of this research is to perform fast and accurate estimation of 3D object configurations from a 3D dataset. We propose a fast 3D fitting method to improve the computational complexity over a full-search 3D MPP method. Experiments show that the fast 3D fitting method significantly decreases execution time compared to the full 3D MPP method

Interactive Learning for the Analysis of Biomedical and Industrial Imagery

In der vorliegenden Dissertation werden Methoden des überwachten Lernens untersucht und auf die Analyse und die Segmentierung digitaler Bilddaten angewendet, die aus diversen Forschungsgebieten stammen. Die Segmentierung und die Klassifikation spielen eine wichtige Rolle in der biomedizinischen und industriellen Bildverarbeitung, häufig basiert darauf weitere Erkennung und Quantifikation. Viele problemspezifische Ansätze existieren für die unterschiedlichsten Fragestellungen und nutzen meist spezifisches Vorwissen aus den jeweiligen Bilddaten aus. In dieser Arbeit wird ein überwachtes Lernverfahren vorgestellt, das mehrere Objekte und deren Klassen gleichzeitig segmentieren und unterscheiden kann. Die Methode ist generell genug um einen wichtigen Bereich von Anwendungen abzudecken, für deren Lösung lokale Merkmale eine Rolle spielen. Segmentierungsergebnisse dieses Ansatzes werden auf verschiedenen Datensätzen mit unterschiedlichen Problemstellungen gezeigt. Die Resultate unterstreichen die Anwendbarkeit der Lernmethode für viele biomedizinische und industrielle Anwendungen, ohne dass explizite Kenntnisse der Bildverarbeitung und Programmierung vorausgesetzt werden müssen. Der Ansatz basiert auf generellen Merkmalsklassen, die es erlauben lokal Strukturen wie Farbe, Textur und Kanten zu beschreiben. Zu diesem Zweck wurde eine interaktive Software implementiert, welche, für gewöhnliche Bildgrößen, in Echtzeit arbeitet und es somit einem Domänenexperten erlaubt Segmentierungs- und Klassifikationsaufgaben interaktiv zu bearbeiten. Dafür sind keine Kenntnisse in der Bildverarbeitung nötig, da sich die Benutzerinteraktion auf intuitives Markieren mit einem Pinselwerkzeug beschränkt. Das interaktiv trainierte System kann dann ohne weitere Benutzerinteraktion auf viele neue Bilder angewendet werden. Der Ansatz ist auf Segmentierungsprobleme beschränkt, für deren Lösung lokale diskriminative Merkmale ausreichen. Innerhalb dieser Einschränkung zeigt der Algorithmus jedoch erstaunlich gute Resultate, die in einer applikationsspezifischen Prozedur weiter verbessert werden können. Das Verfahren unterstützt bis zu vierdimensionale, multispektrale Bilddaten in vereinheitlichter Weise. Um die Anwendbar- und Übertragbarkeit der Methode weiter zu illustrieren wurden mehrere echte Anwendungsfälle, kommend aus verschiedenen bildgebenden Bereichen, untersucht. Darunter sind u. A. die Segmentierung von Tumorgewebe, aufgenommen mittelsWeitfeldmikroskopie, die Quantifikation von Zellwanderungen in konfokalmikroskopischen Aufnahmen für die Untersuchung der adulten Neurogenese, die Segmentierung von Blutgefäßen in der Retina des Auges, das Verfolgen von Kupferdrähten in einer Anwendung zur Produktauthentifikation und die Qualitätskontrolle von Mikroskopiebildern im Kontext von Hochdurchsatz-Experimenten. Desweiteren wurde eine neue Klassifikationsmethode basierend auf globalen Frequenzschätzungen für die Prozesskontrolle des Papieranlegers an Druckmaschinen entwickelt

Mathematical Morphology for Quantification in Biological & Medical Image Analysis

Mathematical morphology is an established field of image processing first introduced as an application of set and lattice theories. Originally used to characterise particle distributions, mathematical morphology has gone on to be a core tool required for such important analysis methods as skeletonisation and the watershed transform. In this thesis, I introduce a selection of new image analysis techniques based on mathematical morphology. Utilising assumptions of shape, I propose a new approach for the enhancement of vessel-like objects in images: the bowler-hat transform. Built upon morphological operations, this approach is successful at challenges such as junctions and robust against noise. The bowler-hat transform is shown to give better results than competitor methods on challenging data such as retinal/fundus imagery. Building further on morphological operations, I introduce two novel methods for particle and blob detection. The first of which is developed in the context of colocalisation, a standard biological assay, and the second, which is based on Hilbert-Edge Detection And Ranging (HEDAR), with regard to nuclei detection and counting in fluorescent microscopy. These methods are shown to produce accurate and informative results for sub-pixel and supra-pixel object counting in complex and noisy biological scenarios. I propose a new approach for the automated extraction and measurement of object thickness for intricate and complicated vessels, such as brain vascular in medical images. This pipeline depends on two key technologies: semi-automated segmentation by advanced level-set methods and automatic thickness calculation based on morphological operations. This approach is validated and results demonstrating the broad range of challenges posed by these images and the possible limitations of this pipeline are shown. This thesis represents a significant contribution to the field of image processing using mathematical morphology and the methods within are transferable to a range of complex challenges present across biomedical image analysis

Computational models for stuctural analysis of retinal images

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University LondonThe evaluation of retina structures has been of great interest because it could be used as a non-intrusive diagnosis in modern ophthalmology to detect many important eye diseases as well as cardiovascular disorders. A variety of retinal image analysis tools have been developed to assist ophthalmologists and eye diseases experts by reducing the time required in eye screening, optimising the costs as well as providing efficient disease treatment and management systems. A key component in these tools is the segmentation and quantification of retina structures. However, the imaging artefacts such as noise, intensity homogeneity and the overlapping tissue of retina structures can cause significant degradations to the performance of these automated image analysis tools. This thesis aims to provide robust and reliable automated retinal image analysis technique to allow for early detection of various retinal and other diseases. In particular, four innovative segmentation methods have been proposed, including two for retinal vessel network segmentation, two for optic disc segmentation and one for retina nerve fibre layers detection. First, three pre-processing operations are combined in the segmentation method to remove noise and enhance the appearance of the blood vessel in the image, and a Mixture of Gaussians is used to extract the blood vessel tree. Second, a graph cut segmentation approach is introduced, which incorporates the mechanism of vectors flux into the graph formulation to allow for the segmentation of very narrow blood vessels. Third, the optic disc segmentation is performed using two alternative methods: the Markov random field image reconstruction approach detects the optic disc by removing the blood vessels from the optic disc area, and the graph cut with compensation factor method achieves that using prior information of the blood vessels. Fourth, the boundaries of the retinal nerve fibre layer (RNFL) are detected by adapting a graph cut segmentation technique that includes a kernel-induced space and a continuous multiplier based max-flow algorithm. The strong experimental results of our retinal blood vessel segmentation methods including Mixture of Gaussian, Graph Cut achieved an average accuracy of 94:33%, 94:27% respectively. Our optic disc segmentation methods including Markov Random Field and Compensation Factor also achieved an average sensitivity of 92:85% and 85:70% respectively. These results obtained on several public datasets and compared with existing methods have shown that our proposed methods are robust and efficient in the segmenting retinal structures such the blood vessels and the optic disc.Brunel University Londonhttp://bura.brunel.ac.uk/bitstream/2438/10387/1/FulltextThesis.pd