A Comprehensive Overview of Computational Nuclei Segmentation Methods in Digital Pathology

In the cancer diagnosis pipeline, digital pathology plays an instrumental role in the identification, staging, and grading of malignant areas on biopsy tissue specimens. High resolution histology images are subject to high variance in appearance, sourcing either from the acquisition devices or the H\&E staining process. Nuclei segmentation is an important task, as it detects the nuclei cells over background tissue and gives rise to the topology, size, and count of nuclei which are determinant factors for cancer detection. Yet, it is a fairly time consuming task for pathologists, with reportedly high subjectivity. Computer Aided Diagnosis (CAD) tools empowered by modern Artificial Intelligence (AI) models enable the automation of nuclei segmentation. This can reduce the subjectivity in analysis and reading time. This paper provides an extensive review, beginning from earlier works use traditional image processing techniques and reaching up to modern approaches following the Deep Learning (DL) paradigm. Our review also focuses on the weak supervision aspect of the problem, motivated by the fact that annotated data is scarce. At the end, the advantages of different models and types of supervision are thoroughly discussed. Furthermore, we try to extrapolate and envision how future research lines will potentially be, so as to minimize the need for labeled data while maintaining high performance. Future methods should emphasize efficient and explainable models with a transparent underlying process so that physicians can trust their output.Comment: 47 pages, 27 figures, 9 table

Beyond imaging with coherent anti-Stokes Raman scattering microscopy

La microscopie optique permet de visualiser des échantillons biologiques avec une bonne sensibilité et une résolution spatiale élevée tout en interférant peu avec les échantillons. La microscopie par diffusion Raman cohérente (CARS) est une technique de microscopie non linéaire basée sur l’effet Raman qui a comme avantage de fournir un mécanisme de contraste endogène sensible aux vibrations moléculaires. La microscopie CARS est maintenant une modalité d’imagerie reconnue, en particulier pour les expériences in vivo, car elle élimine la nécessité d’utiliser des agents de contraste exogènes, et donc les problèmes liés à leur distribution, spécificité et caractère invasif. Cependant, il existe encore plusieurs obstacles à l’adoption à grande échelle de la microscopie CARS en biologie et en médecine : le coût et la complexité des systèmes actuels, les difficultés d’utilisation et d’entretient, la rigidité du mécanisme de contraste, la vitesse de syntonisation limitée et le faible nombre de méthodes d’analyse d’image adaptées. Cette thèse de doctorat vise à aller au-delà de certaines des limites actuelles de l’imagerie CARS dans l’espoir que cela encourage son adoption par un public plus large. Tout d’abord, nous avons introduit un nouveau système d’imagerie spectrale CARS ayant une vitesse de syntonisation de longueur d’onde beaucoup plus rapide que les autres techniques similaires. Ce système est basé sur un laser à fibre picoseconde synchronisé qui est à la fois robuste et portable. Il peut accéder à des lignes de vibration Raman sur une plage importante (2700–2950 cm-1) à des taux allant jusqu’à 10 000 points spectrales par seconde. Il est parfaitement adapté pour l’acquisition d’images spectrales dans les tissus épais. En second lieu, nous avons proposé une nouvelle méthode d’analyse d’images pour l’évaluation de la structure de la myéline dans des images de sections longitudinales de moelle épinière. Nous avons introduit un indicateur quantitatif sensible à l’organisation de la myéline et démontré comment il pourrait être utilisé pour étudier certaines pathologies. Enfin, nous avons développé une méthode automatisé pour la segmentation d’axones myélinisés dans des images CARS de coupes transversales de tissu nerveux. Cette méthode a été utilisée pour extraire des informations morphologique des fibres nerveuses dans des images CARS de grande échelle.Optical-based microscopy techniques can sample biological specimens using many contrast mechanisms providing good sensitivity and high spatial resolution while minimally interfering with the samples. Coherent anti-Stokes Raman scattering (CARS) microscopy is a nonlinear microscopy technique based on the Raman effect. It shares common characteristics of other optical microscopy modalities with the added benefit of providing an endogenous contrast mechanism sensitive to molecular vibrations. CARS is now recognized as a great imaging modality, especially for in vivo experiments since it eliminates the need for exogenous contrast agents, and hence problems related to the delivery, specificity, and invasiveness of those markers. However, there are still several obstacles preventing the wide-scale adoption of CARS in biology and medicine: cost and complexity of current systems as well as difficulty to operate and maintain them, lack of flexibility of the contrast mechanism, low tuning speed and finally, poor accessibility to adapted image analysis methods. This doctoral thesis strives to move beyond some of the current limitations of CARS imaging in the hope that it might encourage a wider adoption of CARS as a microscopy technique. First, we introduced a new CARS spectral imaging system with vibrational tuning speed many orders of magnitude faster than other narrowband techniques. The system presented in this original contribution is based on a synchronized picosecond fibre laser that is both robust and portable. It can access Raman lines over a significant portion of the highwavenumber region (2700–2950 cm-1) at rates of up to 10,000 spectral points per second and is perfectly suitable for the acquisition of CARS spectral images in thick tissue. Secondly, we proposed a new image analysis method for the assessment of myelin health in images of longitudinal sections of spinal cord. We introduced a metric sensitive to the organization/disorganization of the myelin structure and showed how it could be used to study pathologies such as multiple sclerosis. Finally, we have developped a fully automated segmentation method specifically designed for CARS images of transverse cross sections of nerve tissue.We used our method to extract nerve fibre morphology information from large scale CARS images

Deep learning for digitized histology image analysis

“Cervical cancer is the fourth most frequent cancer that affects women worldwide. Assessment of cervical intraepithelial neoplasia (CIN) through histopathology remains as the standard for absolute determination of cancer. The examination of tissue samples under a microscope requires considerable time and effort from expert pathologists. There is a need to design an automated tool to assist pathologists for digitized histology slide analysis. Pre-cervical cancer is generally determined by examining the CIN which is the growth of atypical cells from the basement membrane (bottom) to the top of the epithelium. It has four grades, including: Normal, CIN1, CIN2, and CIN3. In this research, different facets of an automated digitized histology epithelium assessment pipeline have been explored to mimic the pathologist diagnostic approach. The entire pipeline from slide to epithelium CIN grade has been designed and developed using deep learning models and imaging techniques to analyze the whole slide image (WSI). The process is as follows: 1) identification of epithelium by filtering the regions extracted from a low-resolution image with a binary classifier network; 2) epithelium segmentation; 3) deep regression for pixel-wise segmentation of epithelium by patch-based image analysis; 4) attention-based CIN classification with localized sequential feature modeling. Deep learning-based nuclei detection by superpixels was performed as an extension of our research. Results from this research indicate an improved performance of CIN assessment over state-of-the-art methods for nuclei segmentation, epithelium segmentation, and CIN classification, as well as the development of a prototype WSI-level tool”--Abstract, page iv

Detecting Red Blood Cells Morphological Abnormalities Using Genetic Algorithm and Kmeans

Vision is the most advanced of our senses, so it is not surprising that images play the single most important role in human perception. Computer-aided diagnosis is another important application of pattern recognition, aiming at assisting doctors in making diagnostic decisions. Many diseases which are not blood diseases in origin have hematological abnormalities and manifestation (have symptoms appeared on the blood). CBC (cell blood count test) for instance, is still the first test to be requested by the physicians or become in their mind. Blood abnormality can be in white blood cells, red blood cells and plasma. In this thesis, red blood cells are the suggested for detecting it is abnormality. The abnormality of blood cells shapes can't be detected easily, where the CBC (cell blood count) device give a count number and percentages not a description of the shapes of the blood cells, when the blood cells shapes wanted to be known, hematologist asked to view the blood films under the microscope which is time consuming task besides that the human error risk is high. Since the number of abnormal cells to normal cells in a given blood sample give a measure of the disease severity, detecting one cell with potential abnormality can give premature warning for future illness that can be avoided or treated earlier. This case can't be detected by hematologist. Computer involved in such task to save time and effort besides minimizing human error. This thesis name is "DETECTING RED BLOOD CELLS MORPHOLOGICAL ABNORMALITIES USING GENETIC ALGORITHM AND KMEANS". In this thesis, the thesis divided into four phases. First phase data collection where blood samples was drawn from healthy and sick people and then blood films made and viewed under microscope and an images captured for these blood films. Second phase preprocessing phase where the images prepared for the next phase. Third phase feature extraction was executed where these features are spatial domain and frequency domain features. Fourth phase is the classification phase where the features fed into the classifier to be classified. An acceptable detection rate is achieved by the proposed system. The genetic algorithm classifier success rate was 92.31% and the K-means classifier success rate was 94.00%

Biological image analysis

In biological research images are extensively used to monitor growth, dynamics and changes in biological specimen, such as cells or plants. Many of these images are used solely for observation or are manually annotated by an expert. In this dissertation we discuss several methods to automate the annotating and analysis of bio-images. Two large clusters of methods have been investigated and developed. A first set of methods focuses on the automatic delineation of relevant objects in bio-images, such as individual cells in microscopic images. Since these methods should be useful for many different applications, e.g. to detect and delineate different objects (cells, plants, leafs, ...) in different types of images (different types of microscopes, regular colour photographs, ...), the methods should be easy to adjust. Therefore we developed a methodology relying on probability theory, where all required parameters can easily be estimated by a biologist, without requiring any knowledge on the techniques used in the actual software. A second cluster of investigated techniques focuses on the analysis of shapes. By defining new features that describe shapes, we are able to automatically classify shapes, retrieve similar shapes from a database and even analyse how an object deforms through time

Towards Accurate and Efficient Cell Tracking During Fly Wing Development

Understanding the development, organization, and function of tissues is a central goal in developmental biology. With modern time-lapse microscopy, it is now possible to image entire tissues during development and thereby localize subcellular proteins. A particularly productive area of research is the study of single layer epithelial tissues, which can be simply described as a 2D manifold. For example, the apical band of cell adhesions in epithelial cell layers actually forms a 2D manifold within the tissue and provides a 2D outline of each cell. The Drosophila melanogaster wing has become an important model system, because its 2D cell organization has the potential to reveal mechanisms that create the final fly wing shape. Other examples include structures that naturally localize at the surface of the tissue, such as the ciliary components of planarians. Data from these time-lapse movies typically consists of mosaics of overlapping 3D stacks. This is necessary because the surface of interest exceeds the field of view of todays microscopes. To quantify cellular tissue dynamics, these mosaics need to be processed in three main steps: (a) Extracting, correcting, and stitching individ- ual stacks into a single, seamless 2D projection per time point, (b) obtaining cell characteristics that occur at individual time points, and (c) determine cell dynamics over time. It is therefore necessary that the applied methods are capable of handling large amounts of data efficiently, while still producing accurate results. This task is made especially difficult by the low signal to noise ratios that are typical in live-cell imaging. In this PhD thesis, I develop algorithms that cover all three processing tasks men- tioned above and apply them in the analysis of polarity and tissue dynamics in large epithelial cell layers, namely the Drosophila wing and the planarian epithelium. First, I introduce an efficient pipeline that preprocesses raw image mosaics. This pipeline accurately extracts the stained surface of interest from each raw image stack and projects it onto a single 2D plane. It then corrects uneven illumination, aligns all mosaic planes, and adjusts brightness and contrast before finally stitching the processed images together. This preprocessing does not only significantly reduce the data quantity, but also simplifies downstream data analyses. Here, I apply this pipeline to datasets of the developing fly wing as well as a planarian epithelium. I additionally address the problem of determining cell polarities in chemically fixed samples of planarians. Here, I introduce a method that automatically estimates cell polarities by computing the orientation of rootlets in motile cilia. With this technique one can for the first time routinely measure and visualize how tissue polarities are established and maintained in entire planarian epithelia. Finally, I analyze cell migration patterns in the entire developing wing tissue in Drosophila. At each time point, cells are segmented using a progressive merging ap- proach with merging criteria that take typical cell shape characteristics into account. The method enforces biologically relevant constraints to improve the quality of the resulting segmentations. For cases where a full cell tracking is desired, I introduce a pipeline using a tracking-by-assignment approach. This allows me to link cells over time while considering critical events such as cell divisions or cell death. This work presents a very accurate large-scale cell tracking pipeline and opens up many avenues for further study including several in-vivo perturbation experiments as well as biophysical modeling. The methods introduced in this thesis are examples for computational pipelines that catalyze biological insights by enabling the quantification of tissue scale phenomena and dynamics. I provide not only detailed descriptions of the methods, but also show how they perform on concrete biological research projects

New algorithms for the analysis of live-cell images acquired in phase contrast microscopy

La détection et la caractérisation automatisée des cellules constituent un enjeu important dans de nombreux domaines de recherche tels que la cicatrisation, le développement de l'embryon et des cellules souches, l’immunologie, l’oncologie, l'ingénierie tissulaire et la découverte de nouveaux médicaments. Étudier le comportement cellulaire in vitro par imagerie des cellules vivantes et par le criblage à haut débit implique des milliers d'images et de vastes quantités de données. Des outils d'analyse automatisés reposant sur la vision numérique et les méthodes non-intrusives telles que la microscopie à contraste de phase (PCM) sont nécessaires. Comme les images PCM sont difficiles à analyser en raison du halo lumineux entourant les cellules et de la difficulté à distinguer les cellules individuelles, le but de ce projet était de développer des algorithmes de traitement d'image PCM dans Matlab® afin d’en tirer de l’information reliée à la morphologie cellulaire de manière automatisée. Pour développer ces algorithmes, des séries d’images de myoblastes acquises en PCM ont été générées, en faisant croître les cellules dans un milieu avec sérum bovin (SSM) ou dans un milieu sans sérum (SFM) sur plusieurs passages. La surface recouverte par les cellules a été estimée en utilisant un filtre de plage de valeurs, un seuil et une taille minimale de coupe afin d'examiner la cinétique de croissance cellulaire. Les résultats ont montré que les cellules avaient des taux de croissance similaires pour les deux milieux de culture, mais que celui-ci diminue de façon linéaire avec le nombre de passages. La méthode de transformée par ondelette continue combinée à l’analyse d'image multivariée (UWT-MIA) a été élaborée afin d’estimer la distribution de caractéristiques morphologiques des cellules (axe majeur, axe mineur, orientation et rondeur). Une analyse multivariée réalisée sur l’ensemble de la base de données (environ 1 million d’images PCM) a montré d'une manière quantitative que les myoblastes cultivés dans le milieu SFM étaient plus allongés et plus petits que ceux cultivés dans le milieu SSM. Les algorithmes développés grâce à ce projet pourraient être utilisés sur d'autres phénotypes cellulaires pour des applications de criblage à haut débit et de contrôle de cultures cellulaires.Automated cell detection and characterization is important in many research fields such as wound healing, embryo development, immune system studies, cancer research, parasite spreading, tissue engineering, stem cell research and drug research and testing. Studying in vitro cellular behavior via live-cell imaging and high-throughput screening involves thousands of images and vast amounts of data, and automated analysis tools relying on machine vision methods and non-intrusive methods such as phase contrast microscopy (PCM) are a necessity. However, there are still some challenges to overcome, since PCM images are difficult to analyze because of the bright halo surrounding the cells and blurry cell-cell boundaries when they are touching. The goal of this project was to develop image processing algorithms to analyze PCM images in an automated fashion, capable of processing large datasets of images to extract information related to cellular viability and morphology. To develop these algorithms, a large dataset of myoblasts images acquired in live-cell imaging (in PCM) was created, growing the cells in either a serum-supplemented (SSM) or a serum-free (SFM) medium over several passages. As a result, algorithms capable of computing the cell-covered surface and cellular morphological features were programmed in Matlab®. The cell-covered surface was estimated using a range filter, a threshold and a minimum cut size in order to look at the cellular growth kinetics. Results showed that the cells were growing at similar paces for both media, but their growth rate was decreasing linearly with passage number. The undecimated wavelet transform multivariate image analysis (UWT-MIA) method was developed, and was used to estimate cellular morphological features distributions (major axis, minor axis, orientation and roundness distributions) on a very large PCM image dataset using the Gabor continuous wavelet transform. Multivariate data analysis performed on the whole database (around 1 million PCM images) showed in a quantitative manner that myoblasts grown in SFM were more elongated and smaller than cells grown in SSM. The algorithms developed through this project could be used in the future on other cellular phenotypes for high-throughput screening and cell culture control applications