Automatic annotation of cellular data

Life scientists often need to count cells in microscopy images, which is very tedious and a time consuming task. Henceforth, automatic approaches can be a solution to this problem. Several works have been devised for this issue, but the majority of these approaches degrade their performance in case of cell overlapping. In this dissertation we propose a method to determine the position of macrophages and parasites in uorescence images of Leishmania-infected macrophages. The proposed strategy is mainly based on blob detection, clustering and separation using concave regions of the cells' contour. By carrying out a comparison with other approaches that also addressed this type of images, we concluded that the proposed methodology achieves better performance in the automatic annotation of Leishmania infections.A anotação de células é uma tarefa comum a diversas áreas da investigação biomédica. Normalmente, esta tarefa é realizada de forma manual, sendo um processo demorado, cansativo e propício a erros. Neste trabalho, focamos o nosso interesse na anotação de imagens de uorescência com infeções de Leishmania, que representa um destes casos. Leishmania são parasitas unicelulares que infectam mamíferos, sendo responsáveis por um conjunto de doenças conhecidas por leishmanioses. Leishmania usam vertebrados como hospedeiros residindo dentro dos seus macrófagos. Por conseguinte, um modelo adequado para o estudo destes parasitas é infectar in vitro culturas de macrófagos. A capacidade de sobrevivência/replicação da Leishmania nessas condições arti - ciais pode então ser avaliada por parâmetros, como, por exemplo, a percentagem de macrófagos infectados, o número médio de parasitas por macrófagos infectados e o índice de infeção. Essas métricas são geralmente determinadas pela contagem de parasitas e macrófagos ao microscópio. Ambos os tipos de células podem ser facilmente distinguidos com base no seu tamanho e cor, resultante de diferentes a nidades de corantes uorescentes. A passagem desta tarefa do microscópio para o computador já foi conseguida através de aplicações como o CellNote, contudo, apesar de mais fácil e interativa, a anotação continua a ser manual. A evolução da abordagem manual para um processo automático representa um passo natural e lógico, constituindo o principal objetivo deste trabalho. Para isto iniciámos a investigação pela revisão dos principais métodos de deteção e contagem celular. As características das imagens com infeções de Leishmania impossibilitam a utilização dos métodos estudados, de tal modo que optámos por desenvolver uma nova abordagem, capaz de lidar com as várias especi cidades destas imagens. Também durante o processo de revis ão de literatura analisámos os dois métodos previamente propostos para realizar a anotação automática de infeções de Leishmania. Estes revelaram um desempenho abaixo do requerido pelos parasitologistas, justi cando também o desenvolvimento de uma nova abordagem. Durante a concepção do sistema investigámos diversas técnicas de deteção celular, onde a deteção de blobs se destacou pelos resultados positivos. Para segmentar as regiões citoplasmáticas optámos pela utilização de algoritmos de clustering. Estes não foram capazes de solucionar casos em que existia sobreposição de estruturas celulares, motivando assim o método de separação desenvolvido. Este método baseia-se maioritariamente na análise de contorno, sendo as suas concavidades geradoras de separação entre citoplasmas. Através da combinação destas fases foi possível detetar macrófagos e parasitas com mais precisão. Para con rmar esta conclusão testámos não só a nossa abordagem mas também as duas abordagens previamente desenvolvidas para este problema. Os desempenhos alcançados evidenciam não só uma melhoria comparativamente às restantes abordagens como também mostram que a nossa abordagem assegura resultados satisfatórios comparativamente aos obtidos manualmente. Em suma, o trabalho desenvolvido produziu um sistema capaz de realizar a anotação automática de imagens de uorescência com infeções de Leishmania, tendo originado um artigo aceite para publicação na conferência International Conference on Image Analysis and Recognition (ICIAR) 2013

Automatic biological object segmentation and tracking in unconstrained microscopic video conditions

Cell and small biological organism tracking research is of fundamental importance for the analysis of dynamic behaviour for assisting the development of many biomedical image related applications. With the rapid development of digitised imaging systems, the immense collections of experimental (microscopic) videos make it nearly impossible to manually analyse the obtained data. Therefore, recent research has drawn attention to building automatic tracking systems to track the movement of cells and small biological organism models using videos taken by microscopes. Although general object tracking (such as traffic cars and pedestrians) has been studied for decades, existing general object tracking systems cannot directly be applied to cell and small biological organism tracking, due to the differences in the imaging devices and conditions of the targets. This research therefore investigates the novel application of computer vision techniques to reliably, accurately and effectively track the movement of cells and small biological organisms automatically. Due to difficulties in generating video segmentation ground-truth, there is a general lack of segmentation datasets with annotated ground-truth (particularly for biomedical images). This work proposes an efficient and scalable crowdsourced approach to generate video segmentation ground-truth and develops a tracking ground-truth generation system. To illustrate the proposed approach, an annotated zebrafish larvae video segmentation dataset and three tracking datasets have been generated and made freely available online. Automatic cell tracking techniques require accurate cell image segmentation; however, current general object segmentation techniques are susceptible to errors due to the poor microscopic imaging conditions, which include low contrast typical of cell microscopic images. This work proposes a novel image pre-processing technique to enhance low greyscale image contrast for improved cell image segmentation accuracy. An adaptive, shifted bi-Gaussian mixture model is matched to the original cell image intensity histogram for greater differentiation between the cell foreground and image background, while maintaining the original intensity histogram shape. Small biological organism videos taken by microscope imaging devices under realistic experimental conditions have more complex video backgrounds than cell videos. This work first investigates single zebrafish larvae tracking using dense SIFT flow and downsampling techniques. Many existing multiple small organism tracking systems require very strict video imaging conditions, which typically result in unreliable tracking results for realistic experimental conditions. Thus, this research further investigates the adaptation of advanced segmentation techniques to improve the performance of small organism segmentation under complex imaging conditions. Finally, this work improves the multiple object association method based on the segmentation module for the proposed system, to address object misdetection and overlapping problems. This system is then evaluated on zebrafish videos, Artemia franciscana videos and Daphnia magna videos, under a wide variety of (complex) video conditions, including shadowing, labels, and background artefacts (such as water bubbles of different sizes). The tracking accuracy of the proposed system outperforms three existing tracking systems. Thus, the work in this thesis has contributions in automatic cell and biological organism tracking, where the investigation studied the region-based segmentation dataset construction generalised for biological organisms, intensity contrast enhancement for micrographs, segmentation improvement by removing imaging constraints and the final tracking accuracy enhancement

Mathematical Morphology for Quantification in Biological & Medical Image Analysis

Mathematical morphology is an established field of image processing first introduced as an application of set and lattice theories. Originally used to characterise particle distributions, mathematical morphology has gone on to be a core tool required for such important analysis methods as skeletonisation and the watershed transform. In this thesis, I introduce a selection of new image analysis techniques based on mathematical morphology. Utilising assumptions of shape, I propose a new approach for the enhancement of vessel-like objects in images: the bowler-hat transform. Built upon morphological operations, this approach is successful at challenges such as junctions and robust against noise. The bowler-hat transform is shown to give better results than competitor methods on challenging data such as retinal/fundus imagery. Building further on morphological operations, I introduce two novel methods for particle and blob detection. The first of which is developed in the context of colocalisation, a standard biological assay, and the second, which is based on Hilbert-Edge Detection And Ranging (HEDAR), with regard to nuclei detection and counting in fluorescent microscopy. These methods are shown to produce accurate and informative results for sub-pixel and supra-pixel object counting in complex and noisy biological scenarios. I propose a new approach for the automated extraction and measurement of object thickness for intricate and complicated vessels, such as brain vascular in medical images. This pipeline depends on two key technologies: semi-automated segmentation by advanced level-set methods and automatic thickness calculation based on morphological operations. This approach is validated and results demonstrating the broad range of challenges posed by these images and the possible limitations of this pipeline are shown. This thesis represents a significant contribution to the field of image processing using mathematical morphology and the methods within are transferable to a range of complex challenges present across biomedical image analysis

Segmentation of epidermal tissue with histopathological damage in images of haematoxylin and eosin stained human skin.

Background: Digital image analysis has the potential to address issues surrounding traditional histological techniques including a lack of objectivity and high variability, through the application of quantitative analysis. A key initial step in image analysis is the identification of regions of interest. A widely applied methodology is that of segmentation. This paper proposes the application of image analysis techniques to segment skin tissue with varying degrees of histopathological damage. The segmentation of human tissue is challenging as a consequence of the complexity of the tissue structures and inconsistencies in tissue preparation, hence there is a need for a new robust method with the capability to handle the additional challenges materialising from histopathological damage.Methods: A new algorithm has been developed which combines enhanced colour information, created following a transformation to the L*a*b* colourspace, with general image intensity information. A colour normalisation step is included to enhance the algorithm's robustness to variations in the lighting and staining of the input images. The resulting optimised image is subjected to thresholding and the segmentation is fine-tuned using a combination of morphological processing and object classification rules. The segmentation algorithm was tested on 40 digital images of haematoxylin & eosin (H&E) stained skin biopsies. Accuracy, sensitivity and specificity of the algorithmic procedure were assessed through the comparison of the proposed methodology against manual methods.Results: Experimental results show the proposed fully automated methodology segments the epidermis with a mean specificity of 97.7%, a mean sensitivity of 89.4% and a mean accuracy of 96.5%. When a simple user interaction step is included, the specificity increases to 98.0%, the sensitivity to 91.0% and the accuracy to 96.8%. The algorithm segments effectively for different severities of tissue damage.Conclusions: Epidermal segmentation is a crucial first step in a range of applications including melanoma detection and the assessment of histopathological damage in skin. The proposed methodology is able to segment the epidermis with different levels of histological damage. The basic method framework could be applied to segmentation of other epithelial tissues

Automatic plant features recognition using stereo vision for crop monitoring

Machine vision and robotic technologies have potential to accurately monitor plant parameters which reflect plant stress and water requirements, for use in farm management decisions. However, autonomous identification of individual plant leaves on a growing plant under natural conditions is a challenging task for vision-guided agricultural robots, due to the complexity of data relating to various stage of growth and ambient environmental conditions. There are numerous machine vision studies that are concerned with describing the shape of leaves that are individually-presented to a camera. The purpose of these studies is to identify plant species, or for the autonomous detection of multiple leaves from small seedlings under greenhouse conditions. Machine vision-based detection of individual leaves and challenges presented by overlapping leaves on a developed plant canopy using depth perception properties under natural outdoor conditions is yet to be reported. Stereo vision has recently emerged for use in a variety of agricultural applications and is expected to provide an accurate method for plant segmentation and identification which can benefit from depth properties and robustness. This thesis presents a plant leaf extraction algorithm using a stereo vision sensor. This algorithm is used on multiple leaf segmentation and overlapping leaves separation using a combination of image features, specifically colour, shape and depth. The separation between the connected and the overlapping leaves relies on the measurement of the discontinuity in depth gradient for the disparity maps. Two techniques have been developed to implement this task based on global and local measurement. A geometrical plane from each segmented leaf can be extracted and used to parameterise a 3D model of the plant image and to measure the inclination angle of each individual leaf. The stem and branch segmentation and counting method was developed based on the vesselness measure and Hough transform technique. Furthermore, a method for reconstructing the segmented parts of hibiscus plants is presented and a 2.5D model is generated for the plant. Experimental tests were conducted with two different selected plants: cotton of different sizes, and hibiscus, in an outdoor environment under varying light conditions. The proposed algorithm was evaluated using 272 cotton and hibiscus plant images. The results show an observed enhancement in leaf detection when utilising depth features, where many leaves in various positions and shapes (single, touching and overlapping) were detected successfully. Depth properties were more effective in separating between occluded and overlapping leaves with a high separation rate of 84% and these can be detected automatically without adding any artificial tags on the leaf boundaries. The results exhibit an acceptable segmentation rate of 78% for individual plant leaves thereby differentiating the leaves from their complex backgrounds and from each other. The results present almost identical performance for both species under various lighting and environmental conditions. For the stem and branch detection algorithm, experimental tests were conducted on 64 colour images of both species under different environmental conditions. The results show higher stem and branch segmentation rates for hibiscus indoor images (82%) compared to hibiscus outdoor images (49.5%) and cotton images (21%). The segmentation and counting of plant features could provide accurate estimation about plant growth parameters which can be beneficial for many agricultural tasks and applications

Face Detection And Lip Localization

Integration of audio and video signals for automatic speech recognition has become an important field of study. The Audio-Visual Speech Recognition (AVSR) system is known to have accuracy higher than audio-only or visual-only system. The research focused on the visual front end and has been centered around lip segmentation. Experiments performed for lip feature extraction were mainly done in constrained environment with controlled background noise. In this thesis we focus our attention to a database collected in the environment of a moving car which hampered the quality of the imagery. We first introduce the concept of illumination compensation, where we try to reduce the dependency of light from over- or under-exposed images. As a precursor to lip segmentation, we focus on a robust face detection technique which reaches an accuracy of 95%. We have detailed and compared three different face detection techniques and found a successful way of concatenating them in order to increase the overall accuracy. One of the detection techniques used was the object detection algorithm proposed by Viola-Jones. We have experimented with different color spaces using the Viola-Jones algorithm and have reached interesting conclusions. Following face detection we implement a lip localization algorithm based on the vertical gradients of hybrid equations of color. Despite the challenging background and image quality, success rate of 88% was achieved for lip segmentation

Digital Rock Reconstruction And Property Calculation Of Fractured Shale Rock Samples

As the preferential flow channels in the shale reservoir, the fracture systems including the natural micro-cracks and hydraulic fractures have received great attention from the whole energy industry worldwide. However, it is challenging to quantify the fracture systems in the shale rocks precisely because most of well-developed “histogram-based” image processing techniques cannot handle the case of small target segmentation. Because the fracture apertures are very thin, the over-segmentation or insufficient segmentation would lead to significant error in the quantification, including the fracture porosity, aperture, length, tortuosity etc., which would lead to serious mistakes to the property calculation. In this research, two novel image processing methods are proposed. The self-adaptive image enhancement method employs incomplete beta function and simulated annealing algorithm to modify the grayscale intensity histogram. The contrast between the target and the background of the transformed gray image reaches the maximum. Also, “self-adaptive” means the enhancement process is specified by the input images. The comparison of segmentation results before and after the image enhancement show that the target becomes more obvious to the naked eyes and the precise fracture porosity of the test image is 4.02 %. The multi-stage image segmentation (MSS) method combines the global and local information of the image to finish the segmentation. The generated three-dimensional model provides visualization of the fracture systems existing in the core. Also, the important parameters of the fractures can be obtained, including aperture, length, tortuosity, and porosity. Compared with the real permeability from the core-flooding experiments, the permeability calculated from the MSS method has the minimum error of 22.1 %. The results show that the proposed methods in this research can be effective tools for the precise quantification of the thin fracture systems