Towards Automatic Identification and Delineation of Tissues and Pathologies in H&E Stained Images

We propose a framework and methodology for the automated identification and delineation of tissues and their pathologies in hematoxylin and eosin (H&E) stained images. Histopathology is vital to medicine and research as it enables quantitative and qualitative analysis of tissue samples, stained and visualized via microscopes; the most routine and cost-effective of these stains is H&E. In clinical diagnostic surgical pathology, the pathologist interprets H&E-stained tissue slides by determining whether a given sample represents normal or abnormal tissue for the given anatomical location. Although pathologists accurately and consistently identify and delineate such tissues and their pathologies, this is a time-consuming and expensive task; thus the need for automated algorithms for improved throughput and robustness. We develop such an algorithm that uses local histograms and occlusion models as a mathematical framework for pixel-level classification. We also develop an expert-guided feature set called the histopathology vocabulary that mimics the visual process used by pathologists. To expand applicability, we achieve simultaneous identification and delineation by performing pixel-level classification. Experimental results on both a clinical application (active colitis) and a research one (tissue development in teratoma tumors) validate the discriminative power of our approach. We also present comparisons to popular, though general, feature types to demonstrate the power of our expert-guided feature set. Our framework and methodology demonstrates great promise towards the creation of a framework and methodology for the automated identification and delineation of tissues and their pathologies in H&E-stained images.</p

Local Histograms for Classifying H&E Stained Tissues.

We introduce a rigorous mathematical theory for the analysis of local histograms, and consider the appropriateness of their use in the automated classification of textures commonly encountered in images of H&E stained tissues. We first discuss some of the many image features that pathologists indicate they use when classifying tissues, focusing on simple, locally-defined features that essentially involve pixel counting: the number of cells in a region of given size, the size of the nuclei within these cells, and the distribution of color within both. We then introduce a probabilistic, occlusion-based model for textures that exhibit these features, in particular demonstrating how certain tissue-similar textures can be built up from simpler ones. After considering the basic notions and properties of local histogram transforms, we then formally demonstrate that such transforms are natural tools for analyzing the textures produced by our model. In particular, we discuss how local histogram transforms can be used to produce numerical features that, when fed into mainstream classification schemes, mimic the baser aspects of a pathologist's thought process.</p

Exploring Indirect Vehicle-Bridge Interaction for Bridge SHM

In this paper, we explore an indirect measurement approach for bridge structural health monitoring (SHM) that collects sensed information from the dynamic responses of many vehicles travelling over a bridge and then makes extensive use of advanced signal processing techniques to determine information about the state of the bridge. We refer to this approach as vehicle-data driven and indirect. We discuss some of the advantages of this indirect approach over direct monitoring of structures. We simplified the vehicle-bridge interaction and used a numerical oscillator-beam interaction model to generate some preliminary interaction response data with which to begin to assess the validity of this approach. A Multiresolution image classifier was used to analyze the preliminary data. We present the basic idea behind this approach and preliminary results that demonstrate its viability.</p

Local histograms and image occlusion models.

<p>The local histogram transform of an image is a data cube that consists of the histograms of the pixel values that lie within a fixed neighborhood of any given pixel location. Such transforms are useful in image processing applications such as classification and segmentation, especially when dealing with textures that can be distinguished by the distributions of their pixel intensities and colors. We, in particular, use them to identify and delineate biological tissues found in histology images obtained via digital microscopy. In this paper, we introduce a mathematical formalism that rigorously justifies the use of local histograms for such purposes. We begin by discussing how local histograms can be computed as systems of convolutions. We then introduce probabilistic image models that can emulate textures one routinely encounters in histology images. These models are rooted in the concept of image occlusion. A simple model may, for example, generate textures by randomly speckling opaque blobs of one color on top of blobs of another. Under certain conditions, we show that, on average, the local histograms of such model-generated-textures are convex combinations of more basic distributions. We further provide several methods for creating models that meet these conditions; the textures generated by some of these models resemble those found in histology images. Taken together, these results suggest that histology textures can be analyzed by decomposing their local histograms into more basic components. We conclude with a proof-of-concept segmentation-and-classification algorithm based on these ideas, supported by numerical experimentation.</p

A domain-knowledge-inspired mathematical framework for the description and classification of H&E stained histopathology images.

We present the current state of our work on a mathematical framework for identification and delineation of histopathology images-local histograms and occlusion models. Local histograms are histograms computed over defined spatial neighborhoods whose purpose is to characterize an image locally. This unit of description is augmented by our occlusion models that describe a methodology for image formation. In the context of this image formation model, the power of local histograms with respect to appropriate families of images will be shown through various proved statements about expected performance. We conclude by presenting a preliminary study to demonstrate the power of the framework in the context of histopathology image classification tasks that, while differing greatly in application, both originate from what is considered an appropriate class of images for this framework.</p

Automated colitis detection from endoscopic biopsies as a tissue screening tool in diagnostic pathology

<p>We present a method for identifying colitis in colon biopsies as an extension of our framework for the automated identification of tissues in histology images. Histology is a critical tool in both clinical and research applications, yet even mundane histological analysis, such as the screening of colon biopsies, must be carried out by highly-trained pathologists at a high cost per hour, indicating a niche for potential automation. To this end, we build upon our previous work by extending the histopathology vocabulary (a set of features based on visual cues used by pathologists) with new features driven by the colitis application. We use the multiple-instance learning framework to allow our pixel-level classifier to learn from image-level training labels. The new system achieves accuracy comparable to state-of-the-art biological image classifiers with fewer and more intuitive features.</p

A vocabulary for the identification and delineation of teratoma tissue components in hematoxylin and eosin-stained samples.

<p>We propose a methodology for the design of features mimicking the visual cues used by pathologists when identifying tissues in hematoxylin and eosin (H&E)-stained samples.</p> <p>BACKGROUND: H&E staining is the gold standard in clinical histology; it is cheap and universally used, producing a vast number of histopathological samples. While pathologists accurately and consistently identify tissues and their pathologies, it is a time-consuming and expensive task, establishing the need for automated algorithms for improved throughput and robustness.</p> <p>METHODS: We use an iterative feedback process to design a histopathology vocabulary (HV), a concise set of features that mimic the visual cues used by pathologists, e.g. "cytoplasm color" or "nucleus density". These features are based in histology and understood by both pathologists and engineers. We compare our HV to several generic texture-feature sets in a pixel-level classification algorithm.</p> <p>RESULTS: Results on delineating and identifying tissues in teratoma tumor samples validate our expert knowledge-based approach.</p> <p>CONCLUSIONS: The HV can be an effective tool for identifying and delineating teratoma components from images of H&E-stained tissue samples.</p

Automatic identification and delineation of germ layer components in H&E stained images of teratomas derived from human and nonhuman primate embryonic stem cells

We present a methodology for the automatic identification and delineation of germ-layer components in H&E stained images of teratomas derived from human and nonhuman primate embryonic stem cells. A knowledge and understanding of the biology of these cells may lead to advances in tissue regeneration and repair, the treatment of genetic and developmental syndromes, and drug testing and discovery. As a teratoma is a chaotic organization of tissues derived from the three primary embryonic germ layers, H&E teratoma images often present multiple tissues, each of having complex and unpredictable positions, shapes, and appearance with respect to each individual tissue as well as with respect to other tissues. While visual identification of these tissues is time-consuming, it is surprisingly accurate, indicating that there exist enough visual cues to accomplish the task. We propose automatic identification and delineation of these tissues by mimicking these visual cues. We use pixel-based classification, resulting in an encouraging range of classification accuracies from 74.9% to 93.2% for 2- to 5-tissue classification experiments at different scales.</p