Superpixel-based conditional random fields (SuperCRF) : incorporating global and local context for enhanced deep learning in melanoma histopathology

Computational pathology-based cell classification algorithms are revolutionizing the study of the tumor microenvironment and can provide novel predictive/prognosis biomarkers crucial for the delivery of precision oncology. Current algorithms used on hematoxylin and eosin slides are based on individual cell nuclei morphology with limited local context features. Here, we propose a novel multi-resolution hierarchical framework (SuperCRF) inspired by the way pathologists perceive regional tissue architecture to improve cell classification and demonstrate its clinical applications. We develop SuperCRF by training a state-of-art deep learning spatially constrained- convolution neural network (SC-CNN) to detect and classify cells from 105 high-resolution (20×) H&E-stained slides of The Cancer Genome Atlas melanoma dataset and subsequently, a conditional random field (CRF) by combining cellular neighborhood with tumor regional classification from lower resolution images (5, 1.25×) given by a superpixel-based machine learning framework. SuperCRF led to an 11.85% overall improvement in the accuracy of the state-of-art deep learning SC-CNN cell classifier. Consistent with a stroma-mediated immune suppressive microenvironment, SuperCRF demonstrated that (i) a high ratio of lymphocytes to all lymphocytes within the stromal compartment (p = 0.026) and (ii) a high ratio of stromal cells to all cells (p < 0.0001 compared to p = 0.039 for SC-CNN only) are associated with poor survival in patients with melanoma. SuperCRF improves cell classification by introducing global and local context-based information and can be implemented in combination with any single-cell classifier. SuperCRF provides valuable tools to study the tumor microenvironment and identify predictors of survival and response to therapy

Continuous single cell imaging reveals sequential steps of plasmacytoid dendritic cell development from common dendritic cell progenitors

Functionally distinct plasmacytoid and conventional dendritic cells (pDC and cDC) shape innate and adaptive immunity. They are derived from common dendritic cell progenitors (CDPs) in the murine bone marrow, which give rise to CD11c(+) MHCII- precursors with early commitment to DC subpopulations. In this study, we dissect pDC development from CDP into an ordered sequence of differentiation events by monitoring the expression of CD11c, MHC class II, Siglec H and CCR9 in CDP cultures by continuous single cell imaging and tracking. Analysis of CDP genealogies revealed a stepwise differentiation of CDPs into pDCs in a part of the CDP colonies. This developmental pathway involved an early CD11c(+) SiglecH(-) pre-DC stage and a Siglec H+ CCR9(low) precursor stage, which was followed rapidly by upregulation of CCR9 indicating final pDC differentiation. In the majority of the remaining CDP pedigrees however the Siglec H+ CCR9(low) precursor state was maintained for several generations. Thus, although a fraction of CDPs transits through precursor stages rapidly to give rise to a first wave of pDCs, the majority of CDP progeny differentiate more slowly and give rise to longer lived precursor cells which are poised to differentiate on demand

Factor graph analysis of live cell-imaging data reveals mechanisms of cell fate decisions

Motivation: Cell fate decisions have a strong stochastic component. The identification of the underlying mechanisms therefore requires a rigorous statistical analysis of large ensembles of single cells that were tracked and phenotyped over time. Results: We introduce a probabilistic framework for testing elementary hypotheses on dynamic cell behavior using time-lapse cell-imaging data. Factor graphs, probabilistic graphical models, are used to properly account for cell lineage and cell phenotype information. Our model is applied to time-lapse movies of murine granulocyte-macrophage progenitor (GMP) cells. It decides between competing hypotheses on the mechanisms of their differentiation. Our results theoretically substantiate previous experimental observations that lineage instruction, not selection is the cause for the differentiation of GMP cells into mature monocytes or neutrophil granulocytes. Availability and implementation: The Matlab source code is available at http://treschgroup.de/Genealogies.html Contact: [email protected] Supplementary information: Supplementary data are available at Bioinformatics onlin

Unsupervised automated high throughput phenotyping of RNAi time-lapse movies

Background: Gene perturbation experiments in combination with fluorescence time-lapse cell imaging are a powerful tool in reverse genetics. High content applications require tools for the automated processing of the large amounts of data. These tools include in general several image processing steps, the extraction of morphological descriptors, and the grouping of cells into phenotype classes according to their descriptors. This phenotyping can be applied in a supervised or an unsupervised manner. Unsupervised methods are suitable for the discovery of formerly unknown phenotypes, which are expected to occur in high-throughput RNAi time-lapse screens. Results: We developed an unsupervised phenotyping approach based on Hidden Markov Models (HMMs) with multivariate Gaussian emissions for the detection of knockdown-specific phenotypes in RNAi time-lapse movies. The automated detection of abnormal cell morphologies allows us to assign a phenotypic fingerprint to each gene knockdown. By applying our method to the Mitocheck database, we show that a phenotypic fingerprint is indicative of a gene's function. Conclusion: Our fully unsupervised HMM-based phenotyping is able to automatically identify cell morphologies that are specific for a certain knockdown. Beyond the identification of genes whose knockdown affects cell morphology, phenotypic fingerprints can be used to find modules of functionally related genes

Computational Tumor Infiltration Phenotypes Enable the Spatial and Genomic Analysis of Immune Infiltration in Colorectal Cancer

Cancer immunotherapy has led to significant therapeutic progress in the treatment of metastatic and formerly untreatable tumors. However, drug response rates are variable and often only a subgroup of patients will show durable response to a treatment. Biomarkers that help to select those patients that will benefit the most from immunotherapy are thus of crucial importance. Here, we aim to identify such biomarkers by investigating the tumor microenvironment, i.e., the interplay between different cell types like immune cells, stromal cells and malignant cells within the tumor and developed a computational method that determines spatial tumor infiltration phenotypes. Our method is based on spatial point pattern analysis of immunohistochemically stained colorectal cancer tumor tissue and accounts for the intra-tumor heterogeneity of immune infiltration. We show that, compared to base-line models, tumor infiltration phenotypes provide significant additional support for the prediction of established biomarkers in a colorectal cancer patient cohort (n = 80). Integration of tumor infiltration phenotypes with genetic and genomic data from the same patients furthermore revealed significant associations between spatial infiltration patterns and common mutations in colorectal cancer and gene expression signatures. Based on these associations, we computed novel gene signatures that allow one to predict spatial tumor infiltration patterns from gene expression data only and validated this approach in a separate dataset from the Cancer Genome Atlas

MowJoe: a method for automated-high throughput dissected leaf phenotyping

Background: Accurate and automated phenotyping of leaf images is necessary for high throughput studies of leaf form like genome-wide association analysis and other forms of quantitative trait locus mapping. Dissected leaves (also referred to as compound) that are subdivided into individual units are an attractive system to study diversification of form. However, there are only few software tools for their automated analysis. Thus, high-throughput image processing algorithms are needed that can partition these leaves in their phenotypically relevant units and calculate morphological features based on these units. Results: We have developed MowJoe, an image processing algorithm that dissects a dissected leaf into leaflets, petiolule, rachis and petioles. It employs image skeletonization to convert leaves into graphs, and thereafter applies algorithms operating on graph structures. This partitioning of a leaf allows the derivation of morphological features such as leaf size, or eccentricity of leaflets. Furthermore, MowJoe automatically places landmarks onto the terminal leaflet that can be used for further leaf shape analysis. It generates specific output files that can directly be imported into downstream shape analysis tools. We applied the algorithm to two accessions of Cardamine hirsuta and show that our features are able to robustly discriminate between these accessions. Conclusion: MowJoe is a tool for the semi-automated, quantitative high throughput shape analysis of dissected leaf images. It provides the statistical power for the detection of the genetic basis of quantitative morphological variations

Semi-automated 3D Leaf Reconstruction and Analysis of Trichome Patterning from Light Microscopic Images

<div><p>Trichomes are leaf hairs that are formed by single cells on the leaf surface. They are known to be involved in pathogen resistance. Their patterning is considered to emerge from a field of initially equivalent cells through the action of a gene regulatory network involving trichome fate promoting and inhibiting factors. For a quantitative analysis of single and double mutants or the phenotypic variation of patterns in different ecotypes, it is imperative to statistically evaluate the pattern reliably on a large number of leaves. Here we present a method that enables the analysis of trichome patterns at early developmental leaf stages and the automatic analysis of various spatial parameters. We focus on the most challenging young leaf stages that require the analysis in three dimensions, as the leaves are typically not flat. Our software TrichEratops reconstructs 3D surface models from 2D stacks of conventional light-microscope pictures. It allows the GUI-based annotation of different stages of trichome development, which can be analyzed with respect to their spatial distribution to capture trichome patterning events. We show that 3D modeling removes biases of simpler 2D models and that novel trichome patterning features increase the sensitivity for inter-accession comparisons.</p> </div