Symmetric Dense Inception Network for Simultaneous Cell Detection and Classification in Multiplex Immunohistochemistry Images

Deep-learning based automatic analysis of the multiplex immunohistochemistry (mIHC) enables distinct cell populations to be localized on a large scale, providing insights into disease biology and therapeutic targets. However, standard deep-learning pipelines performed cell detection and classification as two-stage tasks, which is computationally inefficient and faces challenges to incorporate neighbouring tissue context for determining the cell identity. To overcome these limitations and to obtain a more accurate mapping of cell phenotypes, we presented a symmetric dense inception neural network for detecting and classifying cells in mIHC slides simultaneously. The model was applied with a novel stop-gradient strategy and a loss function accounted for class imbalance. When evaluated on an ovarian cancer dataset containing 6 cell types, the model achieved an F1 score of 0.835 in cell detection, and a weighted F1-score of 0.867 in cell classification, which outperformed separate models trained on individual tasks by 1.9% and 3.8% respectively. Taken together, the proposed method boosts the learning efficiency and prediction accuracy of cell detection and classification by simultaneously learning from both tasks

Self-supervised deep learning for highly efficient spatial immunophenotypingResearch in context

Summary: Background: Efficient biomarker discovery and clinical translation depend on the fast and accurate analytical output from crucial technologies such as multiplex imaging. However, reliable cell classification often requires extensive annotations. Label-efficient strategies are urgently needed to reveal diverse cell distribution and spatial interactions in large-scale multiplex datasets. Methods: This study proposed Self-supervised Learning for Antigen Detection (SANDI) for accurate cell phenotyping while mitigating the annotation burden. The model first learns intrinsic pairwise similarities in unlabelled cell images, followed by a classification step to map learnt features to cell labels using a small set of annotated references. We acquired four multiplex immunohistochemistry datasets and one imaging mass cytometry dataset, comprising 2825 to 15,258 single-cell images to train and test the model. Findings: With 1% annotations (18–114 cells), SANDI achieved weighted F1-scores ranging from 0.82 to 0.98 across the five datasets, which was comparable to the fully supervised classifier trained on 1828–11,459 annotated cells (−0.002 to −0.053 of averaged weighted F1-score, Wilcoxon rank-sum test, P = 0.31). Leveraging the immune checkpoint markers stained in ovarian cancer slides, SANDI-based cell identification reveals spatial expulsion between PD1-expressing T helper cells and T regulatory cells, suggesting an interplay between PD1 expression and T regulatory cell-mediated immunosuppression. Interpretation: By striking a fine balance between minimal expert guidance and the power of deep learning to learn similarity within abundant data, SANDI presents new opportunities for efficient, large-scale learning for histology multiplex imaging data. Funding: This study was funded by the Royal Marsden/ICR National Institute of Health Research Biomedical Research Centre