Combining Multiplexed Ion Beam Imaging (MIBI) with Convolutional Neural Networks to accurately segment cells in human tissue

Background: Multiplexed imaging is a rapidly growing field that promises to substantially increase the number of proteins that can be imaged simultaneously. We have developed Multiplexed Ion Beam Imaging by Time of Flight (MIBI-TOF), which uses elemental reporters conjugated to primary antibodies that are then quantified using a time of flight mass-spectrometer. This technique allows for more than 40 distinct proteins to visualized at once in the same clinical samples. This has already yielded significant insights into the interactions and relationships between the many different immune cell populations present in the tumor microenvironment. However, one of the remaining challenges in analyzing such data is accurately determining target protein expression values for each cell in the image. This requires the precise delineation of boundaries between cells that are often tightly packed next to one another. Current methods to address this challenge largely rely on DNA intensity to make these splits, and are thus mostly limited to nuclear segmentation. Methods: We have developed a novel convolutional neural network to perform whole-cell segmentation from multiplexed imaging data. Rather than relying only on DNA signal, we use a panel of morphological markers. Our method integrates the information from these distinct proteins, allowing it to segment large cancer cells, small lymphocytes, and normal epithelium at the same time without requiring fine-tuning or manual adjustment. Results: By combining our novel imaging platform with new computational tools, we are able to achieve extremely accurate segmentation of whole cells in tissue. Our approach compares favorably with many of the currently used tools for segmentation. We show that our improvements in accuracy come both from our novel imaging approach as well as algorithmic advances. We perform significantly better than traditional machine learning algorithms trained on the same dataset. Additionally, we show that our algorithm can be trained to identify cells across a range of cancer histologies and disease grades. Conclusions: We have developed a robust and accurate approach to whole-cell segmentation in human tissues. We show the superiority over this method over current state of the art algorithms. The accurate segmentation generated by our approach will enable the analysis of complex tissue architectures with highly overlapping cell types, and will help to advance our understanding of the interactions between cell types in the diseased state

Whole-cell segmentation of tissue images with human-level performance using large-scale data annotation and deep learning

Understanding the spatial organization of tissues is of critical importance for both basic and translational research. While recent advances in tissue imaging are opening an exciting new window into the biology of human tissues, interpreting the data that they create is a significant computational challenge. Cell segmentation, the task of uniquely identifying each cell in an image, remains a substantial barrier for tissue imaging, as existing approaches are inaccurate or require a substantial amount of manual curation to yield useful results. Here, we addressed the problem of cell segmentation in tissue imaging data through large-scale data annotation and deep learning. We constructed TissueNet, an image dataset containing >1 million paired whole-cell and nuclear annotations for tissue images from nine organs and six imaging platforms. We created Mesmer, a deep learning-enabled segmentation algorithm trained on TissueNet that performs nuclear and whole-cell segmentation in tissue imaging data. We demonstrated that Mesmer has better speed and accuracy than previous methods, generalizes to the full diversity of tissue types and imaging platforms in TissueNet, and achieves human-level performance for whole-cell segmentation. Mesmer enabled the automated extraction of key cellular features, such as subcellular localization of protein signal, which was challenging with previous approaches. We further showed that Mesmer could be adapted to harness cell lineage information present in highly multiplexed datasets. We used this enhanced version to quantify cell morphology changes during human gestation. All underlying code and models are released with permissive licenses as a community resource

DSP107 combines inhibition of CD47/SIRPα axis with activation of 4-1BB to trigger anticancer immunity

BACKGROUND: Treatment of Diffuse Large B Cell Lymphoma (DLBCL) patients with rituximab and the CHOP treatment regimen is associated with frequent intrinsic and acquired resistance. However, treatment with a CD47 monoclonal antibody in combination with rituximab yielded high objective response rates in patients with relapsed/refractory DLBCL in a phase I trial. Here, we report on a new bispecific and fully human fusion protein comprising the extracellular domains of SIRPα and 4-1BBL, termed DSP107, for the treatment of DLBCL. DSP107 blocks the CD47:SIRPα ‘don’t eat me’ signaling axis on phagocytes and promotes innate anticancer immunity. At the same time, CD47-specific binding of DSP107 enables activation of the costimulatory receptor 4-1BB on activated T cells, thereby, augmenting anticancer T cell immunity. METHODS: Using macrophages, polymorphonuclear neutrophils (PMNs), and T cells of healthy donors and DLBCL patients, DSP107-mediated reactivation of immune cells against B cell lymphoma cell lines and primary patient-derived blasts was studied with phagocytosis assays, T cell activation and cytotoxicity assays. DSP107 anticancer activity was further evaluated in a DLBCL xenograft mouse model and safety was evaluated in cynomolgus monkey. RESULTS: Treatment with DSP107 alone or in combination with rituximab significantly increased macrophage- and PMN-mediated phagocytosis and trogocytosis, respectively, of DLBCL cell lines and primary patient-derived blasts. Further, prolonged treatment of in vitro macrophage/cancer cell co-cultures with DSP107 and rituximab decreased cancer cell number by up to 85%. DSP107 treatment activated 4-1BB-mediated costimulatory signaling by HT1080.4-1BB reporter cells, which was strictly dependent on the SIRPα-mediated binding of DSP107 to CD47. In mixed cultures with CD47-expressing cancer cells, DSP107 augmented T cell cytotoxicity in vitro in an effector-to-target ratio-dependent manner. In mice with established SUDHL6 xenografts, the treatment with human PBMCs and DSP107 strongly reduced tumor size compared to treatment with PBMCs alone and increased the number of tumor-infiltrated T cells. Finally, DSP107 had an excellent safety profile in cynomolgus monkeys. CONCLUSIONS: DSP107 effectively (re)activated innate and adaptive anticancer immune responses and may be of therapeutic use alone and in combination with rituximab for the treatment of DLBCL patients. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s13046-022-02256-x

The Society for Immunotherapy of Cancer statement on best practices for multiplex immunohistochemistry (IHC) and immunofluorescence (IF) staining and validation.

OBJECTIVES: The interaction between the immune system and tumor cells is an important feature for the prognosis and treatment of cancer. Multiplex immunohistochemistry (mIHC) and multiplex immunofluorescence (mIF) analyses are emerging technologies that can be used to help quantify immune cell subsets, their functional state, and their spatial arrangement within the tumor microenvironment. METHODS: The Society for Immunotherapy of Cancer (SITC) convened a task force of pathologists and laboratory leaders from academic centers as well as experts from pharmaceutical and diagnostic companies to develop best practice guidelines for the optimization and validation of mIHC/mIF assays across platforms. RESULTS: Representative outputs and the advantages and disadvantages of mIHC/mIF approaches, such as multiplexed chromogenic IHC, multiplexed immunohistochemical consecutive staining on single slide, mIF (including multispectral approaches), tissue-based mass spectrometry, and digital spatial profiling are discussed. CONCLUSIONS: mIHC/mIF technologies are becoming standard tools for biomarker studies and are likely to enter routine clinical practice in the near future. Careful assay optimization and validation will help ensure outputs are robust and comparable across laboratories as well as potentially across mIHC/mIF platforms. Quantitative image analysis of mIHC/mIF output and data management considerations will be addressed in a complementary manuscript from this task force