11 research outputs found

    Federated Benchmarking of Medical Artificial Intelligence With MedPerf

    Get PDF
    Medical artificial intelligence (AI) has tremendous potential to advance healthcare by supporting and contributing to the evidence-based practice of medicine, personalizing patient treatment, reducing costs, and improving both healthcare provider and patient experience. Unlocking this potential requires systematic, quantitative evaluation of the performance of medical AI models on large-scale, heterogeneous data capturing diverse patient populations. Here, to meet this need, we introduce MedPerf, an open platform for benchmarking AI models in the medical domain. MedPerf focuses on enabling federated evaluation of AI models, by securely distributing them to different facilities, such as healthcare organizations. This process of bringing the model to the data empowers each facility to assess and verify the performance of AI models in an efficient and human-supervised process, while prioritizing privacy. We describe the current challenges healthcare and AI communities face, the need for an open platform, the design philosophy of MedPerf, its current implementation status and real-world deployment, our roadmap and, importantly, the use of MedPerf with multiple international institutions within cloud-based technology and on-premises scenarios. Finally, we welcome new contributions by researchers and organizations to further strengthen MedPerf as an open benchmarking platform

    Federated learning enables big data for rare cancer boundary detection.

    Get PDF
    Although machine learning (ML) has shown promise across disciplines, out-of-sample generalizability is concerning. This is currently addressed by sharing multi-site data, but such centralization is challenging/infeasible to scale due to various limitations. Federated ML (FL) provides an alternative paradigm for accurate and generalizable ML, by only sharing numerical model updates. Here we present the largest FL study to-date, involving data from 71 sites across 6 continents, to generate an automatic tumor boundary detector for the rare disease of glioblastoma, reporting the largest such dataset in the literature (n = 6, 314). We demonstrate a 33% delineation improvement for the surgically targetable tumor, and 23% for the complete tumor extent, over a publicly trained model. We anticipate our study to: 1) enable more healthcare studies informed by large diverse data, ensuring meaningful results for rare diseases and underrepresented populations, 2) facilitate further analyses for glioblastoma by releasing our consensus model, and 3) demonstrate the FL effectiveness at such scale and task-complexity as a paradigm shift for multi-site collaborations, alleviating the need for data-sharing

    Author Correction: Federated learning enables big data for rare cancer boundary detection.

    Get PDF
    10.1038/s41467-023-36188-7NATURE COMMUNICATIONS14

    Federated Learning Enables Big Data for Rare Cancer Boundary Detection

    Get PDF
    Although machine learning (ML) has shown promise across disciplines, out-of-sample generalizability is concerning. This is currently addressed by sharing multi-site data, but such centralization is challenging/infeasible to scale due to various limitations. Federated ML (FL) provides an alternative paradigm for accurate and generalizable ML, by only sharing numerical model updates. Here we present the largest FL study to-date, involving data from 71 sites across 6 continents, to generate an automatic tumor boundary detector for the rare disease of glioblastoma, reporting the largest such dataset in the literature (n = 6, 314). We demonstrate a 33% delineation improvement for the surgically targetable tumor, and 23% for the complete tumor extent, over a publicly trained model. We anticipate our study to: 1) enable more healthcare studies informed by large diverse data, ensuring meaningful results for rare diseases and underrepresented populations, 2) facilitate further analyses for glioblastoma by releasing our consensus model, and 3) demonstrate the FL effectiveness at such scale and task-complexity as a paradigm shift for multi-site collaborations, alleviating the need for data-sharing

    Diagnostic and Prognostic Markers for Pancreatitis and Pancreatic Ductal Adenocarcinoma

    No full text
    Diagnostic markers are desperately needed for the early detection of pancreatic ductal adenocarcinoma (PDA). We describe sets of markers expressed in temporal order in mouse models during pancreatitis, PDA initiation and progression. Cell type specificity and the differential expression of PDA markers were identified by screening single cell (sc) RNAseq from tumor samples of a mouse model for PDA (KIC) at early and late stages of PDA progression compared to that of a normal pancreas. Candidate genes were identified from three sources: (1) an unsupervised screening of the genes preferentially expressed in mouse PDA tumors; (2) signaling pathways that drive PDA, including the Ras pathway, calcium signaling, and known cancer genes, or genes encoding proteins that were identified by differential mass spectrometry (MS) of mouse tumors and conditioned media from human cancer cell lines; and (3) genes whose expression is associated with poor or better prognoses (PAAD, oncolnc.org). The developmental progression of PDA was detected in the temporal order of gene expression in the cancer cells of the KIC mice. The earliest diagnostic markers were expressed in epithelial cancer cells in early-stage, but not late-stage, PDA tumors. Other early markers were expressed in the epithelium of both early- and late-state PDA tumors. Markers that were expressed somewhat later were first elevated in the epithelial cancer cells of the late-stage tumors, then in both epithelial and mesenchymal cells, or only in mesenchymal cells. Stromal markers were differentially expressed in early- and/or late-stage PDA neoplasia in fibroblast and hematopoietic cells (lymphocytes and/or macrophages) or broadly expressed in cancer and many stromal cell types. Pancreatitis is a risk factor for PDA in humans. Mouse models of pancreatitis, including caerulein treatment and the acinar-specific homozygous deletion of differentiation transcription factors (dTFs), were screened for the early expression of all PDA markers identified in the KIC neoplasia. Prognostic markers associated with a more rapid decline were identified and showed differential and cell-type-specific expression in PDA, predominately in late-stage epithelial and/or mesenchymal cancer cells. Select markers were validated by immunohistochemistry in mouse and human samples of a normal pancreas and those with early- and late-stage PDA. In total, we present 2165 individual diagnostic and prognostic markers for disease progression to be tested in humans from pancreatitis to late-stage PDA
    corecore