Fully-automated patient-level malaria assessment on field-prepared thin blood film microscopy images, including Supplementary Information

Malaria is a life-threatening disease affecting millions. Microscopy-based assessment of thin blood films is a standard method to (i) determine malaria species and (ii) quantitate high-parasitemia infections. Full automation of malaria microscopy by machine learning (ML) is a challenging task because field-prepared slides vary widely in quality and presentation, and artifacts often heavily outnumber relatively rare parasites. In this work, we describe a complete, fully-automated framework for thin film malaria analysis that applies ML methods, including convolutional neural nets (CNNs), trained on a large and diverse dataset of field-prepared thin blood films. Quantitation and species identification results are close to sufficiently accurate for the concrete needs of drug resistance monitoring and clinical use-cases on field-prepared samples. We focus our methods and our performance metrics on the field use-case requirements. We discuss key issues and important metrics for the application of ML methods to malaria microscopy.Comment: 16 pages, 13 figure

Performance of a fully‐automated system on a WHO malaria microscopy evaluation slide set

Background: Manual microscopy remains a widely-used tool for malaria diagnosis and clinical studies, but it has inconsistent quality in the field due to variability in training and field practices. Automated diagnostic systems based on machine learning hold promise to improve quality and reproducibility of field microscopy. The World Health Organization (WHO) has designed a 55-slide set (WHO 55) for their External Competence Assessment of Malaria Microscopists (ECAMM) programme, which can also serve as a valuable benchmark for automated systems. The performance of a fully-automated malaria diagnostic system, EasyScan GO, on a WHO 55 slide set was evaluated. Methods: The WHO 55 slide set is designed to evaluate microscopist competence in three areas of malaria diagnosis using Giemsa-stained blood films, focused on crucial field needs: malaria parasite detection, malaria parasite species identification (ID), and malaria parasite quantitation. The EasyScan GO is a fully-automated system that combines scanning of Giemsa-stained blood films with assessment algorithms to deliver malaria diagnoses. This system was tested on a WHO 55 slide set. Results: The EasyScan GO achieved 94.3 % detection accuracy, 82.9 % species ID accuracy, and 50 % quantitation accuracy, corresponding to WHO microscopy competence Levels 1, 2, and 1, respectively. This is, to our knowledge, the best performance of a fully-automated system on a WHO 55 set. Conclusions: EasyScan GO’s expert ratings in detection and quantitation on the WHO 55 slide set point towards its potential value in drug efficacy use-cases, as well as in some case management situations with less stringent species ID needs. Improved runtime may enable use in general case management settings

The human body at cellular resolution: the NIH Human Biomolecular Atlas Program

Abstract: Transformative technologies are enabling the construction of three-dimensional maps of tissues with unprecedented spatial and molecular resolution. Over the next seven years, the NIH Common Fund Human Biomolecular Atlas Program (HuBMAP) intends to develop a widely accessible framework for comprehensively mapping the human body at single-cell resolution by supporting technology development, data acquisition, and detailed spatial mapping. HuBMAP will integrate its efforts with other funding agencies, programs, consortia, and the biomedical research community at large towards the shared vision of a comprehensive, accessible three-dimensional molecular and cellular atlas of the human body, in health and under various disease conditions

Metrics to guide development of machine learning algorithms for malaria diagnosis

Automated malaria diagnosis is a difficult but high-value target for machine learning (ML), and effective algorithms could save many thousands of children's lives. However, current ML efforts largely neglect crucial use case constraints and are thus not clinically useful. Two factors in particular are crucial to developing algorithms translatable to clinical field settings: (i) Clear understanding of the clinical needs that ML solutions must accommodate; and (ii) task-relevant metrics for guiding and evaluating ML models. Neglect of these factors has seriously hampered past ML work on malaria, because the resulting algorithms do not align with clinical needs. In this paper we address these two issues in the context of automated malaria diagnosis via microscopy on Giemsa-stained blood films. First, we describe why domain expertise is crucial to effectively apply ML to malaria, and list technical documents and other resources that provide this domain knowledge. Second, we detail performance metrics tailored to the clinical requirements of malaria diagnosis, to guide development of ML models and evaluate model performance through the lens of clinical needs (versus a generic ML lens). We highlight the importance of a patient-level perspective, interpatient variability, false positive rates, limit of detection, and different types of error. We also discuss reasons why ROC curves, AUC, and F1, as commonly used in ML work, are poorly suited to this context. These findings also apply to other diseases involving parasite loads, including neglected tropical diseases (NTDs) such as schistosomiasis.Comment: 17 pages, 5 figure

Evaluation of an automated microscope using machine learning for the detection of malaria in travelers returned to the UK

Light microscopy remains a standard method for detection of malaria parasites in clinical cases but training to expert level requires considerable time. Moreover, excessive workflow causes fatigue and can impact performance. An automated microscopy tool could aid in clinics with limited access to highly skilled microscopists, where case numbers are excessive, or in multi-site studies where consistency is essential. The EasyScan GO is an automated scanning microscope combined with machine learning software designed to detect malaria parasites in field-prepared Giemsa-stained blood films. This study evaluates the ability of the EasyScan GO to detect, quantify and identify the species of parasite present in blood films compared with expert light microscopy.Travelers returning to the UK and testing positive for malaria were screened for eligibility and enrolled. Blood samples from enrolled participants were used to make Giemsa-stained smears assessed by expert light microscopy and the EasyScan GO to determine parasite density and species. Blood samples were also assessed by PCR to confirm parasite density and species present and resolve discrepancy between manual microscopy and the EasyScan GO.Results: When compared to light microscopy, the EasyScan GO exhibited a sensitivity of 88% (95% CI: 801-934%) and a specificity of 89% (95% CI: 87-91%). Of the 99 samples labelled positive by both, manual microscopy identified 873 as Plasmodium falciparum (Pf) and 12 as non-Pf. The EasyScan GO correctly reported Pf for 862 of the 873 Pf samples and non-Pf for 11 of 12 non-Pf samples. However, it failed to distinguish between non-Pf species, reporting all as P. vivax. The EasyScan GO calculated parasite densities were within +/-25% of light microscopy densities for 33% of samples between 200 and 2000 p/µL, falling short of WHO level 1 (expert) manual microscopy competency (50% of samples should be within +/-25% of the true parasitaemia).Discussion: This study shows that the EasyScan GO can be proficient in detecting malaria parasites in Giemsa-stained blood films relative to expert light microscopy and accurately distinguish between Pf and non-Pf species. Performance at low parasite densities, distinguishing between non-Pf species and accurate quantitation of parasitaemias require further development and evaluation