Fully-automated patient-level malaria assessment on field-prepared thin blood film microscopy images, including Supplementary Information

Malaria is a life-threatening disease affecting millions. Microscopy-based assessment of thin blood films is a standard method to (i) determine malaria species and (ii) quantitate high-parasitemia infections. Full automation of malaria microscopy by machine learning (ML) is a challenging task because field-prepared slides vary widely in quality and presentation, and artifacts often heavily outnumber relatively rare parasites. In this work, we describe a complete, fully-automated framework for thin film malaria analysis that applies ML methods, including convolutional neural nets (CNNs), trained on a large and diverse dataset of field-prepared thin blood films. Quantitation and species identification results are close to sufficiently accurate for the concrete needs of drug resistance monitoring and clinical use-cases on field-prepared samples. We focus our methods and our performance metrics on the field use-case requirements. We discuss key issues and important metrics for the application of ML methods to malaria microscopy.Comment: 16 pages, 13 figure

Metrics to guide development of machine learning algorithms for malaria diagnosis

Automated malaria diagnosis is a difficult but high-value target for machine learning (ML), and effective algorithms could save many thousands of children's lives. However, current ML efforts largely neglect crucial use case constraints and are thus not clinically useful. Two factors in particular are crucial to developing algorithms translatable to clinical field settings: (i) Clear understanding of the clinical needs that ML solutions must accommodate; and (ii) task-relevant metrics for guiding and evaluating ML models. Neglect of these factors has seriously hampered past ML work on malaria, because the resulting algorithms do not align with clinical needs. In this paper we address these two issues in the context of automated malaria diagnosis via microscopy on Giemsa-stained blood films. First, we describe why domain expertise is crucial to effectively apply ML to malaria, and list technical documents and other resources that provide this domain knowledge. Second, we detail performance metrics tailored to the clinical requirements of malaria diagnosis, to guide development of ML models and evaluate model performance through the lens of clinical needs (versus a generic ML lens). We highlight the importance of a patient-level perspective, interpatient variability, false positive rates, limit of detection, and different types of error. We also discuss reasons why ROC curves, AUC, and F1, as commonly used in ML work, are poorly suited to this context. These findings also apply to other diseases involving parasite loads, including neglected tropical diseases (NTDs) such as schistosomiasis.Comment: 17 pages, 5 figure

Evaluation of an automated microscope using machine learning for the detection of malaria in travelers returned to the UK

Light microscopy remains a standard method for detection of malaria parasites in clinical cases but training to expert level requires considerable time. Moreover, excessive workflow causes fatigue and can impact performance. An automated microscopy tool could aid in clinics with limited access to highly skilled microscopists, where case numbers are excessive, or in multi-site studies where consistency is essential. The EasyScan GO is an automated scanning microscope combined with machine learning software designed to detect malaria parasites in field-prepared Giemsa-stained blood films. This study evaluates the ability of the EasyScan GO to detect, quantify and identify the species of parasite present in blood films compared with expert light microscopy.Travelers returning to the UK and testing positive for malaria were screened for eligibility and enrolled. Blood samples from enrolled participants were used to make Giemsa-stained smears assessed by expert light microscopy and the EasyScan GO to determine parasite density and species. Blood samples were also assessed by PCR to confirm parasite density and species present and resolve discrepancy between manual microscopy and the EasyScan GO.Results: When compared to light microscopy, the EasyScan GO exhibited a sensitivity of 88% (95% CI: 801-934%) and a specificity of 89% (95% CI: 87-91%). Of the 99 samples labelled positive by both, manual microscopy identified 873 as Plasmodium falciparum (Pf) and 12 as non-Pf. The EasyScan GO correctly reported Pf for 862 of the 873 Pf samples and non-Pf for 11 of 12 non-Pf samples. However, it failed to distinguish between non-Pf species, reporting all as P. vivax. The EasyScan GO calculated parasite densities were within +/-25% of light microscopy densities for 33% of samples between 200 and 2000 p/µL, falling short of WHO level 1 (expert) manual microscopy competency (50% of samples should be within +/-25% of the true parasitaemia).Discussion: This study shows that the EasyScan GO can be proficient in detecting malaria parasites in Giemsa-stained blood films relative to expert light microscopy and accurately distinguish between Pf and non-Pf species. Performance at low parasite densities, distinguishing between non-Pf species and accurate quantitation of parasitaemias require further development and evaluation

Automated microscopy for routine malaria diagnosis: a field comparison on Giemsa-stained blood films in Peru

Abstract Background Microscopic examination of Giemsa-stained blood films remains a major form of diagnosis in malaria case management, and is a reference standard for research. However, as with other visualization-based diagnoses, accuracy depends on individual technician performance, making standardization difficult and reliability poor. Automated image recognition based on machine-learning, utilizing convolutional neural networks, offers potential to overcome these drawbacks. A prototype digital microscope device employing an algorithm based on machine-learning, the Autoscope, was assessed for its potential in malaria microscopy. Autoscope was tested in the Iquitos region of Peru in 2016 at two peripheral health facilities, with routine microscopy and PCR as reference standards. The main outcome measures include sensitivity and specificity of diagnosis of malaria from Giemsa-stained blood films, using PCR as reference. Methods A cross-sectional, observational trial was conducted at two peripheral primary health facilities in Peru. 700 participants were enrolled with the criteria: (1) age between 5 and 75 years, (2) history of fever in the last 3 days or elevated temperature on admission, (3) informed consent. The main outcome measures included sensitivity and specificity of diagnosis of malaria from Giemsa-stained blood films, using PCR as reference. Results At the San Juan clinic, sensitivity of Autoscope for diagnosing malaria was 72% (95% CI 64–80%), and specificity was 85% (95% CI 79–90%). Microscopy performance was similar to Autoscope, with sensitivity 68% (95% CI 59–76%) and specificity 100% (95% CI 98–100%). At San Juan, 85% of prepared slides had a minimum of 600 WBCs imaged, thus meeting Autoscope’s design assumptions. At the second clinic, Santa Clara, the sensitivity of Autoscope was 52% (95% CI 44–60%) and specificity was 70% (95% CI 64–76%). Microscopy performance at Santa Clara was 42% (95% CI 34–51) and specificity was 97% (95% CI 94–99). Only 39% of slides from Santa Clara met Autoscope’s design assumptions regarding WBCs imaged. Conclusions Autoscope’s diagnostic performance was on par with routine microscopy when slides had adequate blood volume to meet its design assumptions, as represented by results from the San Juan clinic. Autoscope’s diagnostic performance was poorer than routine microscopy on slides from the Santa Clara clinic, which generated slides with lower blood volumes. Results of the study reflect both the potential for artificial intelligence to perform tasks currently conducted by highly-trained experts, and the challenges of replicating the adaptiveness of human thought processes