Development, evaluation, and validation of machine learning models for COVID-19 detection based on routine blood tests

The rRT-PCR test, the current gold standard for the detection of coronavirus disease (COVID-19), presents with known shortcomings, such as long turnaround time, potential shortage of reagents, false-negative rates around 15-20%, and expensive equipment. The hematochemical values of routine blood exams could represent a faster and less expensive alternative. Three different training data set of hematochemical values from 1,624 patients (52% COVID-19 positive), admitted at San Raphael Hospital (OSR) from February to May 2020, were used for developing machine learning (ML) models: the complete OSR dataset (72 features: complete blood count (CBC), biochemical, coagulation, hemogasanalysis and CO-Oxymetry values, age, sex and specific symptoms at triage) and two sub-datasets (COVID-specific and CBC dataset, 32 and 21 features respectively). 58 cases (50% COVID-19 positive) from another hospital, and 54 negative patients collected in 2018 at OSR, were used for internal-external and external validation. We developed five ML models: for the complete OSR dataset, the area under the receiver operating characteristic curve (AUC) for the algorithms ranged from 0.83 to 0.90; for the COVID-specific dataset from 0.83 to 0.87; and for the CBC dataset from 0.74 to 0.86. The validations also achieved good results: respectively, AUC from 0.75 to 0.78; and specificity from 0.92 to 0.96. ML can be applied to blood tests as both an adjunct and alternative method to rRT-PCR for the fast and cost-effective identification of COVID-19-positive patients. This is especially useful in developing countries, or in countries facing an increase in contagions

What can whiskers tell us about mammalian evolution, behaviour, and ecology?

Most mammals have whiskers; however, nearly everything we know about whiskers derives from just a handful of species, including laboratory rats Rattus norvegicus and mice Mus musculus, as well as some species of pinniped and marsupial. We explore the extent to which the knowledge of the whisker system from a handful of species applies to mammals generally. This will help us understand whisker evolution and function, in order to gain more insights into mammalian behaviour and ecology. This review is structured around Tinbergen’s four questions, since this method is an established, comprehensive, and logical approach to studying behaviour. We ask: how do whiskers work, develop, and evolve? And what are they for? While whiskers are all slender, curved, tapered, keratinised hairs that transmit vibrotactile information, we show that there are marked differences between species with respect to whisker arrangement, numbers, length, musculature, development, and growth cycles. The conservation of form and a common muscle architecture in mammals suggests that early mammals had whiskers. Whiskers may have been functional even in therapsids. However, certain extant mammalian species are equipped with especially long and sensitive whiskers, in particular nocturnal, arboreal species, and aquatic species, which live in complex environments and hunt moving prey. Knowledge of whiskers and whisker use can guide us in developing conservation protocols and designing enriched enclosures for captive mammals. We suggest that further comparative studies, embracing a wider variety of mammalian species, are required before one can make large-scale predictions relating to evolution and function of whiskers. More research is needed to develop robust techniques to enhance the welfare and conservation of mammals

As if sand were stone : New concepts and metrics to probe the ground on which to build trustable AI

Background: We focus on the importance of interpreting the quality of the labeling used as the input of predictive models to understand the reliability of their output in support of human decision-making, especially in critical domains, such as medicine. Methods: Accordingly, we propose a framework distinguishing the reference labeling (or Gold Standard) from the set of annotations from which it is usually derived (the Diamond Standard). We define a set of quality dimensions and related metrics: representativeness (are the available data representative of its reference population?); reliability (do the raters agree with each other in their ratings?); and accuracy (are the raters' annotations a true representation?). The metrics for these dimensions are, respectively, the degree of correspondence, \u3a8, the degree of weighted concordance \u3f1, and the degree of fineness, \u3a6. We apply and evaluate these metrics in a diagnostic user study involving 13 radiologists. Results: We evaluate \u3a8 against hypothesis-testing techniques, highlighting that our metrics can better evaluate distribution similarity in high-dimensional spaces. We discuss how \u3a8 could be used to assess the reliability of new predictions or for train-test selection. We report the value of \u3f1 for our case study and compare it with traditional reliability metrics, highlighting both their theoretical properties and the reasons that they differ. Then, we report the degree of fineness as an estimate of the accuracy of the collected annotations and discuss the relationship between this latter degree and the degree of weighted concordance, which we find to be moderately but significantly correlated. Finally, we discuss the implications of the proposed dimensions and metrics with respect to the context of Explainable Artificial Intelligence (XAI). Conclusion: We propose different dimensions and related metrics to assess the quality of the datasets used to build predictive models and Medical Artificial Intelligence (MAI). We argue that the proposed metrics are feasible for application in real-world settings for the continuous development of trustable and interpretable MAI systems

All you need is higher accuracy? On the quest for minimum acceptable accuracy for medical artificial intelligence

In this paper we will discuss the concept of the minimum level of accuracy an Artificial Intelligence system must exhibit in medical settings to be fit to its intended use and improve the daily practice of its intended users, the medical doctors. We will consider simple binary classification tasks in both diagnostic and prognostic ambit (like to discriminate between normal/abnormal case, and improvement/no improvement prospects). We will make the point that the common ways to determine this minimum acceptable accuracy are fraught with many conceptual and practical troubles. We will report about a small user study conducted to elicit the discriminative requirements from a sample of medical doctors, stratified both in general practitioners and specialists. Finally, we will present a simple nomogram by which to determine the minimum accuracy of a technological aid, once the human average performance and the desired level of accuracy are known. The nomogram is to be intended as a provocative simple tool to recognize that the technological tool is less important than a sound protocol in which to use it, responsibly and paying due attention to the role of the human decision makers

Identification of spinacine as the principal reaction product of \u3b3-casein with formaldehyde in cheese

Formaldehyde is added to milk in the production of grana cheese as an antimicrobial agent. In order to study the fate of the formaldehyde, a grana cheese preparation was made using 14C-labelled formaldehyde. The 14C-activity in the cheese was found to be mainly associated with the caseins, but it was not uniformly distributed among the different fractions (\u3b1(s), \u3b2- and \u3b3-caseins). \u3b3-Casein, separated by electrophoresis, was the most reactive component showing the highest specific activity. In the \u3b3-casein fractions, 99% of 14C-activity was associated with the basic amino acids. The only radioactive reaction product present in the \u3b3-casein fraction was identified by HPTLC and by an amino acid analyser to be spinacine, a condensation product of formaldehyde and histidine. Using the same method, other unknown radioactive products, of much less relevance, were detected in the total casein hydrolysate

Detection of COVID-19 Infection from Routine Blood Exams with Machine Learning: A Feasibility Study

The COVID-19 pandemia due to the SARS-CoV-2 coronavirus, in its first 4 months since its outbreak, has to date reached more than 200 countries worldwide with more than 2 million confirmed cases (probably a much higher number of infected), and almost 200,000 deaths. Amplification of viral RNA by (real time) reverse transcription polymerase chain reaction (rRT-PCR) is the current gold standard test for confirmation of infection, although it presents known shortcomings: long turnaround times (3-4 hours to generate results), potential shortage of reagents, false-negative rates as large as 15-20%, the need for certified laboratories, expensive equipment and trained personnel. Thus there is a need for alternative, faster, less expensive and more accessible tests. We developed two machine learning classification models using hematochemical values from routine blood exams (namely: white blood cells counts, and the platelets, CRP, AST, ALT, GGT, ALP, LDH plasma levels) drawn from 279 patients who, after being admitted to the San Raffaele Hospital (Milan, Italy) emergency-room with COVID-19 symptoms, were screened with the rRT-PCR test performed on respiratory tract specimens. Of these patients, 177 resulted positive, whereas 102 received a negative response. We have developed two machine learning models, to discriminate between patients who are either positive or negative to the SARS-CoV-2: their accuracy ranges between 82% and 86%, and sensitivity between 92% e 95%, so comparably well with respect to the gold standard. We also developed an interpretable Decision Tree model as a simple decision aid for clinician interpreting blood tests (even off-line) for COVID-19 suspect cases. This study demonstrated the feasibility and clinical soundness of using blood tests analysis and machine learning as an alternative to rRT-PCR for identifying COVID-19 positive patients. This is especially useful in those countries, like developing ones, suffering from shortages of rRT-PCR reagents and specialized laboratories. We made available a Web-based tool for clinical reference and evaluation (This tool is available at https://covid19-blood-ml.herokuapp.com/)

Evidence of significant difference in key covid-19 biomarkers during the italian lockdown strategy. A retrospective study on patients admitted to a hospital emergency department in northern italy

Summary. Background. The Lombardy region, Italy, has been severely affected by COVID-19. During the epidemic peak, in March 2020, patients needing intensive care unit treatments were approximately 10% of those infected. This fraction decreased to approximately 2% in the second part of April, and to 0.4% at the beginning of July. COVID-19 is characterized by several biochemical abnormalities whose discrepancy from normal values was associated to the severity of the disease. The aim of this retrospective study was to compare the biochemical patterns of patients during and after the pandemic peak in order to verify whether later patients were experiencing a milder COVID-19 course, as anecdotally observed by several clinicians of the same Hospital. Material and Methods. The laboratory findings of two equivalent groups of 84 patients each, admitted at the emergency department of the San Raffaele Hospital (Milan, Italy), during March and April respectively, were analyzed and compared. Results. White blood cell, platelets, lymphocytes and lactate dehydrogenase showed a statistically significant improvement (i.e. closer or within the normal clinical range) in the April group compared to March. Creatinine, C-reactive protein, Calcium and liver enzymes, were also pointing in that direction, although the differences were not significant. Discussion. The laboratory findings analyzed in this study were consistent with a milder COVID-19 course in the April group. After excluding several hypotheses, we concluded that our observation was likely the consequence of the lockdown strategy enforcement, which, by imposing social distancing and the use of respiratory protective devices, reduced viral loads upon infection. (www.actabiomedica.it)