Machine learning algorithms for systematic review: reducing workload in a preclinical review of animal studies and reducing human screening error

BACKGROUND: Here, we outline a method of applying existing machine learning (ML) approaches to aid citation screening in an on-going broad and shallow systematic review of preclinical animal studies. The aim is to achieve a high-performing algorithm comparable to human screening that can reduce human resources required for carrying out this step of a systematic review. METHODS: We applied ML approaches to a broad systematic review of animal models of depression at the citation screening stage. We tested two independently developed ML approaches which used different classification models and feature sets. We recorded the performance of the ML approaches on an unseen validation set of papers using sensitivity, specificity and accuracy. We aimed to achieve 95% sensitivity and to maximise specificity. The classification model providing the most accurate predictions was applied to the remaining unseen records in the dataset and will be used in the next stage of the preclinical biomedical sciences systematic review. We used a cross-validation technique to assign ML inclusion likelihood scores to the human screened records, to identify potential errors made during the human screening process (error analysis). RESULTS: ML approaches reached 98.7% sensitivity based on learning from a training set of 5749 records, with an inclusion prevalence of 13.2%. The highest level of specificity reached was 86%. Performance was assessed on an independent validation dataset. Human errors in the training and validation sets were successfully identified using the assigned inclusion likelihood from the ML model to highlight discrepancies. Training the ML algorithm on the corrected dataset improved the specificity of the algorithm without compromising sensitivity. Error analysis correction leads to a 3% improvement in sensitivity and specificity, which increases precision and accuracy of the ML algorithm. CONCLUSIONS: This work has confirmed the performance and application of ML algorithms for screening in systematic reviews of preclinical animal studies. It has highlighted the novel use of ML algorithms to identify human error. This needs to be confirmed in other reviews with different inclusion prevalence levels, but represents a promising approach to integrating human decisions and automation in systematic review methodology

Detection of Prion Protein Particles in Blood Plasma of Scrapie Infected Sheep

Prion diseases are transmissible neurodegenerative diseases affecting humans and animals. The agent of the disease is the prion consisting mainly, if not solely, of a misfolded and aggregated isoform of the host-encoded prion protein (PrP). Transmission of prions can occur naturally but also accidentally, e.g. by blood transfusion, which has raised serious concerns about blood product safety and emphasized the need for a reliable diagnostic test. In this report we present a method based on surface-FIDA (fluorescence intensity distribution analysis), that exploits the high state of molecular aggregation of PrP as an unequivocal diagnostic marker of the disease, and show that it can detect infection in blood. To prepare PrP aggregates from blood plasma we introduced a detergent and lipase treatment to separate PrP from blood lipophilic components. Prion protein aggregates were subsequently precipitated by phosphotungstic acid, immobilized on a glass surface by covalently bound capture antibodies, and finally labeled with fluorescent antibody probes. Individual PrP aggregates were visualized by laser scanning microscopy where signal intensity was proportional to aggregate size. After signal processing to remove the background from low fluorescence particles, fluorescence intensities of all remaining PrP particles were summed. We detected PrP aggregates in plasma samples from six out of ten scrapie-positive sheep with no false positives from uninfected sheep. Applying simultaneous intensity and size discrimination, ten out of ten samples from scrapie sheep could be differentiated from uninfected sheep. The implications for ante mortem diagnosis of prion diseases are discussed

Eleven strategies for making reproducible research and open science training the norm at research institutions

Reproducible research and open science practices have the potential to accelerate scientific progress by allowing others to reuse research outputs, and by promoting rigorous research that is more likely to yield trustworthy results. However, these practices are uncommon in many fields, so there is a clear need for training that helps and encourages researchers to integrate reproducible research and open science practices into their daily work. Here, we outline eleven strategies for making training in these practices the norm at research institutions. The strategies, which emerged from a virtual brainstorming event organized in collaboration with the German Reproducibility Network, are concentrated in three areas: (i) adapting research assessment criteria and program requirements; (ii) training; (iii) building communities. We provide a brief overview of each strategy, offer tips for implementation, and provide links to resources. We also highlight the importance of allocating resources and monitoring impact. Our goal is to encourage researchers - in their roles as scientists, supervisors, mentors, instructors, and members of curriculum, hiring or evaluation committees - to think creatively about the many ways they can promote reproducible research and open science practices in their institutions

Mobile phones represent a pathway for microbial transmission: A scoping review

Background Mobile phones have become an integral part of modern society. As possible breeding grounds for microbial organisms, these constitute a potential global public health risk for microbial transmission. Objective Scoping review of literature examining microbial's presence on mobile phones in both health care (HC) and community settings. Methods A search (PubMed&GoogleScholar) was conducted from January 2005–December 2019 to identify English language studies. Studies were included if samples from mobile phones were tested for bacteria, fungi, and/or viruses; and if the sampling was carried out in any HC setting, and/or within the general community. Any other studies exploring mobile phones that did not identify specific microorganisms were excluded. Results A total of 56 studies were included (from 24 countries). Most studies identified the presence of bacteria (54/56), while 16 studies reported the presence of fungi. One study focused solely on RNA viruses. Staphylococcus aureus, and Coagulase-Negative Staphylococci were the most numerous identified organisms present on mobile phones. These two species and Escherichia coli were present in over a third of studies both in HC and community samples. Methicillin-resistant S. aureus, Acinetobacter sp., and Bacillus sp. were present in over a third of the studies in HC settings. Conclusions While this scoping review of literature regarding microbial identification on mobile phones in HC and community settings did not directly address the issue of SARS-CoV-2 responsible for COVID-19, this work exposes the possible role of mobile phones as a ‘Trojan horse’ contributing to the transmission of microbial infections in epidemics and pandemics

The evidence synthesis and meta‑analysis in R conference (ESMARConf): levelling the playing field of conference accessibility and equitability

Rigorous evidence is vital in all disciplines to ensure efficient, appropriate, and fit-for-purpose decision-making with minimised risk of unintended harm. To date, however, disciplines have been slow to share evidence synthesis frameworks, best practices, and tools amongst one another. Recent progress in collaborative digital and programmatic frameworks, such as the free and Open Source software R, have significantly expanded the opportunities for development of free-to-use, incrementally improvable, community driven tools to support evidence synthesis (e.g. EviAtlas, robvis, PRISMA2020 flow diagrams and metadat). Despite this, evidence synthesis (and meta-analysis) practitioners and methodologists who make use of R remain relatively disconnected from one another. Here, we report on a new virtual conference for evidence synthesis and meta-analysis in the R programming environment (ESMARConf ) that aims to connect these communities. By designing an entirely free and online conference from scratch, we have been able to focus efforts on maximising accessibility and equity—making these core missions for our new community of practice. As a community of practice, ESMARConf builds on the success and groundwork of the broader R community and systematic review coordinating bodies (e.g. Cochrane), but fills an important niche. ESMARConf aims to maximise accessibility and equity of participants across regions, contexts, and social backgrounds, forging a level playing field in a digital, connected, and online future of evidence synthesis. We believe that everyone should have the same access to participation and involvement, and we believe ESMARConf provides a vital opportunity to push for equitability across disciplines, regions, and personal situations.publishedVersio

A minimal metadata set (MNMS) to repurpose nonclinical in vivo data for biomedical research.

Although biomedical research is experiencing a data explosion, the accumulation of vast quantities of data alone does not guarantee a primary objective for science: building upon existing knowledge. Data collected that lack appropriate metadata cannot be fully interrogated or integrated into new research projects, leading to wasted resources and missed opportunities for data repurposing. This issue is particularly acute for research using animals, where concerns regarding data reproducibility and ensuring animal welfare are paramount. Here, to address this problem, we propose a minimal metadata set (MNMS) designed to enable the repurposing of in vivo data. MNMS aligns with an existing validated guideline for reporting in vivo data (ARRIVE 2.0) and contributes to making in vivo data FAIR-compliant. Scenarios where MNMS should be implemented in diverse research environments are presented, highlighting opportunities and challenges for data repurposing at different scales. We conclude with a 'call for action' to key stakeholders in biomedical research to adopt and apply MNMS to accelerate both the advancement of knowledge and the betterment of animal welfare

The evidence synthesis and meta‑analysis in R conference (ESMARConf): levelling the playing field of conference accessibility and equitability

Rigorous evidence is vital in all disciplines to ensure efficient, appropriate, and fit-for-purpose decision-making with minimised risk of unintended harm. To date, however, disciplines have been slow to share evidence synthesis frameworks, best practices, and tools amongst one another. Recent progress in collaborative digital and programmatic frameworks, such as the free and Open Source software R, have significantly expanded the opportunities for development of free-to-use, incrementally improvable, community driven tools to support evidence synthesis (e.g. EviAtlas, robvis, PRISMA2020 flow diagrams and metadat). Despite this, evidence synthesis (and meta-analysis) practitioners and methodologists who make use of R remain relatively disconnected from one another. Here, we report on a new virtual conference for evidence synthesis and meta-analysis in the R programming environment (ESMARConf ) that aims to connect these communities. By designing an entirely free and online conference from scratch, we have been able to focus efforts on maximising accessibility and equity—making these core missions for our new community of practice. As a community of practice, ESMARConf builds on the success and groundwork of the broader R community and systematic review coordinating bodies (e.g. Cochrane), but fills an important niche. ESMARConf aims to maximise accessibility and equity of participants across regions, contexts, and social backgrounds, forging a level playing field in a digital, connected, and online future of evidence synthesis. We believe that everyone should have the same access to participation and involvement, and we believe ESMARConf provides a vital opportunity to push for equitability across disciplines, regions, and personal situations

A new ecosystem for evidence synthesis

Nakagawa S, Dunn AG, Lagisz M, et al. A new ecosystem for evidence synthesis. Nature Ecology & Evolution. 2020;4:498–501