The CAFA challenge reports improved protein function prediction and new functional annotations for hundreds of genes through experimental screens

Background The Critical Assessment of Functional Annotation (CAFA) is an ongoing, global, community-driven effort to evaluate and improve the computational annotation of protein function. Results Here, we report on the results of the third CAFA challenge, CAFA3, that featured an expanded analysis over the previous CAFA rounds, both in terms of volume of data analyzed and the types of analysis performed. In a novel and major new development, computational predictions and assessment goals drove some of the experimental assays, resulting in new functional annotations for more than 1000 genes. Specifically, we performed experimental whole-genome mutation screening in Candida albicans and Pseudomonas aureginosa genomes, which provided us with genome-wide experimental data for genes associated with biofilm formation and motility. We further performed targeted assays on selected genes in Drosophila melanogaster, which we suspected of being involved in long-term memory. Conclusion We conclude that while predictions of the molecular function and biological process annotations have slightly improved over time, those of the cellular component have not. Term-centric prediction of experimental annotations remains equally challenging; although the performance of the top methods is significantly better than the expectations set by baseline methods in C. albicans and D. melanogaster, it leaves considerable room and need for improvement. Finally, we report that the CAFA community now involves a broad range of participants with expertise in bioinformatics, biological experimentation, biocuration, and bio-ontologies, working together to improve functional annotation, computational function prediction, and our ability to manage big data in the era of large experimental screens.Peer reviewe

The CAFA challenge reports improved protein function prediction and new functional annotations for hundreds of genes through experimental screens

BackgroundThe Critical Assessment of Functional Annotation (CAFA) is an ongoing, global, community-driven effort to evaluate and improve the computational annotation of protein function.ResultsHere, we report on the results of the third CAFA challenge, CAFA3, that featured an expanded analysis over the previous CAFA rounds, both in terms of volume of data analyzed and the types of analysis performed. In a novel and major new development, computational predictions and assessment goals drove some of the experimental assays, resulting in new functional annotations for more than 1000 genes. Specifically, we performed experimental whole-genome mutation screening in Candida albicans and Pseudomonas aureginosa genomes, which provided us with genome-wide experimental data for genes associated with biofilm formation and motility. We further performed targeted assays on selected genes in Drosophila melanogaster, which we suspected of being involved in long-term memory.ConclusionWe conclude that while predictions of the molecular function and biological process annotations have slightly improved over time, those of the cellular component have not. Term-centric prediction of experimental annotations remains equally challenging; although the performance of the top methods is significantly better than the expectations set by baseline methods in C. albicans and D. melanogaster, it leaves considerable room and need for improvement. Finally, we report that the CAFA community now involves a broad range of participants with expertise in bioinformatics, biological experimentation, biocuration, and bio-ontologies, working together to improve functional annotation, computational function prediction, and our ability to manage big data in the era of large experimental screens.</p

Interventions for treatment of COVID-19 : A living systematic review with meta-analyses and trial sequential analyses (The LIVING Project)

Background Coronavirus disease 2019 (COVID-19) is a rapidly spreading disease that has caused extensive burden to individuals, families, countries, and the world. Effective treatments of COVID-19 are urgently needed. Methods and findings This is the first edition of a living systematic review of randomized clinical trials comparing the effects of all treatment interventions for participants in all age groups with COVID-19. We planned to conduct aggregate data meta-analyses, trial sequential analyses, network meta-analysis, and individual patient data meta-analyses. Our systematic review is based on Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) and Cochrane guidelines, and our 8-step procedure for better validation of clinical significance of meta-analysis results. We performed both fixed-effect and random-effects meta-analyses. Primary outcomes were all-cause mortality and serious adverse events. Secondary outcomes were admission to intensive care, mechanical ventilation, renal replacement therapy, quality of life, and nonserious adverse events. We used Grading of Recommendations Assessment, Development and Evaluation (GRADE) to assess the certainty of evidence. We searched relevant databases and websites for published and unpublished trials until August 7, 2020. Two reviewers independently extracted data and assessed trial methodology. We included 33 randomized clinical trials enrolling a total of 13,312 participants. All trials were at overall high risk of bias. We identified one trial randomizing 6,425 participants to dexamethasone versus standard care. This trial showed evidence of a beneficial effect of dexamethasone on all-cause mortality (rate ratio 0.83; 95% confidence interval [CI] 0.75-0.93; p < 0.001; low certainty) and on mechanical ventilation (risk ratio [RR] 0.77; 95% CI 0.62-0.95; p = 0.021; low certainty). It was possible to perform meta-analysis of 10 comparisons. Meta-analysis showed no evidence of a difference between remdesivir versus placebo on all-cause mortality (RR 0.74; 95% CI 0.40-1.37; p = 0.34, I2 = 58%; 2 trials; very low certainty) or nonserious adverse events (RR 0.94; 95% CI 0.80-1.11; p = 0.48, I2 = 29%; 2 trials; low certainty). Meta-analysis showed evidence of a beneficial effect of remdesivir versus placebo on serious adverse events (RR 0.77; 95% CI 0.63-0.94; p = 0.009, I2 = 0%; 2 trials; very low certainty) mainly driven by respiratory failure in one trial. Meta-analyses and trial sequential analyses showed that we could exclude the possibility that hydroxychloroquine versus standard care reduced the risk of all-cause mortality (RR 1.07; 95% CI 0.97-1.19; p = 0.17; I2 = 0%; 7 trials; low certainty) and serious adverse events (RR 1.07; 95% CI 0.96-1.18; p = 0.21; I2 = 0%; 7 trials; low certainty) by 20% or more, and meta-analysis showed evidence of a harmful effect on nonserious adverse events (RR 2.40; 95% CI 2.01-2.87; p < 0.00001; I2 = 90%; 6 trials; very low certainty). Meta-analysis showed no evidence of a difference between lopinavir-ritonavir versus standard care on serious adverse events (RR 0.64; 95% CI 0.39-1.04; p = 0.07, I2 = 0%; 2 trials; very low certainty) or nonserious adverse events (RR 1.14; 95% CI 0.85-1.53; p = 0.38, I2 = 75%; 2 trials; very low certainty). Meta-analysis showed no evidence of a difference between convalescent plasma versus standard care on all-cause mortality (RR 0.60; 95% CI 0.33-1.10; p = 0.10, I2 = 0%; 2 trials; very low certainty). Five single trials showed statistically significant results but were underpowered to confirm or reject realistic intervention effects. None of the remaining trials showed evidence of a difference on our predefined outcomes. Because of the lack of relevant data, it was not possible to perform other meta-analyses, network meta-analysis, or individual patient data meta-analyses. The main limitation of this living review is the paucity of data currently available. Furthermore, the included trials were all at risks of systematic errors and random errors. Conclusions Our results show that dexamethasone and remdesivir might be beneficial for COVID-19 patients, but the certainty of the evidence was low to very low, so more trials are needed. We can exclude the possibility of hydroxychloroquine versus standard care reducing the risk of death and serious adverse events by 20% or more. Otherwise, no evidence-based treatment for COVID-19 currently exists. This review will continuously inform best practice in treatment and clinical research of COVID-19

Interventions for treatment of COVID-19: Second edition of a living systematic review with meta-analyses and trial sequential analyses (The LIVING Project).

BackgroundCOVID-19 is a rapidly spreading disease that has caused extensive burden to individuals, families, countries, and the world. Effective treatments of COVID-19 are urgently needed. This is the second edition of a living systematic review of randomized clinical trials assessing the effects of all treatment interventions for participants in all age groups with COVID-19.Methods and findingsWe planned to conduct aggregate data meta-analyses, trial sequential analyses, network meta-analysis, and individual patient data meta-analyses. Our systematic review was based on PRISMA and Cochrane guidelines, and our eight-step procedure for better validation of clinical significance of meta-analysis results. We performed both fixed-effect and random-effects meta-analyses. Primary outcomes were all-cause mortality and serious adverse events. Secondary outcomes were admission to intensive care, mechanical ventilation, renal replacement therapy, quality of life, and non-serious adverse events. According to the number of outcome comparisons, we adjusted our threshold for significance to p = 0.033. We used GRADE to assess the certainty of evidence. We searched relevant databases and websites for published and unpublished trials until November 2, 2020. Two reviewers independently extracted data and assessed trial methodology. We included 82 randomized clinical trials enrolling a total of 40,249 participants. 81 out of 82 trials were at overall high risk of bias. Meta-analyses showed no evidence of a difference between corticosteroids versus control on all-cause mortality (risk ratio [RR] 0.89; 95% confidence interval [CI] 0.79 to 1.00; p = 0.05; I2 = 23.1%; eight trials; very low certainty), on serious adverse events (RR 0.89; 95% CI 0.80 to 0.99; p = 0.04; I2 = 39.1%; eight trials; very low certainty), and on mechanical ventilation (RR 0.86; 95% CI 0.55 to 1.33; p = 0.49; I2 = 55.3%; two trials; very low certainty). The fixed-effect meta-analyses showed indications of beneficial effects. Trial sequential analyses showed that the required information size for all three analyses was not reached. Meta-analysis (RR 0.93; 95% CI 0.82 to 1.07; p = 0.31; I2 = 0%; four trials; moderate certainty) and trial sequential analysis (boundary for futility crossed) showed that we could reject that remdesivir versus control reduced the risk of death by 20%. Meta-analysis (RR 0.82; 95% CI 0.68 to 1.00; p = 0.05; I2 = 38.9%; four trials; very low certainty) and trial sequential analysis (required information size not reached) showed no evidence of difference between remdesivir versus control on serious adverse events. Fixed-effect meta-analysis showed indications of a beneficial effect of remdesivir on serious adverse events. Meta-analysis (RR 0.40; 95% CI 0.19 to 0.87; p = 0.02; I2 = 0%; two trials; very low certainty) showed evidence of a beneficial effect of intravenous immunoglobulin versus control on all-cause mortality, but trial sequential analysis (required information size not reached) showed that the result was severely underpowered to confirm or reject realistic intervention effects. Meta-analysis (RR 0.63; 95% CI 0.35 to 1.14; p = 0.12; I2 = 77.4%; five trials; very low certainty) and trial sequential analysis (required information size not reached) showed no evidence of a difference between tocilizumab versus control on serious adverse events. Fixed-effect meta-analysis showed indications of a beneficial effect of tocilizumab on serious adverse events. Meta-analysis (RR 0.70; 95% CI 0.51 to 0.96; p = 0.02; I2 = 0%; three trials; very low certainty) showed evidence of a beneficial effect of tocilizumab versus control on mechanical ventilation, but trial sequential analysis (required information size not reached) showed that the result was severely underpowered to confirm of reject realistic intervention effects. Meta-analysis (RR 0.32; 95% CI 0.15 to 0.69; p ConclusionsNo evidence-based treatment for COVID-19 currently exists. Very low certainty evidence indicates that corticosteroids might reduce the risk of death, serious adverse events, and mechanical ventilation; that remdesivir might reduce the risk of serious adverse events; that intravenous immunoglobin might reduce the risk of death and serious adverse events; that tocilizumab might reduce the risk of serious adverse events and mechanical ventilation; and that bromhexine might reduce the risk of non-serious adverse events. More trials with low risks of bias and random errors are urgently needed. This review will continuously inform best practice in treatment and clinical research of COVID-19.Systematic review registrationPROSPERO CRD42020178787