Large expert-curated database for benchmarking document similarity detection in biomedical literature search

Document recommendation systems for locating relevant literature have mostly relied on methods developed a decade ago. This is largely due to the lack of a large offline gold-standard benchmark of relevant documents that cover a variety of research fields such that newly developed literature search techniques can be compared, improved and translated into practice. To overcome this bottleneck, we have established the RElevant LIterature SearcH consortium consisting of more than 1500 scientists from 84 countries, who have collectively annotated the relevance of over 180 000 PubMed-listed articles with regard to their respective seed (input) article/s. The majority of annotations were contributed by highly experienced, original authors of the seed articles. The collected data cover 76% of all unique PubMed Medical Subject Headings descriptors. No systematic biases were observed across different experience levels, research fields or time spent on annotations. More importantly, annotations of the same document pairs contributed by different scientists were highly concordant. We further show that the three representative baseline methods used to generate recommended articles for evaluation (Okapi Best Matching 25, Term Frequency-Inverse Document Frequency and PubMed Related Articles) had similar overall performances. Additionally, we found that these methods each tend to produce distinct collections of recommended articles, suggesting that a hybrid method may be required to completely capture all relevant articles. The established database server located at https://relishdb.ict.griffith.edu.au is freely available for the downloading of annotation data and the blind testing of new methods. We expect that this benchmark will be useful for stimulating the development of new powerful techniques for title and title/abstract-based search engines for relevant articles in biomedical research.Peer reviewe

Risk of COVID-19 after natural infection or vaccinationResearch in context

Background: While vaccines have established utility against COVID-19, phase 3 efficacy studies have generally not comprehensively evaluated protection provided by previous infection or hybrid immunity (previous infection plus vaccination). Individual patient data from US government-supported harmonized vaccine trials provide an unprecedented sample population to address this issue. We characterized the protective efficacy of previous SARS-CoV-2 infection and hybrid immunity against COVID-19 early in the pandemic over three-to six-month follow-up and compared with vaccine-associated protection. Methods: In this post-hoc cross-protocol analysis of the Moderna, AstraZeneca, Janssen, and Novavax COVID-19 vaccine clinical trials, we allocated participants into four groups based on previous-infection status at enrolment and treatment: no previous infection/placebo; previous infection/placebo; no previous infection/vaccine; and previous infection/vaccine. The main outcome was RT-PCR-confirmed COVID-19 >7–15 days (per original protocols) after final study injection. We calculated crude and adjusted efficacy measures. Findings: Previous infection/placebo participants had a 92% decreased risk of future COVID-19 compared to no previous infection/placebo participants (overall hazard ratio [HR] ratio: 0.08; 95% CI: 0.05–0.13). Among single-dose Janssen participants, hybrid immunity conferred greater protection than vaccine alone (HR: 0.03; 95% CI: 0.01–0.10). Too few infections were observed to draw statistical inferences comparing hybrid immunity to vaccine alone for other trials. Vaccination, previous infection, and hybrid immunity all provided near-complete protection against severe disease. Interpretation: Previous infection, any hybrid immunity, and two-dose vaccination all provided substantial protection against symptomatic and severe COVID-19 through the early Delta period. Thus, as a surrogate for natural infection, vaccination remains the safest approach to protection. Funding: National Institutes of Health

Risk of COVID-19 after natural infection or vaccinationResearch in context

Summary: Background: While vaccines have established utility against COVID-19, phase 3 efficacy studies have generally not comprehensively evaluated protection provided by previous infection or hybrid immunity (previous infection plus vaccination). Individual patient data from US government-supported harmonized vaccine trials provide an unprecedented sample population to address this issue. We characterized the protective efficacy of previous SARS-CoV-2 infection and hybrid immunity against COVID-19 early in the pandemic over three-to six-month follow-up and compared with vaccine-associated protection. Methods: In this post-hoc cross-protocol analysis of the Moderna, AstraZeneca, Janssen, and Novavax COVID-19 vaccine clinical trials, we allocated participants into four groups based on previous-infection status at enrolment and treatment: no previous infection/placebo; previous infection/placebo; no previous infection/vaccine; and previous infection/vaccine. The main outcome was RT-PCR-confirmed COVID-19 >7–15 days (per original protocols) after final study injection. We calculated crude and adjusted efficacy measures. Findings: Previous infection/placebo participants had a 92% decreased risk of future COVID-19 compared to no previous infection/placebo participants (overall hazard ratio [HR] ratio: 0.08; 95% CI: 0.05–0.13). Among single-dose Janssen participants, hybrid immunity conferred greater protection than vaccine alone (HR: 0.03; 95% CI: 0.01–0.10). Too few infections were observed to draw statistical inferences comparing hybrid immunity to vaccine alone for other trials. Vaccination, previous infection, and hybrid immunity all provided near-complete protection against severe disease. Interpretation: Previous infection, any hybrid immunity, and two-dose vaccination all provided substantial protection against symptomatic and severe COVID-19 through the early Delta period. Thus, as a surrogate for natural infection, vaccination remains the safest approach to protection. Funding: National Institutes of Health

Evidence for three genetic loci involved in both anorexia nervosa risk and variation of body mass index

The maintenance of normal body weight is disrupted in patients with anorexia nervosa (AN) for prolonged periods of time. Prior to the onset of AN, premorbid body mass index (BMI) spans the entire range from underweight to obese. After recovery, patients have reduced rates of overweight and obesity. As such, loci involved in body weight regulation may also be relevant for AN and vice versa. Our primary analysis comprised a cross-trait analysis of the 1000 single-nucleotide polymorphisms (SNPs) with the lowest P-values in a genome-wide association meta-analysis (GWAMA) of AN (GCAN) for evidence of association in the largest published GWAMA for BMI (GIANT). Subsequently we performed sex-stratified analyses for these 1000 SNPs. Functional ex vivo studies on four genes ensued. Lastly, a look-up of GWAMA-derived BMI-related loci was performed in the AN GWAMA. We detected significant associations (P-values <5 × 10-5, Bonferroni-corrected P<0.05) for nine SNP alleles at three independent loci. Interestingly, all AN susceptibility alleles were consistently associated with increased BMI. None of the genes (chr. 10: CTBP2, chr. 19: CCNE1, chr. 2: CARF and NBEAL1; the latter is a region with high linkage disequilibrium) nearest to these SNPs has previously been associated with AN or obesity. Sex-stratified analyses revealed that the strongest BMI signal originated predominantly from females (chr. 10 rs1561589; Poverall: 2.47 × 10-06/Pfemales: 3.45 × 10-07/Pmales: 0.043). Functional ex vivo studies in mice revealed reduced hypothalamic expression of Ctbp2 and Nbeal1 after fasting. Hypothalamic expression of Ctbp2 was increased in diet-induced obese (DIO) mice as compared with age-matched lean controls. We observed no evidence for associations for the look-up of BMI-related loci in the AN GWAMA. A cross-trait analysis of AN and BMI loci revealed variants at three chromosomal loci with potential joint impact. The chromosome 10 locus is particularly promising given that the association with obesity was primarily driven by females. In addition, the detected altered hypothalamic expression patterns of Ctbp2 and Nbeal1 as a result of fasting and DIO implicate these genes in weight regulation

Evidence for three genetic loci involved in both anorexia nervosa risk and variation of body mass index

The maintenance of normal body weight is disrupted in patients with anorexia nervosa (AN) for prolonged periods of time. Prior to the onset of AN, premorbid body mass index (BMI) spans the entire range from underweight to obese. After recovery, patients have reduced rates of overweight and obesity. As such, loci involved in body weight regulation may also be relevant for AN and vice versa. Our primary analysis comprised a cross-trait analysis of the 1000 single-nucleotide polymorphisms (SNPs) with the lowest P-values in a genome-wide association meta-analysis (GWAMA) of AN (GCAN) for evidence of association in the largest published GWAMA for BMI (GIANT). Subsequently we performed sex-stratified analyses for these 1000 SNPs. Functional ex vivo studies on four genes ensued. Lastly, a look-up of GWAMA-derived BMI-related loci was performed in the AN GWAMA. We detected significant associations (P-values <5 × 10-5, Bonferroni-corrected P<0.05) for nine SNP alleles at three independent loci. Interestingly, all AN susceptibility alleles were consistently associated with increased BMI. None of the genes (chr. 10: CTBP2, chr. 19: CCNE1, chr. 2: CARF and NBEAL1; the latter is a region with high linkage disequilibrium) nearest to these SNPs has previously been associated with AN or obesity. Sex-stratified analyses revealed that the strongest BMI signal originated predominantly from females (chr. 10 rs1561589; Poverall: 2.47 × 10-06/Pfemales: 3.45 × 10-07/Pmales: 0.043). Functional ex vivo studies in mice revealed reduced hypothalamic expression of Ctbp2 and Nbeal1 after fasting. Hypothalamic expression of Ctbp2 was increased in diet-induced obese (DIO) mice as compared with age-matched lean controls. We observed no evidence for associations for the look-up of BMI-related loci in the AN GWAMA. A cross-trait analysis of AN and BMI loci revealed variants at three chromosomal loci with potential joint impact. The chromosome 10 locus is particularly promising given that the association with obesity was primarily driven by females. In addition, the detected altered hypothalamic expression patterns of Ctbp2 and Nbeal1 as a result of fasting and DIO implicate these genes in weight regulation

Evidence for three genetic loci involved in both anorexia nervosa risk and variation of body mass index

The maintenance of normal body weight is disrupted in patients with anorexia nervosa (AN) for prolonged periods of time. Prior to the onset of AN, premorbid body mass index (BMI) spans the entire range from underweight to obese. After recovery, patients have reduced rates of overweight and obesity. As such, loci involved in body weight regulation may also be relevant for AN and vice versa. Our primary analysis comprised a cross-trait analysis of the 1000 single-nucleotide polymorphisms (SNPs) with the lowest P-values in a genome-wide association meta-analysis (GWAMA) of AN (GCAN) for evidence of association in the largest published GWAMA for BMI (GIANT). Subsequently we performed sex-stratified analyses for these 1000 SNPs. Functional ex vivo studies on four genes ensued. Lastly, a look-up of GWAMA-derived BMI-related loci was performed in the AN GWAMA. We detected significant associations (P-values <5 × 10−5, Bonferroni-corrected P<0.05) for nine SNP alleles at three independent loci. Interestingly, all AN susceptibility alleles were consistently associated with increased BMI. None of the genes (chr. 10: CTBP2, chr. 19: CCNE1, chr. 2: CARF and NBEAL1; the latter is a region with high linkage disequilibrium) nearest to these SNPs has previously been associated with AN or obesity. Sex-stratified analyses revealed that the strongest BMI signal originated predominantly from females (chr. 10 rs1561589; Poverall: 2.47 × 10−06/Pfemales: 3.45 × 10−07/Pmales: 0.043). Functional ex vivo studies in mice revealed reduced hypothalamic expression of Ctbp2 and Nbeal1 after fasting. Hypothalamic expression of Ctbp2 was increased in diet-induced obese (DIO) mice as compared with age-matched lean controls. We observed no evidence for associations for the look-up of BMI-related loci in the AN GWAMA. A cross-trait analysis of AN and BMI loci revealed variants at three chromosomal loci with potential joint impact. The chromosome 10 locus is particularly promising given that the association with obesity was primarily driven by females. In addition, the detected altered hypothalamic expression patterns of Ctbp2 and Nbeal1 as a result of fasting and DIO implicate these genes in weight regulation

Large expert-curated database for benchmarking document similarity detection in biomedical literature search

Document recommendation systems for locating relevant literature have mostly relied on methods developed a decade ago. This is largely due to the lack of a large offline gold-standard benchmark of relevant documents that cover a variety of research fields such that newly developed literature search techniques can be compared, improved and translated into practice. To overcome this bottleneck, we have established the RElevant LIterature SearcH consortium consisting of more than 1500 scientists from 84 countries, who have collectively annotated the relevance of over 180 000 PubMed-listed articles with regard to their respective seed (input) article/s. The majority of annotations were contributed by highly experienced, original authors of the seed articles. The collected data cover 76% of all unique PubMed Medical Subject Headings descriptors. No systematic biases were observed across different experience levels, research fields or time spent on annotations. More importantly, annotations of the same document pairs contributed by different scientists were highly concordant. We further show that the three representative baseline methods used to generate recommended articles for evaluation (Okapi Best Matching 25, Term Frequency-Inverse Document Frequency and PubMed Related Articles) had similar overall performances. Additionally, we found that these methods each tend to produce distinct collections of recommended articles, suggesting that a hybrid method may be required to completely capture all relevant articles. The established database server located at https://relishdb.ict.griffith.edu.au is freely available for the downloading of annotation data and the blind testing of new methods. We expect that this benchmark will be useful for stimulating the development of new powerful techniques for title and title/abstract-based search engines for relevant articles in biomedical science. © The Author(s) 2019. Published by Oxford University Press