Large expert-curated database for benchmarking document similarity detection in biomedical literature search

Document recommendation systems for locating relevant literature have mostly relied on methods developed a decade ago. This is largely due to the lack of a large offline gold-standard benchmark of relevant documents that cover a variety of research fields such that newly developed literature search techniques can be compared, improved and translated into practice. To overcome this bottleneck, we have established the RElevant LIterature SearcH consortium consisting of more than 1500 scientists from 84 countries, who have collectively annotated the relevance of over 180 000 PubMed-listed articles with regard to their respective seed (input) article/s. The majority of annotations were contributed by highly experienced, original authors of the seed articles. The collected data cover 76% of all unique PubMed Medical Subject Headings descriptors. No systematic biases were observed across different experience levels, research fields or time spent on annotations. More importantly, annotations of the same document pairs contributed by different scientists were highly concordant. We further show that the three representative baseline methods used to generate recommended articles for evaluation (Okapi Best Matching 25, Term Frequency-Inverse Document Frequency and PubMed Related Articles) had similar overall performances. Additionally, we found that these methods each tend to produce distinct collections of recommended articles, suggesting that a hybrid method may be required to completely capture all relevant articles. The established database server located at https://relishdb.ict.griffith.edu.au is freely available for the downloading of annotation data and the blind testing of new methods. We expect that this benchmark will be useful for stimulating the development of new powerful techniques for title and title/abstract-based search engines for relevant articles in biomedical research.Peer reviewe

Was steckt hinter der Eule?

Peer Reviewe

Mutations in FKBP14 cause a variant of Ehlers-Danlos syndrome with progressive kyphoscoliosis, myopathy, and hearing loss

We report on an autosomal-recessive variant of Ehlers-Danlos syndrome (EDS) characterized by severe muscle hypotonia at birth, progressive scoliosis, joint hypermobility, hyperelastic skin, myopathy, sensorineural hearing impairment, and normal pyridinoline excretion in urine. Clinically, the disorder shares many features with the kyphoscoliotic type of EDS (EDS VIA) and Ullrich congenital muscular dystrophy. Linkage analysis in a large Tyrolean kindred identified a homozygous frameshift mutation in FKBP14 in two affected individuals. Based on the cardinal clinical characteristics of the disorder, four additional individuals originating from different European countries were identified who carried either homozygous or compound heterozygous mutations in FKBP14. FKBP14 belongs to the family of FK506-binding peptidyl-prolyl cis-trans isomerases (PPIases). ER-resident FKBPs have been suggested to act as folding catalysts by accelerating cis-trans isomerization of peptidyl-prolyl bonds and to act occasionally also as chaperones. We demonstrate that FKBP14 is localized in the endoplasmic reticulum (ER) and that deficiency of FKBP14 leads to enlarged ER cisterns in dermal fibroblasts in vivo. Furthermore, indirect immunofluorescence of FKBP14-deficient fibroblasts indicated an altered assembly of the extracellular matrix in vitro. These findings suggest that a disturbance of protein folding in the ER affecting one or more components of the extracellular matrix might cause the generalized connective tissue involvement in this disorder. FKBP14 mutation analysis should be considered in all individuals with apparent kyphoscoliotic type of EDS and normal urinary pyridinoline excretion, in particular in conjunction with sensorineural hearing impairment

Large expert-curated database for benchmarking document similarity detection in biomedical literature search

Document recommendation systems for locating relevant literature have mostly relied on methods developed a decade ago. This is largely due to the lack of a large offline gold-standard benchmark of relevant documents that cover a variety of research fields such that newly developed literature search techniques can be compared, improved and translated into practice. To overcome this bottleneck, we have established the RElevant LIterature SearcH consortium consisting of more than 1500 scientists from 84 countries, who have collectively annotated the relevance of over 180 000 PubMed-listed articles with regard to their respective seed (input) article/s. The majority of annotations were contributed by highly experienced, original authors of the seed articles. The collected data cover 76% of all unique PubMed Medical Subject Headings descriptors. No systematic biases were observed across different experience levels, research fields or time spent on annotations. More importantly, annotations of the same document pairs contributed by different scientists were highly concordant. We further show that the three representative baseline methods used to generate recommended articles for evaluation (Okapi Best Matching 25, Term Frequency-Inverse Document Frequency and PubMed Related Articles) had similar overall performances. Additionally, we found that these methods each tend to produce distinct collections of recommended articles, suggesting that a hybrid method may be required to completely capture all relevant articles. The established database server located at https://relishdb.ict.griffith.edu.au is freely available for the downloading of annotation data and the blind testing of new methods. We expect that this benchmark will be useful for stimulating the development of new powerful techniques for title and title/abstract-based search engines for relevant articles in biomedical science. © The Author(s) 2019. Published by Oxford University Press