Large expert-curated database for benchmarking document similarity detection in biomedical literature search

Document recommendation systems for locating relevant literature have mostly relied on methods developed a decade ago. This is largely due to the lack of a large offline gold-standard benchmark of relevant documents that cover a variety of research fields such that newly developed literature search techniques can be compared, improved and translated into practice. To overcome this bottleneck, we have established the RElevant LIterature SearcH consortium consisting of more than 1500 scientists from 84 countries, who have collectively annotated the relevance of over 180 000 PubMed-listed articles with regard to their respective seed (input) article/s. The majority of annotations were contributed by highly experienced, original authors of the seed articles. The collected data cover 76% of all unique PubMed Medical Subject Headings descriptors. No systematic biases were observed across different experience levels, research fields or time spent on annotations. More importantly, annotations of the same document pairs contributed by different scientists were highly concordant. We further show that the three representative baseline methods used to generate recommended articles for evaluation (Okapi Best Matching 25, Term Frequency-Inverse Document Frequency and PubMed Related Articles) had similar overall performances. Additionally, we found that these methods each tend to produce distinct collections of recommended articles, suggesting that a hybrid method may be required to completely capture all relevant articles. The established database server located at https://relishdb.ict.griffith.edu.au is freely available for the downloading of annotation data and the blind testing of new methods. We expect that this benchmark will be useful for stimulating the development of new powerful techniques for title and title/abstract-based search engines for relevant articles in biomedical research.Peer reviewe

HNA-3a and HNA-3b antigens among 9 ethnic populations and the Han population in Southwest China

Abstract Background Human neutrophil antigen 3 (HNA-3) is encoded by the SLC44A2 gene. Antibodies against HNAs can cause severe, often fatal, transfusion reactions, known as transfusion-related acute lung injury, and neonatal neutropenia. We explored the 2 common HNA-3 variants in 9 ethnic populations residing in Sichuan and Yunnan provinces of China as compared to the Han population. Methods We genotyped for SLC44A2 (rs2288904) by polymerase chain reaction sequence-based typing among blood donors, for a total of 2206 individuals in Yunnan and 376 in Sichuan. Results The SLC44A2*02 allele (HNA-3b antigen) frequency varied between 0.24 and 0.33 for all 9 ethnic populations in Yunnan, including Zhuang, Derung, Hani, Lisu, Bai, Miao, Dai, Naxi, and Yi. Specifically, the Yi ethnicity did not present an unusually great SLC44A2*02 frequency at any of the 4 locations examined in Yunnan. Except of the Yi ethnicity in Sichuan (0.40), the Han ethnicity, as the majority population group, had the greatest SLC44A2*02 frequency with 0.39 in Yunnan and 0.35 in Sichuan. Conclusion The ethnic populations in Southwest China are not at an increased risk for anti-HNA3a compared to the Han population, with the possible exception of Yi in Sichuan. Our data, however, corroborated the known high prevalence of SLC44A2*02 in Han populations. Hence, the Han populations in Yunnan, Sichuan and elsewhere in China are at a comparatively great risk for developing HNA-3a antibodies

Recommendation for validation and quality assurance of non-invasive prenatal testing for foetal blood groups and implications for IVD risk classification according to EU regulations

Background and Objectives: Non-invasive assays for predicting foetal blood group status in pregnancy serve as valuable clinical tools in the management of pregnancies at risk of detrimental consequences due to blood group antigen incompatibility. To secure clinical applicability, assays for non-invasive prenatal testing of foetal blood groups need to follow strict rules for validation and quality assurance. Here, we present a multi-national position paper with specific recommendations for validation and quality assurance for such assays and discuss their risk classification according to EU regulations. Materials and Methods: We reviewed the literature covering validation for in-vitro diagnostic (IVD) assays in general and for non-invasive foetal RHD genotyping in particular. Recommendations were based on the result of discussions between co-authors. Results: In relation to Annex VIII of the In-Vitro-Diagnostic Medical Device Regulation 2017/746 of the European Parliament and the Council, assays for non-invasive prenatal testing of foetal blood groups are risk class D devices. In our opinion, screening for targeted anti-D prophylaxis for non-immunized RhD negative women should be placed under risk class C. To ensure high quality of non-invasive foetal blood group assays within and beyond the European Union, we present specific recommendations for validation and quality assurance in terms of analytical detection limit, range and linearity, precision, robustness, pre-analytics and use of controls in routine testing. With respect to immunized women, different requirements for validation and IVD risk classification are discussed. Conclusion: These recommendations should be followed to ensure appropriate assay performance and applicability for clinical use of both commercial and in-house assays

Large expert-curated database for benchmarking document similarity detection in biomedical literature search

Document recommendation systems for locating relevant literature have mostly relied on methods developed a decade ago. This is largely due to the lack of a large offline gold-standard benchmark of relevant documents that cover a variety of research fields such that newly developed literature search techniques can be compared, improved and translated into practice. To overcome this bottleneck, we have established the RElevant LIterature SearcH consortium consisting of more than 1500 scientists from 84 countries, who have collectively annotated the relevance of over 180 000 PubMed-listed articles with regard to their respective seed (input) article/s. The majority of annotations were contributed by highly experienced, original authors of the seed articles. The collected data cover 76% of all unique PubMed Medical Subject Headings descriptors. No systematic biases were observed across different experience levels, research fields or time spent on annotations. More importantly, annotations of the same document pairs contributed by different scientists were highly concordant. We further show that the three representative baseline methods used to generate recommended articles for evaluation (Okapi Best Matching 25, Term Frequency-Inverse Document Frequency and PubMed Related Articles) had similar overall performances. Additionally, we found that these methods each tend to produce distinct collections of recommended articles, suggesting that a hybrid method may be required to completely capture all relevant articles. The established database server located at https://relishdb.ict.griffith.edu.au is freely available for the downloading of annotation data and the blind testing of new methods. We expect that this benchmark will be useful for stimulating the development of new powerful techniques for title and title/abstract-based search engines for relevant articles in biomedical science. © The Author(s) 2019. Published by Oxford University Press