Large expert-curated database for benchmarking document similarity detection in biomedical literature search

Document recommendation systems for locating relevant literature have mostly relied on methods developed a decade ago. This is largely due to the lack of a large offline gold-standard benchmark of relevant documents that cover a variety of research fields such that newly developed literature search techniques can be compared, improved and translated into practice. To overcome this bottleneck, we have established the RElevant LIterature SearcH consortium consisting of more than 1500 scientists from 84 countries, who have collectively annotated the relevance of over 180 000 PubMed-listed articles with regard to their respective seed (input) article/s. The majority of annotations were contributed by highly experienced, original authors of the seed articles. The collected data cover 76% of all unique PubMed Medical Subject Headings descriptors. No systematic biases were observed across different experience levels, research fields or time spent on annotations. More importantly, annotations of the same document pairs contributed by different scientists were highly concordant. We further show that the three representative baseline methods used to generate recommended articles for evaluation (Okapi Best Matching 25, Term Frequency-Inverse Document Frequency and PubMed Related Articles) had similar overall performances. Additionally, we found that these methods each tend to produce distinct collections of recommended articles, suggesting that a hybrid method may be required to completely capture all relevant articles. The established database server located at https://relishdb.ict.griffith.edu.au is freely available for the downloading of annotation data and the blind testing of new methods. We expect that this benchmark will be useful for stimulating the development of new powerful techniques for title and title/abstract-based search engines for relevant articles in biomedical research.Peer reviewe

Spatial panorama of malaria prevalence in Africa under climate change and interventions scenarios

Abstract Background Malaria is highly sensitive to climatic variables and is strongly influenced by the presence of vectors in a region that further contribute to parasite development and sustained disease transmission. Mathematical analysis of malaria transmission through the use and application of the value of the basic reproduction number (R0) threshold is an important and useful tool for the understanding of disease patterns. Methods Temperature dependence aspect of R0 obtained from dynamical mathematical network model was used to derive the spatial distribution maps for malaria transmission under different climatic and intervention scenarios. Model validation was conducted using MARA map and the Annual Plasmodium falciparum Entomological Inoculation Rates for Africa. Results The inclusion of the coupling between patches in dynamical model seems to have no effects on the estimate of the optimal temperature (about 25 °C) for malaria transmission. In patches environment, we were able to establish a threshold value (about α = 5) representing the ratio between the migration rates from one patch to another that has no effect on the magnitude of R0. Such findings allow us to limit the production of the spatial distribution map of R0 to a single patch model. Future projections using temperature changes indicated a shift in malaria transmission areas towards the southern and northern areas of Africa and the application of the interventions scenario yielded a considerable reduction in transmission within malaria endemic areas of the continent. Conclusions The approach employed here is a sole study that defined the limits of contemporary malaria transmission, using R0 derived from a dynamical mathematical model. It has offered a unique prospect for measuring the impacts of interventions through simple manipulation of model parameters. Projections at scale provide options to visualize and query the results, when linked to the human population could potentially deliver adequate highlight on the number of individuals at risk of malaria infection across Africa. The findings provide a reasonable basis for understanding the fundamental effects of malaria control and could contribute towards disease elimination, which is considered as a challenge especially in the context of climate change