3 research outputs found

    Large expert-curated database for benchmarking document similarity detection in biomedical literature search

    Get PDF
    Document recommendation systems for locating relevant literature have mostly relied on methods developed a decade ago. This is largely due to the lack of a large offline gold-standard benchmark of relevant documents that cover a variety of research fields such that newly developed literature search techniques can be compared, improved and translated into practice. To overcome this bottleneck, we have established the RElevant LIterature SearcH consortium consisting of more than 1500 scientists from 84 countries, who have collectively annotated the relevance of over 180 000 PubMed-listed articles with regard to their respective seed (input) article/s. The majority of annotations were contributed by highly experienced, original authors of the seed articles. The collected data cover 76% of all unique PubMed Medical Subject Headings descriptors. No systematic biases were observed across different experience levels, research fields or time spent on annotations. More importantly, annotations of the same document pairs contributed by different scientists were highly concordant. We further show that the three representative baseline methods used to generate recommended articles for evaluation (Okapi Best Matching 25, Term Frequency-Inverse Document Frequency and PubMed Related Articles) had similar overall performances. Additionally, we found that these methods each tend to produce distinct collections of recommended articles, suggesting that a hybrid method may be required to completely capture all relevant articles. The established database server located at https://relishdb.ict.griffith.edu.au is freely available for the downloading of annotation data and the blind testing of new methods. We expect that this benchmark will be useful for stimulating the development of new powerful techniques for title and title/abstract-based search engines for relevant articles in biomedical research.Peer reviewe

    The fate of nitrogen and phosphorus at the land-sea margin of the North Atlantic Ocean

    No full text
    Five large rivers that discharge on the western North Atlantic continental shelf carry about 45% of the nitrogen (N) and 70% of the phosphorus (P) that others estimate to be the total flux of these elements from the entire North Atlantic watershed, including North, Central and South America, Europe, and Northwest Africa. We estimate that 61 · 10 9 moles y - 1 of N and 20 · 10 9 moles y -1 of P from the large rivers are buried with sediments in their deltas, and that an equal amount of N and P from the large rivers is lost to the shelf through burial of river sediments that are deposited directly on the continental slope. The effective transport of active N and P from land to the shelf through the very large rivers is thus reduced to 292 · 10 9 moles y -1 of N and 13 · 10 9 moles y -1 of P. The remaining riverine fluxes from land must pass through estuaries. An analysis of annual total N and total P budgets for various estuaries around the North Atlantic revealed that the net fractional transport of these nutrients through estuaries to the continental shelf is inversely correlated with the log mean residence time of water in the system. This is consistent with numerous observations of nutrient retention and loss in temperate lakes. Denitrification is the major process responsible for removing N in most estuaries, and the fraction of total N input that is denitrified appears to be directly proportional to the log mean water residence time. In general, we estimate that estuarine processes retain and remove 30-65% of the total N and 10-55% of the total P that would otherwise pass into the coastal ocean. The resulting transport through estuaries to the shelf amounts to 172-335 · 10 9 moles y -1 of N and 11-19 · 10 9 moles y -1 of P. These values are similar to the effective contribution from the large rivers that discharge directly on the shelf. For the North Atlantic shelf as a whole, N fluxes from major rivers and estuaries exceed atmospheric deposition by a factor of 3.5-4.7, but this varies widely among regions of the shelf. For example, on the U.S. Atlantic shelf and on the northwest European shelf, atmospheric deposition of N may exceed estuarine exports. Denitrification in shelf sediments exceeds the combined N input from land and atmosphere by a factor of 1.4-2.2. This deficit must be met by a flux of N from the deeper ocean. Burial of organic matter fixed on the shelf removes only a small fraction of the total N and P input (2-12% of N from land and atmosphere; 1-17% of P), but it may be a significant loss for P in the North Sea and some other regions. The removal of N and P in fisheries landings is very small. The gross exchange of N and P between the shelf and the open ocean is much larger than inputs from land and, for the North Atlantic shelf as a whole, it may be much larger than the N and P removed through denitrification, burial, and fisheries. Overall, the North Atlantic continental shelf appears to remove some 700-950 · 10 9 moles of N each year from the deep ocean and to transport somewhere between 18 and 30 · 10 9 moles of P to the open sea. If the N and P associated with riverine sediments deposited on the continental slope are included in the total balance, the net flux of N to the shelf is reduced by 60 · 10 9 moles y -1 and the P flux to the ocean is increased by 20 γ 10 9 moles y -1. These conclusions are quite tentative, however, because of large uncertainties in our estimates of some important terms in the shelf mass balance.SCOPUS: ar.jinfo:eu-repo/semantics/publishe

    Large expert-curated database for benchmarking document similarity detection in biomedical literature search

    No full text
    Document recommendation systems for locating relevant literature have mostly relied on methods developed a decade ago. This is largely due to the lack of a large offline gold-standard benchmark of relevant documents that cover a variety of research fields such that newly developed literature search techniques can be compared, improved and translated into practice. To overcome this bottleneck, we have established the RElevant LIterature SearcH consortium consisting of more than 1500 scientists from 84 countries, who have collectively annotated the relevance of over 180 000 PubMed-listed articles with regard to their respective seed (input) article/s. The majority of annotations were contributed by highly experienced, original authors of the seed articles. The collected data cover 76% of all unique PubMed Medical Subject Headings descriptors. No systematic biases were observed across different experience levels, research fields or time spent on annotations. More importantly, annotations of the same document pairs contributed by different scientists were highly concordant. We further show that the three representative baseline methods used to generate recommended articles for evaluation (Okapi Best Matching 25, Term Frequency-Inverse Document Frequency and PubMed Related Articles) had similar overall performances. Additionally, we found that these methods each tend to produce distinct collections of recommended articles, suggesting that a hybrid method may be required to completely capture all relevant articles. The established database server located at https://relishdb.ict.griffith.edu.au is freely available for the downloading of annotation data and the blind testing of new methods. We expect that this benchmark will be useful for stimulating the development of new powerful techniques for title and title/abstract-based search engines for relevant articles in biomedical science. © The Author(s) 2019. Published by Oxford University Press
    corecore