Large expert-curated database for benchmarking document similarity detection in biomedical literature search

Document recommendation systems for locating relevant literature have mostly relied on methods developed a decade ago. This is largely due to the lack of a large offline gold-standard benchmark of relevant documents that cover a variety of research fields such that newly developed literature search techniques can be compared, improved and translated into practice. To overcome this bottleneck, we have established the RElevant LIterature SearcH consortium consisting of more than 1500 scientists from 84 countries, who have collectively annotated the relevance of over 180 000 PubMed-listed articles with regard to their respective seed (input) article/s. The majority of annotations were contributed by highly experienced, original authors of the seed articles. The collected data cover 76% of all unique PubMed Medical Subject Headings descriptors. No systematic biases were observed across different experience levels, research fields or time spent on annotations. More importantly, annotations of the same document pairs contributed by different scientists were highly concordant. We further show that the three representative baseline methods used to generate recommended articles for evaluation (Okapi Best Matching 25, Term Frequency–Inverse Document Frequency and PubMed Related Articles) had similar overall performances. Additionally, we found that these methods each tend to produce distinct collections of recommended articles, suggesting that a hybrid method may be required to completely capture all relevant articles. The established database server located at https://relishdb.ict.griffith.edu.au is freely available for the downloading of annotation data and the blind testing of new methods. We expect that this benchmark will be useful for stimulating the development of new powerful techniques for title and title/abstract-based search engines for relevant articles in biomedical research

Measuring improvement in knowledge of drug policy reforms following a police education program in Tijuana, Mexico

Abstract Background Mexico’s 2009 “narcomenudeo reform” decriminalized small amounts of drugs, shifting some drug law enforcement to the states and mandating drug treatment diversion instead of incarceration. Data from Tijuana suggested limited implementation of this harm reduction-oriented policy. We studied whether a police education program (PEP) improved officers’ drug and syringe policy knowledge, and aimed to identify participant characteristics associated with improvement of drug policy knowledge. Methods Pre- and post-training surveys were self-administered by municipal police officers to measure legal knowledge. Training impact was assessed through matched paired nominal data using McNemar’s tests. Multivariable logistic regression was used to identify predictors of improved legal knowledge, as measured by officers’ ability to identify conceptual legal provisions related to syringe possession and thresholds of drugs covered under the reform. Results Of 1750 respondents comparing pre- versus post training, officers reported significant improvement (p < 0.001) in their technical understanding of syringe possession (56 to 91%) and drug amounts decriminalized, including marijuana (9 to 52%), heroin (8 to 71%), and methamphetamine (7 to 70%). The training was associated with even greater success in improving conceptual legal knowledge for syringe possession (67 to 96%) (p < 0.001), marijuana (16 to 91%), heroin (11 to 91%), and methamphetamine (11 to 89%). In multivariable modeling, those with at least a high school education were more likely to exhibit improvement of conceptual legal knowledge of syringe possession (adjusted odds ratio [aOR] 2.6, 95% CI 1.4–3.2) and decriminalization for heroin (aOR 2.7, 95% CI 1.3–4.3), methamphetamine (aOR 2.2, 95% CI 1.4–3.2), and marijuana (aOR 2.5, 95% CI 1.6–4). Conclusions Drug policy reform is often necessary, but not sufficient to achieve public health goals because of gaps in translating formal laws to policing practice. To close such gaps, PEP initiatives bundling occupational safety information with relevant legal content demonstrate clear promise. Our findings underscore additional efforts needed to raise technical knowledge of the law among personnel tasked with its enforcement. Police professionalization, including minimum educational standards, appear critical for aligning policing with harm reduction goals