Large expert-curated database for benchmarking document similarity detection in biomedical literature search

Document recommendation systems for locating relevant literature have mostly relied on methods developed a decade ago. This is largely due to the lack of a large offline gold-standard benchmark of relevant documents that cover a variety of research fields such that newly developed literature search techniques can be compared, improved and translated into practice. To overcome this bottleneck, we have established the RElevant LIterature SearcH consortium consisting of more than 1500 scientists from 84 countries, who have collectively annotated the relevance of over 180 000 PubMed-listed articles with regard to their respective seed (input) article/s. The majority of annotations were contributed by highly experienced, original authors of the seed articles. The collected data cover 76% of all unique PubMed Medical Subject Headings descriptors. No systematic biases were observed across different experience levels, research fields or time spent on annotations. More importantly, annotations of the same document pairs contributed by different scientists were highly concordant. We further show that the three representative baseline methods used to generate recommended articles for evaluation (Okapi Best Matching 25, Term Frequency-Inverse Document Frequency and PubMed Related Articles) had similar overall performances. Additionally, we found that these methods each tend to produce distinct collections of recommended articles, suggesting that a hybrid method may be required to completely capture all relevant articles. The established database server located at https://relishdb.ict.griffith.edu.au is freely available for the downloading of annotation data and the blind testing of new methods. We expect that this benchmark will be useful for stimulating the development of new powerful techniques for title and title/abstract-based search engines for relevant articles in biomedical research.Peer reviewe

User-friendly Bayesian regression modeling: A tutorial with rstanarm and shinystan

This tutorial provides a pragmatic introduction to specifying, estimating and interpreting single-level and hierarchical linear regression models in the Bayesian framework. We start by summarizing why one should consider the Bayesian approach to the most common forms of regression. Next we introduce the R package rstanarm for Bayesian applied regression modeling. An overview of rstanarm fundamentals accompanies step-by-step guidance for fitting a single-level regression model with the stan_glm function, and fitting hierarchical regression models with the stan_lmer function, illustrated with data from an experience sampling study on changes in affective states. Exploration of the results is facilitated by the intuitive and user-friendly shinystan package. Data and scripts are available on the Open Science Framework page of the project. For readers unfamiliar with R, this tutorial is self-contained to enable all researchers who apply regression techniques to try these methods with their own data. Regression modeling with the functions in the rstanarm package will be a straightforward transition for researchers familiar with their frequentist counterparts, lm (or glm) and lmer

Do People Agree on What Makes One Feel Loved? A Cognitive Psychometric Approach to the Consensus on Felt Love

<div><p>This pragmatic study examines love as a mode of communication. Our focus is on the receiver side: what makes an individual feel loved and how <i>felt love</i> is defined through daily interactions. Our aim is to explore everyday life scenarios in which people might experience love, and to consider people’s converging and diverging judgments about which scenarios indicate felt love. We apply a cognitive psychometric approach to quantify a receiver’s ability to detect, understand, and know that they are loved. Through crowd-sourcing, we surveyed lay participants about whether various scenarios were indicators of felt love. We thus quantify these responses to make inference about consensus judgments of felt love, measure individual levels of agreement with consensus, and assess individual response styles. More specifically, we (1) derive consensus judgments on felt love; (2) describe its characteristics in qualitative and quantitative terms, (3) explore individual differences in both (a) participant agreement with consensus, and (b) participant judgment when uncertain about shared knowledge, and (4) test whether individual differences can be meaningfully linked to explanatory variables. Results indicate that people converge towards a shared cognitive model of felt love. Conversely, respondents showed heterogeneity in knowledge of consensus, and in dealing with uncertainty. We found that, when facing uncertainty, female respondents and people in relationships more frequently judge scenarios as indicators of felt love. Moreover, respondents from smaller households tend to know more about consensus judgments of felt love, while respondents from larger households are more willing to guess when unsure of consensus.</p></div

Distribution of person- and item-specific Extended Condorcet Model parameter estimates.

<p>Frequency of posterior mean estimates. The middle of the bar indicates their mean; The end points are 1 standard deviation away.</p

Raw data means and model based estimates on selected felt love items.

<p>The second columns shows the mean of the answers for the item with ‘True’ coded as 1 and ‘False’ as 0. The posterior distribution on the consensus parameters for each item is summarized in columns 3 and 4, in terms of posterior median estimate, labeled as ‘True’ for 1 and ‘False’ for 0 and posterior standard deviation (abbreviated as ‘psd’). It quantifies standard error around the point estimate. The last column shows the item difficulty rank of the item in ascending order.</p

Processing tree depiction of the Extended Condorcet Model.

<p>Processing tree depiction of the Extended Condorcet Model.</p

Alternative Models for Small Samples in Psychological Research: Applying Linear Mixed Effects Models and Generalized Estimating Equations to Repeated Measures Data.

Unavoidable sample size issues beset psychological research that involves scarce populations or costly laboratory procedures. When incorporating longitudinal designs these samples are further reduced by traditional modeling techniques, which perform listwise deletion for any instance of missing data. Moreover, these techniques are limited in their capacity to accommodate alternative correlation structures that are common in repeated measures studies. Researchers require sound quantitative methods to work with limited but valuable measures without degrading their data sets. This article provides a brief tutorial and exploration of two alternative longitudinal modeling techniques, linear mixed effects models and generalized estimating equations, as applied to a repeated measures study (n = 12) of pairmate attachment and social stress in primates. Both techniques provide comparable results, but each model offers unique information that can be helpful when deciding the right analytic tool

Challenges to the pair bond: neural and hormonal effects of separation and reunion in a monogamous primate

Social monogamy at its most basic is a group structure in which two adults form a unit and share a territory. However, many socially monogamous pairs display attachment relationships known as pair bonds, in which there is a mutual preference for the partner and distress upon separation. The neural and hormonal basis of this response to separation from the adult pair mate is under-studied. In this project, we examined this response in male titi monkeys (Callicebus cupreus), a socially monogamous New World primate. Males underwent a baseline scan, a short separation (48 hours), a long separation (app. two weeks), a reunion with the female pair mate, and an encounter with a female stranger (with nine males completing all five conditions). Regional cerebral glucose metabolism was measured via positron emission tomography (PET) imaging using 18F-fluorodeoxyglucose (FDG) co-registered with structural MRI, and region of interest analysis was carried out. In addition, plasma was collected and assayed for cortisol, oxytocin (OT), vasopressin (AVP), glucose, and insulin concentrations. Cerebrospinal fluid (CSF) was collected and assayed for OT and AVP. We used generalized estimating equations to examine significant changes from baseline. Short separations were characterized by decreases in FDG uptake, in comparison to baseline, in the lateral septum (LS), ventral pallidum (VP), paraventricular nucleus of the hypothalamus (PVN), periaqueductal gray (PAG), and cerebellum, as well as increases in CSF OT, and plasma cortisol and insulin. Long separations differed from baseline in reduced FDG uptake in the central amygdala (CeA), reduced whole brain FDG uptake, increased CSF OT and increased plasma insulin. The response on encounter with a stranger female depended on whether or not the male had previously reproduced with his pair mate, suggesting that transitions to fatherhood contribute to the neurobiology underlying response to a novel female. Reunion with the partner appeared to stimulate coordinated release of central and peripheral OT. The observed changes suggest the involvement of OT and AVP systems, as well as limbic and striatal areas, during separation and reunion from the pair mate