Large expert-curated database for benchmarking document similarity detection in biomedical literature search

Document recommendation systems for locating relevant literature have mostly relied on methods developed a decade ago. This is largely due to the lack of a large offline gold-standard benchmark of relevant documents that cover a variety of research fields such that newly developed literature search techniques can be compared, improved and translated into practice. To overcome this bottleneck, we have established the RElevant LIterature SearcH consortium consisting of more than 1500 scientists from 84 countries, who have collectively annotated the relevance of over 180 000 PubMed-listed articles with regard to their respective seed (input) article/s. The majority of annotations were contributed by highly experienced, original authors of the seed articles. The collected data cover 76% of all unique PubMed Medical Subject Headings descriptors. No systematic biases were observed across different experience levels, research fields or time spent on annotations. More importantly, annotations of the same document pairs contributed by different scientists were highly concordant. We further show that the three representative baseline methods used to generate recommended articles for evaluation (Okapi Best Matching 25, Term Frequency–Inverse Document Frequency and PubMed Related Articles) had similar overall performances. Additionally, we found that these methods each tend to produce distinct collections of recommended articles, suggesting that a hybrid method may be required to completely capture all relevant articles. The established database server located at https://relishdb.ict.griffith.edu.au is freely available for the downloading of annotation data and the blind testing of new methods. We expect that this benchmark will be useful for stimulating the development of new powerful techniques for title and title/abstract-based search engines for relevant articles in biomedical research

Large expert-curated database for benchmarking document similarity detection in biomedical literature search

Document recommendation systems for locating relevant literature have mostly relied on methods developed a decade ago. This is largely due to the lack of a large offline gold-standard benchmark of relevant documents that cover a variety of research fields such that newly developed literature search techniques can be compared, improved and translated into practice. To overcome this bottleneck, we have established the RElevant LIterature SearcH consortium consisting of more than 1500 scientists from 84 countries, who have collectively annotated the relevance of over 180 000 PubMed-listed articles with regard to their respective seed (input) article/s. The majority of annotations were contributed by highly experienced, original authors of the seed articles. The collected data cover 76% of all unique PubMed Medical Subject Headings descriptors. No systematic biases were observed across different experience levels, research fields or time spent on annotations. More importantly, annotations of the same document pairs contributed by different scientists were highly concordant. We further show that the three representative baseline methods used to generate recommended articles for evaluation (Okapi Best Matching 25, Term Frequency-Inverse Document Frequency and PubMed Related Articles) had similar overall performances. Additionally, we found that these methods each tend to produce distinct collections of recommended articles, suggesting that a hybrid method may be required to completely capture all relevant articles. The established database server located at https://relishdb.ict.griffith.edu.au is freely available for the downloading of annotation data and the blind testing of new methods. We expect that this benchmark will be useful for stimulating the development of new powerful techniques for title and title/abstract-based search engines for relevant articles in biomedical research.Peer reviewe

Facilitators of and barriers to reducing thirty-day readmissions and improving patient-reported outcomes after surgical aortic valve replacement: a process evaluation of the AVRre trial

Abstract Background The Aortic Valve Replacement Readmission (AVRre) randomized control trial tested whether a telephone intervention would reduce hospital readmissions following surgical aortic valve replacement (SAVR). The telephone support provided 30 days of continuous phone-support (hotline) and two scheduled phone-calls from the hospital after discharge. The intervention had no effect on reducing 30-day all-cause readmission rate (30-DACR) but did reduce participants’ anxiety compared to a control group receiving usual care. Depression and participant-reported health state were unaffected by the intervention. To better understand these outcomes, we conducted a process evaluation of the AVRre trial to gain insight into the (1) the dose and fidelity of the intervention, (2) mechanism of impacts, and (3) contextual factors that may have influenced the outcomes. Methods The process evaluation was informed by the Medical Research Council framework, a widely used set of guidelines for evaluating complex interventions. A mix of quantitative (questionnaire and journal records) and qualitative data (field notes, memos, registration forms, questionnaire) was prospectively collected, and retrospective interviews were conducted. We performed descriptive analyses of the quantitative data. Content analyses, assisted by NVivo, were performed to evaluate qualitative data. Results The nurses who were serving the 24/7 hotline intervention desired to receive more preparation before intervention implementation. SAVR patient participants were highly satisfied with the telephone intervention (58%), felt safe (86%), and trusted having the option of calling in for support (91%). The support for the telephone hotline staff was perceived as a facilitator of the intervention implementation. Content analyses revealed themes: “gap in the care continuum,” “need for individualized care,” and “need for easy access to health information” after SAVR. Differences in local hospital discharge management practices influenced the 30-DACR incidence. Conclusions The prospective follow-up of the hotline service during the trial facilitated implementation of the intervention, contributing to high participant satisfaction and likely reduced their anxiety after SAVR. Perceived less-than-optimal preparations for the hotline could be a barrier to AVRre trial implementation. Integrating user experiences into a mixed-methods evaluation of clinical trials is important for broadening understanding of trial outcomes, the mechanism of impact, and contextual factors that influence clinical trials. Trial registration ClinicalTrials.gov, NCT02522663. Registered on 11 August 2015

The anatomy of safe surgical teams: an interview-based qualitative study among members of surgical teams at tertiary referral hospitals in Norway

Abstract Background In spite of the global implementation of surgical safety checklists to improve patient safety, patients undergoing surgical procedures remain vulnerable to a high risk of potentially preventable complications and adverse outcomes. The present study was designed to explore the surgical teams’ perceptions of patient safety culture, capture their perceptions of the risk for adverse events, and identify themes of interest for quality improvement within the surgical department. Methods This qualitative study had an explorative design with an abductive approach. Individual semi-structured in-depth interviews were conducted between 10/01/23 and 11/05/23. The participants were members of surgical teams (n = 17), general and orthopedic surgeons (n = 5), anesthesiologists (n = 4), nurse anesthetists (n = 4) and operating room nurses (n = 4). Middle managers recruited purposively from general and orthopedic surgical teams in two tertiary hospitals in Norway, aiming for a maximum variation due to gender, age, and years within the specialty. The data material was analyzed following Braun and Clarke’s method for reflexive thematic analysis to generate patterns of meaning and develop themes and subthemes. Results The analysis process resulted in three themes describing the participants’ perceptions of patient safety culture in the surgical context: (1) individual accountability as a safety net, (2) psychological safety as a catalyst for well-being and safe performance in the operating room, and (3) the importance of proactive structures and participation in organizational learning. Conclusions This study provided an empirical insight into the culture of patient safety in the surgical context. The study highlighted the importance of supporting the individuals’ competence, building psychological safety in the surgical team, and creating structures and culture promoting a learning organization. Quality improvement projects, including interventions based on these results, may increase patient safety culture and reduce the frequency of adverse events in the surgical context

Additional file 1 of The anatomy of safe surgical teams: an interview-based qualitative study among members of surgical teams at tertiary referral hospitals in Norway

Additional file 1. Mind Map