Large expert-curated database for benchmarking document similarity detection in biomedical literature search

Document recommendation systems for locating relevant literature have mostly relied on methods developed a decade ago. This is largely due to the lack of a large offline gold-standard benchmark of relevant documents that cover a variety of research fields such that newly developed literature search techniques can be compared, improved and translated into practice. To overcome this bottleneck, we have established the RElevant LIterature SearcH consortium consisting of more than 1500 scientists from 84 countries, who have collectively annotated the relevance of over 180 000 PubMed-listed articles with regard to their respective seed (input) article/s. The majority of annotations were contributed by highly experienced, original authors of the seed articles. The collected data cover 76% of all unique PubMed Medical Subject Headings descriptors. No systematic biases were observed across different experience levels, research fields or time spent on annotations. More importantly, annotations of the same document pairs contributed by different scientists were highly concordant. We further show that the three representative baseline methods used to generate recommended articles for evaluation (Okapi Best Matching 25, Term Frequency-Inverse Document Frequency and PubMed Related Articles) had similar overall performances. Additionally, we found that these methods each tend to produce distinct collections of recommended articles, suggesting that a hybrid method may be required to completely capture all relevant articles. The established database server located at https://relishdb.ict.griffith.edu.au is freely available for the downloading of annotation data and the blind testing of new methods. We expect that this benchmark will be useful for stimulating the development of new powerful techniques for title and title/abstract-based search engines for relevant articles in biomedical research.Peer reviewe

Early discharge compared with ordinary discharge after percutaneous coronary intervention - a systematic review and meta-analysis of safety and cost

Aim: We aimed to summarize the pooled effect of early discharge compared with ordinary discharge after percutaneous coronary intervention (PCI) on the composite endpoint of re-infarction, revascularization, stroke, death, and incidence of rehospitalization. We also aimed to compare costs for the two strategies. Methods: The study was a systematic review and a meta-analysis of 12 randomized controlled trials including 2962 patients, followed by trial sequential analysis. An estimation of cost was considered. Follow-up time was 30 days. Results: For early discharge, pooled effect for the composite endpoint was relative risk of efficacy (RRe)=0.65, 95% confidence interval (CI) (0.52–0.81). Rehospitalization had a pooled effect of RRe=1.10, 95% CI (0.88–1.38). Early discharge had an increasing risk of rehospitalization with increasing frequency of hypertension for all populations, except those with stable angina, where a decreasing risk was noted. Advancing age gave increased risk of revascularization. Early discharge had a cost reduction of 655 Euros per patient compared with ordinary discharge. Conclusion: The pooled effect supports the safe use of early discharge after PCI in the treatment of a heterogeneous population of patients with coronary artery disease. There was an increased risk of rehospitalization for all subpopulations, except patients with stable angina. Clinical trials with homogeneous populations of acute coronary syndrome are needed to be conclusive on this issue

Double reading of current chest CT examinations: Clinical importance of changes to radiology reports

Objectives Misinterpretation of radiological examinations is an important contributing factor to diagnostic errors. Double reading reduces interpretation errors and increases sensitivity. Consultant radiologists in Norwegian hospitals submit 39% of computed tomography (CT) reports for quality assurance by double reading. Our objective was to estimate the proportion of radiology reports that were changed during double reading and to assess the potential clinical impact of these changes. Materials and methods In this retrospective cross-sectional study we acquired preliminary and final reports from 1023 consecutive double read chest CT examinations conducted at five public hospitals. The preliminary and final reports were compared for changes in content. Three experienced pulmonologists independently rated the clinical importance of these changes. The severity of the radiological findings in clinically important changes was classified as increased, unchanged, or decreased. Results Changes were classified as clinically important in 91 (9%) of 1023 reports. Of these: 3 were critical (demanding immediate action), 15 were major (implying a change in treatment) and 73 were intermediate (affecting subsequent investigations). More clinically important changes were made to urgent examinations and less to female first readers. Chest radiologist made more clinically important changes than other second readers. The severity of the radiological findings was increased in 73 (80%) of the clinically important changes. Conclusion A 9% rate of clinically important changes made during double reading may justify quality assurance of radiological interpretation. Using expert second readers and targeting a selection of urgent cases prospectively may increase the yield of discrepant cases and reduce harm to patients

Radiologist-initiated double reading of abdominal CT: retrospective analysis of the clinical importance of changes to radiology reports

Background Misinterpretation of radiological examinations is an important contributing factor to diagnostic errors. Consultant radiologists in Norwegian hospitals frequently request second reads by colleagues in real time. Our objective was to estimate the frequency of clinically important changes to radiology reports produced by these prospectively obtained double readings. Methods We retrospectively compared the preliminary and final reports from 1071 consecutive double-read abdominal CT examinations of surgical patients at five public hospitals in Norway. Experienced gastrointestinal surgeons rated the clinical importance of changes from the preliminary to final report. The severity of the radiological findings in clinically important changes was classified as increased, unchanged or decreased. Results Changes were classified as clinically important in 146 of 1071 reports (14%). Changes to 3 reports (0.3%) were critical (demanding immediate action), 35 (3%) were major (implying a change in treatment) and 108 (10%) were intermediate (requiring further investigations). The severity of the radiological findings was increased in 118 (81%) of the clinically important changes. Important changes were made less frequently when abdominal radiologists were first readers, more frequently when they were second readers, and more frequently to urgent examinations. Conclusion A 14% rate of clinically important changes made during double reading may justify quality assurance of radiological interpretation. Using expert second readers and a targeted selection of urgent cases and radiologists reading outside their specialty may increase the yield of discrepant cases

Abstracts of the 6th FECS Conference 1998 Lectures

International audienc