5,985 research outputs found
Faster title and abstract screening? Evaluating Abstrackr, a semi-automated online screening program for systematic reviewers
BACKGROUND: Citation screening is time consuming and inefficient. We sought to evaluate the performance of Abstrackr, a semi-automated online tool for predictive title and abstract screening. METHODS: Four systematic reviews (aHUS, dietary fibre, ECHO, rituximab) were used to evaluate Abstrackr. Citations from electronic searches of biomedical databases were imported into Abstrackr, and titles and abstracts were screened and included or excluded according to the entry criteria. This process was continued until Abstrackr predicted and classified the remaining unscreened citations as relevant or irrelevant. These classification predictions were checked for accuracy against the original review decisions. Sensitivity analyses were performed to assess the effects of including case reports in the aHUS dataset whilst screening and the effects of using larger imbalanced datasets with the ECHO dataset. The performance of Abstrackr was calculated according to the number of relevant studies missed, the workload saving, the false negative rate, and the precision of the algorithm to correctly predict relevant studies for inclusion, i.e. further full text inspection. RESULTS: Of the unscreened citations, Abstrackr’s prediction algorithm correctly identified all relevant citations for the rituximab and dietary fibre reviews. However, one relevant citation in both the aHUS and ECHO reviews was incorrectly predicted as not relevant. The workload saving achieved with Abstrackr varied depending on the complexity and size of the reviews (9 % rituximab, 40 % dietary fibre, 67 % aHUS, and 57 % ECHO). The proportion of citations predicted as relevant, and therefore, warranting further full text inspection (i.e. the precision of the prediction) ranged from 16 % (aHUS) to 45 % (rituximab) and was affected by the complexity of the reviews. The false negative rate ranged from 2.4 to 21.7 %. Sensitivity analysis performed on the aHUS dataset increased the precision from 16 to 25 % and increased the workload saving by 10 % but increased the number of relevant studies missed. Sensitivity analysis performed with the larger ECHO dataset increased the workload saving (80 %) but reduced the precision (6.8 %) and increased the number of missed citations. CONCLUSIONS: Semi-automated title and abstract screening with Abstrackr has the potential to save time and reduce research waste
Exploration of the methodological quality and clinical usefulness of a cross-sectional sample of published guidance about exercise training and physical activity for the secondary prevention of coronary heart disease
Background Clinicians are encouraged to use guidelines to assist in providing evidence-based secondary prevention to patients with coronary heart disease. However, the expanding number of publications providing guidance about exercise training may confuse cardiac rehabilitation clinicians. We therefore sought to explore the number, scope, publication characteristics, methodological quality, and clinical usefulness of published exercise-based cardiac rehabilitation guidance. Methods We included publications recommending physical activity, exercise or cardiac rehabilitation for patients with coronary heart disease. These included systematically developed clinical practice guidelines, as well as other publications intended to support clinician decision making, such as position papers or consensus statements. Publications were obtained via electronic searches of preventive cardiology societies, guideline databases and PubMed, to November 2016. Publication characteristics were extracted, and two independent assessors evaluated quality using the 23-item Appraisal of Guidelines Research and Evaluation II (AGREE) tool. Results Fifty-four international publications from 1994 to 2016 were identified. Most were found on preventive cardiology association websites (n = 35; 65%) and were freely accessible (n = 50; 93%). Thirty (56%) publications contained only broad recommendations for physical activity and cardiac rehabilitation referral, while 24 (44%) contained the necessary detailed exercise training recommendations. Many were labelled as “guidelines”, however publications with other titles (e.g. scientific statements) were common (n = 24; 44%). This latter group of publications contained a significantly greater proportion of detailed exercise training recommendations than clinical guidelines (p = 0.017). Wide variation in quality also existed, with ‘applicability’ the worst scoring AGREE II domain for clinical guidelines (mean score 53%) and ‘rigour of development’ rated lowest for other guidance types (mean score 33%). Conclusions While a large number of guidance documents provide recommendations for exercise-based cardiac rehabilitation, most have limitations in either methodological quality or clinical usefulness. The lack of rigorously developed guidelines which also contain necessary detail about exercise training remains a substantial problem for clinicians
Exercise training characteristics in cardiac rehabilitation programmes: A cross-sectional survey of Australian practice
Introduction Exercise training is a core component of cardiac rehabilitation (CR), however, little information exists regarding the specific exercise interventions currently provided for coronary heart disease in Australian practice. We aimed to analyse the current status of exercise-based CR services across Australia. Design Cross-sectional survey. Methods Australian sites offering exercise-based CR were identified from publically available directories. All sites were invited by email to participate in an online Survey Monkey questionnaire between October 2014 and March 2015, with reminders via email and phone follow-up. Questions investigated the demographics and format of individual programmes, as well as specific exercise training characteristics. Results 297 eligible programmes were identified, with an 82% response rate. Most sites (82%) were based at hospital or outpatient centres, with home (15%), community (18%) or gym-based options (5%) less common. While CR was most often offered in a comprehensive format (72% of sites), the level of exercise intervention varied greatly among programmes. Most frequently, exercise was prescribed 1–2 times per week for 60 min over 7 weeks. Almost one-quarter (24%) had a sole practitioner supervising exercise, although the majority used a nurse/physiotherapist combination. Low to moderate exercise intensities were used in 60% of programmes, however, higher intensity prescriptions were not uncommon. Few sites (Conclusions While advances have been made towards providing flexible and accessible exercise-based CR, much of Australia's service remains within traditional models of care. A continuing focus on service improvement and evidence-based care should, therefore, be considered a core aim of those providing exercise for CR in order to improve health service delivery and optimise outcomes for patients
Poor description of non-pharmacological interventions:Analysis of consecutive sample of randomised trials
Objectives To evaluate the completeness of descriptions of non-pharmacological interventions in randomised trials, identify which elements are most frequently missing, and assess whether authors can provide missing details. Design Analysis of consecutive sample of randomised trials of non-pharmacological interventions. Data sources and study selection All reports of randomised trials of non-pharmacological interventions published in 2009 in six leading general medical journals; 133 trial reports, with 137 interventions, met the inclusion criteria. Data collection Using an eight item checklist, two raters assessed the primary full trial report, plus any reference materials, appendices, or websites. Questions about missing details were emailed to corresponding authors, and relevant items were then reassessed. Results Of 137 interventions, only 53 (39%) were adequately described; this was increased to 81 (59%) by using 63 responses from 88 contacted authors. The most frequently missing item was the “intervention materials” (47% complete), but it also improved the most after author response (92% complete). Whereas some authors (27/70) provided materials or further information, other authors (21/70) could not; their reasons included copyright or intellectual property concerns, not having the materials or intervention details, or being unaware of their importance. Although 46 (34%) trial interventions had further information or materials readily available on a website, many were not mentioned in the report, were not freely accessible, or the URL was no longer functioning. Conclusions Missing essential information about interventions is a frequent, yet remediable, contributor to the worldwide waste in research funding. If trial reports do not have a sufficient description of interventions, other researchers cannot build on the findings, and clinicians and patients cannot reliably implement useful interventions. Improvement will require action by funders, researchers, and publishers, aided by long term repositories of materials linked to publications
Better duplicate detection for systematic reviewers: Evaluation of Systematic Review Assistant-Deduplication Module
BACKGROUND: A major problem arising from searching across bibliographic databases is the retrieval of duplicate citations. Removing such duplicates is an essential task to ensure systematic reviewers do not waste time screening the same citation multiple times. Although reference management software use algorithms to remove duplicate records, this is only partially successful and necessitates removing the remaining duplicates manually. This time-consuming task leads to wasted resources. We sought to evaluate the effectiveness of a newly developed deduplication program against EndNote. METHODS: A literature search of 1,988 citations was manually inspected and duplicate citations identified and coded to create a benchmark dataset. The Systematic Review Assistant-Deduplication Module (SRA-DM) was iteratively developed and tested using the benchmark dataset and compared with EndNote’s default one step auto-deduplication process matching on (‘author’, ‘year’, ‘title’). The accuracy of deduplication was reported by calculating the sensitivity and specificity. Further validation tests, with three additional benchmarked literature searches comprising a total of 4,563 citations were performed to determine the reliability of the SRA-DM algorithm. RESULTS: The sensitivity (84%) and specificity (100%) of the SRA-DM was superior to EndNote (sensitivity 51%, specificity 99.83%). Validation testing on three additional biomedical literature searches demonstrated that SRA-DM consistently achieved higher sensitivity than EndNote (90% vs 63%), (84% vs 73%) and (84% vs 64%). Furthermore, the specificity of SRA-DM was 100%, whereas the specificity of EndNote was imperfect (average 99.75%) with some unique records wrongly assigned as duplicates. Overall, there was a 42.86% increase in the number of duplicates records detected with SRA-DM compared with EndNote auto-deduplication. CONCLUSIONS: The Systematic Review Assistant-Deduplication Module offers users a reliable program to remove duplicate records with greater sensitivity and specificity than EndNote. This application will save researchers and information specialists time and avoid research waste. The deduplication program is freely available online
A comparison of the performance of seven key bibliographic databases in identifying all relevant systematic reviews of interventions for hypertension
BACKGROUND: Bibliographic databases are the primary resource for identifying systematic reviews of health care interventions. Reliable retrieval of systematic reviews depends on the scope of indexing used by database providers. Therefore, searching one database may be insufficient, but it is unclear how many need to be searched. We sought to evaluate the performance of seven major bibliographic databases for the identification of systematic reviews for hypertension. METHODS: We searched seven databases (Cochrane library, Database of Abstracts of Reviews of Effects (DARE), Excerpta Medica Database (EMBASE), Epistemonikos, Medical Literature Analysis and Retrieval System Online (MEDLINE), PubMed Health and Turning Research Into Practice (TRIP)) from 2003 to 2015 for systematic reviews of any intervention for hypertension. Citations retrieved were screened for relevance, coded and checked for screening consistency using a fuzzy text matching query. The performance of each database was assessed by calculating its sensitivity, precision, the number of missed reviews and the number of unique records retrieved. RESULTS: Four hundred systematic reviews were identified for inclusion from 11,381 citations retrieved from seven databases. No single database identified all the retrieved systematic reviews for hypertension. EMBASE identified the most reviews (sensitivity 69 %) but also retrieved the most irrelevant citations with 7.2 % precision (Pr). The sensitivity of the Cochrane library was 60 %, DARE 57 %, MEDLINE 57 %, PubMed Health 53 %, Epistemonikos 49 % and TRIP 33 %. EMBASE contained the highest number of unique records (n = 43). The Cochrane library identified seven unique records and had the highest precision (Pr = 30 %), followed by Epistemonikos (n = 2, Pr = 19 %). No unique records were found in PubMed Health (Pr = 24 %) DARE (Pr = 21 %), TRIP (Pr = 10 %) or MEDLINE (Pr = 10 %). Searching EMBASE and the Cochrane library identified 88 % of all systematic reviews in the reference set, and searching the freely available databases (Cochrane, Epistemonikos, MEDLINE) identified 83 % of all the reviews. The databases were re-analysed after systematic reviews of non-conventional interventions (e.g. yoga, acupuncture) were removed. Similarly, no database identified all the retrieved systematic reviews. EMBASE identified the most relevant systematic reviews (sensitivity 73 %) but also retrieved the most irrelevant citations with Pr = 5 %. The sensitivity of the Cochrane database was 62 %, followed by MEDLINE (60 %), DARE (55 %), PubMed Health (54 %), Epistemonikos (50 %) and TRIP (31 %). The precision of the Cochrane library was the highest (20 %), followed by PubMed Health (Pr = 16 %), DARE (Pr = 13 %), Epistemonikos (Pr = 12 %), MEDLINE (Pr = 6 %), TRIP (Pr = 6 %) and EMBASE (Pr = 5 %). EMBASE contained the most unique records (n = 34). The Cochrane library identified seven unique records. The other databases held no unique records. CONCLUSIONS: The coverage of bibliographic databases varies considerably due to differences in their scope and content. Researchers wishing to identify systematic reviews should not rely on one database but search multiple databases
Focus on sharing individual patient data distracts from other ways of improving trial transparency
The International Committee of Medical Journal Editors (ICMJE) recently reiterated its commitment to improving trial transparency by sharing individual patient data from randomised trials.1 2 But, although sharing individual patient data contributes to transparency, it is not sufficient by itself. Trial transparency requires a data sharing package, which begins with trial registration and contains other elements such as protocols, summary results, and other trial materials.
Valuable as sharing individual patient data can be,3 discussion about it has hijacked the broader conversation about data sharing and trial transparency.4-6 For example, we identified 76 articles published in the six leading general medical journals that had “data” and “sharing” in their title and were about clinical trials. In 64 (84%) articles, the content was focused on individual patient data and did not mention any of the other components of trial transparency (see appendix on bmj.com).
Much of the discussion has focused on the complexities and practical problems associated with sharing individual patient data and on the processes and systems needed for responsible data sharing.6-9 However, many of the data sharing activities that are needed for trial transparency are not complex. We believe that trying to solve the complex issues around availability of individual patient data should not eclipse or distract from a more pressing problem: the unavailability of even summary data and protocols from all controlled trials. Current estimates are that around 85% of research is avoidably “wasted” because of design flaws, poor conduct, non-publication, and poor reporting.10 Focusing efforts and attention on making individual patient data accessible might paradoxically exacerbate this waste in research. We argue that simpler and more cost efficient activities should be prioritised.</p
- …