Identifying and addressing conflicting results across multiple discordant systematic reviews on the same question: protocol for a replication study of the Jadad algorithm

Introduction An increasing growth of systematic reviews (SRs) presents notable challenges for decision-makers seeking to answer clinical questions. In 1997, an algorithm was created by Jadad to assess discordance in results across SRs on the same question. Our study aims to (1) replicate assessments done in a sample of studies using the Jadad algorithm to determine if the same SR would have been chosen, (2) evaluate the Jadad algorithm in terms of utility, efficiency and comprehensiveness, and (3) describe how authors address discordance in results across multiple SRs. Methods and analysis We will use a database of 1218 overviews (2000-2020) created from a bibliometric study as the basis of our search for studies assessing discordance (called discordant reviews). This bibliometric study searched MEDLINE (Ovid), Epistemonikos and Cochrane Database of Systematic Reviews for overviews. We will include any study using Jadad (1997) or another method to assess discordance. The first 30 studies screened at the full-text stage by two independent reviewers will be included. We will replicate the authors' Jadad assessments. We will compare our outcomes qualitatively and evaluate the differences between our Jadad assessment of discordance and the authors' assessment. Ethics and dissemination No ethics approval was required as no human subjects were involved. In addition to publishing in an open-access journal, we will disseminate evidence summaries through formal and informal conferences, academic websites, and across social media platforms. This is the first study to comprehensively evaluate and replicate Jadad algorithm assessments of discordance across multiple SRs

Assessing the format and content of journal published and non-journal published rapid review reports : A comparative study

BACKGROUND: As production of rapid reviews (RRs) increases in healthcare, knowing how to efficiently convey RR evidence to various end-users is important given they are often intended to directly inform decision-making. Little is known about how often RRs are produced in the published or unpublished domains, and what and how information is structured. OBJECTIVES: To compare and contrast report format and content features of journal-published (JP) and non-journal published (NJP) RRs. METHODS: JP RRs were identified from key databases, and NJP RRs were identified from a grey literature search of 148 RR producing organizations and were sampled proportionate to cluster size by organization and product type to match the JP RR group. We extracted and formally compared 'how' (i.e., visual arrangement) and 'what' information was presented. RESULTS: We identified 103 RRs (52 JP and 51 NJP) from 2016. A higher percentage of certain features were observed in JP RRs compared to NJP RRs (e.g., reporting authors; use of a traditional journal article structure; section headers including abstract, methods, discussion, conclusions, acknowledgments, conflict of interests, and author contributions; and use of figures (e.g., Study Flow Diagram) in the main document). For NJP RRs, a higher percentage of features were observed (e.g., use non-traditional report structures; bannering of executive summary sections and appendices; use of typographic cues; and including outcome tables). NJP RRs were more than double in length versus JP RRs. Including key messages was uncommon in both groups. CONCLUSIONS: This comparative study highlights differences between JP and NJP RRs. Both groups may benefit from better use of plain language, and more clear and concise design. Alternative innovative formats and end-user preferences for content and layout should be studied further with thought given to other considerations to ensure better packaging of RR results to facilitate uptake into policy and practice. STUDY REGISTRATION: The full protocol is available at: https://osf.io/29xvk/

Developing a digital intervention for cancer survivors: an evidence-, theory- and person-based approach

This paper illustrates a rigorous approach to developing digital interventions using an evidence-, theory- and person-based approach. Intervention planning included a rapid scoping review which identified cancer survivors’ needs, including barriers and facilitators to intervention success. Review evidence (N=49 papers) informed the intervention’s Guiding Principles, theory-based behavioural analysis and logic model. The intervention was optimised based on feedback on a prototype intervention through interviews (N=96) with cancer survivors and focus groups with NHS staff and cancer charity workers (N=31). Interviews with cancer survivors highlighted barriers to engagement, such as concerns about physical activity worsening fatigue. Focus groups highlighted concerns about support appointment length and how to support distressed participants. Feedback informed intervention modifications, to maximise acceptability, feasibility and likelihood of behaviour change. Our systematic method for understanding user views enabled us to anticipate and address important barriers to engagement. This methodology may be useful to others developing digital interventions

Identifying models of delivery, care domains and quality indicators relevant to palliative day services: a scoping review protoco

Abstract Background With an ageing population and increasing numbers of people with life-limiting illness, there is a growing demand for palliative day services. There is a need to measure and demonstrate the quality of these services, but there is currently little agreement on which aspects of care should be used to do this. The aim of the scoping review will be to map the extent, range and nature of the evidence around models of delivery, care domains and existing quality indicators used to evaluate palliative day services. Methods Electronic databases (MEDLINE, EMBASE, CINAHL, PsycINFO, Cochrane Central Register of Controlled Trials) will be searched for evidence using consensus development methods; randomised or quasi-randomised controlled trials; mixed methods; and prospective, longitudinal or retrospective case-control studies to develop or test quality indicators for evaluating palliative care within non-residential settings, including day hospices and community or primary care settings. At least two researchers will independently conduct all searches, study selection and data abstraction procedures. Meta-analyses and statistical methods of synthesis are not planned as part of the review. Results will be reported using numerical counts, including number of indicators in each care domain and by using qualitative approach to describe important indicator characteristics. A conceptual model will also be developed to summarise the impact of different aspects of quality in a palliative day service context. Methodological quality relating to indicator development will be assessed using the Appraisal of Indicators through Research and Evaluation (AIRE) tool. Overall strength of evidence will be assessed using the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) system. Final decisions on quality assessment will be made via consensus between review authors. Discussion Identifying, developing and implementing evidence-based quality indicators is critical to the evaluation and continued improvement of palliative care. Review findings will be used to support clinicians and policymakers make decisions on which quality indicators are most appropriate for evaluating day services at the patient and service level, and to identify areas for further research

PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews

The methods and results of systematic reviews should be reported in sufficient detail to allow users to assess the trustworthiness and applicability of the review findings. The Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement was developed to facilitate transparent and complete reporting of systematic reviews and has been updated (to PRISMA 2020) to reflect recent advances in systematic review methodology and terminology. Here, we present the explanation and elaboration paper for PRISMA 2020, where we explain why reporting of each item is recommended, present bullet points that detail the reporting recommendations, and present examples from published reviews. We hope that changes to the content and structure of PRISMA 2020 will facilitate uptake of the guideline and lead to more transparent, complete, and accurate reporting of systematic reviews

Interpreting systematic reviews: are we ready to make our own conclusions? A cross-sectional study

<p>Abstract</p> <p>Background</p> <p>Independent evaluation of clinical evidence is advocated in evidence-based medicine (EBM). However, authors' conclusions are often appealing for readers who look for quick messages. We assessed how well a group of Malaysian hospital practitioners and medical students derived their own conclusions from systematic reviews (SRs) and to what extent these were influenced by their prior beliefs and the direction of the study results.</p> <p>Methods</p> <p>We conducted two cross-sectional studies: one with hospital practitioners (<it>n </it>= 150) attending an EBM course in June 2008 in a tertiary hospital and one with final-year medical students (<it>n </it>= 35) in November 2008. We showed our participants four Cochrane SR abstracts without the authors' conclusions. For each article, the participants chose a conclusion from among six options comprising different combinations of the direction of effect and the strength of the evidence. We predetermined the single option that best reflected the actual authors' conclusions and labelled this as our best conclusion. We compared the participants' choices with our predetermined best conclusions. Two chosen reviews demonstrated that the intervention was beneficial ("positive"), and two others did not ("negative"). We also asked the participants their prior beliefs about the intervention.</p> <p>Results</p> <p>Overall, 60.3% correctly identified the direction of effect, and 30.1% chose the best conclusions, having identified both the direction of effect and the strength of evidence. More students (48.2%) than practitioners (22.2%) chose the best conclusions (<it>P </it>< 0.001). Fewer than one-half (47%) correctly identified the direction of effect against their prior beliefs. "Positive" SRs were more likely than "negative" SRs to change the participants' beliefs about the effect of the intervention (relative risk (RR) 1.8, 95% confidence interval 1.3 to 2.6) and "convert" those who were previously unsure by making them choose the appropriate direction of effect (RR 1.9, 95% confidence interval 1.3 to 2.8).</p> <p>Conclusions</p> <p>The majority of our participants could not generate appropriate conclusions from SRs independently. Judicious direction from the authors' conclusions still appears crucial to guiding our health care practitioners in identifying appropriate messages from research. Authors, editors and reviewers should ensure that the conclusions of a paper accurately reflect the results. Similar studies should be conducted in other settings where awareness and application of EBM are different.</p> <p>Please see Commentary: <url>http://www.biomedcentral.com/1741-7015/9/31/</url>.</p

Evaluation of the clinical and cost effectiveness of intermediate care clinics for diabetes (ICCD): A multicentre cluster randomised controlled trial

Background Configuring high quality care for the rapidly increasing number of people with type 2 diabetes (T2D) is a major challenge worldwide for both providers and commissioners. In the UK, about two thirds of people with T2D are managed entirely in primary care, with wide variation in management strategies and achievement of targets. Pay for performance, introduced in 2004, initially resulted in improvements but disparities exist in ethnic minorities and the improvements are levelling off. Community based, intermediate care clinics for diabetes (ICCDs) were considered one solution and are functioning across the UK. However, there is no randomised trial evidence for the effectiveness of such clinics. Trial Design, Methods and Findings This is a cluster-randomised trial, involving 3 primary care trusts, with 49 general practices randomised to usual care (n = 25) or intervention (ICCDs; n = 24). All eligible adult patients with T2D were invited; 1997 were recruited and 1280 followed-up after 18-months intervention. Primary outcome: achievement of all three of the NICE targets [(HbA1c≤7.0%/53 mmol/mol; Blood Pressure <140/80 mmHg; cholesterol <154 mg/dl (4 mmol/l)]. Primary outcome was achieved in 14.3% in the intervention arm vs. 9.3% in the control arm (p = 0.059 after adjustment for covariates). The odds ratio (95% CI) for achieving primary outcome in the intervention group was 1.56 (0.98, 2.49). Primary care and community clinic costs were significantly higher in the intervention group, but there were no significant differences in hospital costs or overall healthcare costs. An incremental cost-effectiveness ratio (ICER) of +£7,778 per QALY gained, indicated ICCD was marginally more expensive at producing health gain. Conclusions Intermediate care clinics can contribute to improving target achievement in patients with diabetes. Further work is needed to investigate the optimal scale and organisational structure of ICCD services and whether, over time, their role may change as skill levels in primary care increase. Trial Registration ClinicalTrials.gov NCT00945204; National Research Register (NRR) M0014178167.National Institute of Health Research Project number: SDO/110/2005. Initial service for support costs was provided by Department of health.NHS Leicester City, Thames Valley Diabetes Research Network (TVDRN), West Midlands South Comprehensive Local Research Network (CLRN), Primary Care Research Network (PCRN) and DIERT charity provided additional support for the successful completion of the study