Key Terms and Definitions in Acute Porphyrias: Results of an International Delphi Consensus Led by the European Porphyria Network

Background: Acute porphyrias are a group of rare inherited disorders causing acute neurovisceral attacks. Many terms used frequently in the literature and in clinical practice are ambiguous, which can lead to confusion in the way patients are managed, studied, and reported in clinical studies. Agreed definitions are a necessary first step in developing management guidelines and will facilitate communication of results of future clinical research. Methods: The Delphi method was used to generate consensus on key terms and definitions in acute porphyria. The process started with a brainstorming phase offered to all members of the European Porphyria Network followed by 2 Delphi rounds among international experts in the field of porphyria (the Acute Porphyria Expert Panel). A consensus of 75% or more was defined as the agreement threshold. Results: 63 respondents from 26 countries participated in the brainstorming phase, leading to the choice of 9 terms and definitions. 34 experts were invited to take part in the Delphi rounds. 7 of the initial 9 terms and definitions which entered the first Delphi round achieved the threshold for agreement. Following a second Delphi round, all 9 definitions achieved agreement. Conclusion: Agreement on the definitions for 9 important terms describing acute porphyrias represents a significant step forward for the porphyria community. It will facilitate more accurate comparison of outcomes among porphyria centres and in clinical trials and provide a strong framework for developing evidence based clinical guidelines

GRADE Evidence to Decision (EtD) frameworks : A systematic and transparent approach to making well-informed healthcare choices. 1. Introduction

Funding: Work on this article has been partially funded by the European Commission FP7 Program (grant agreement 258583) as part of the DECIDE project. Sole responsibility lies with the authors; the European Commission is not responsible for any use that may be made of the information contained therein.Peer reviewedPublisher PD

GRADE Guidelines: 18. How ROBINS-I and other tools to assess risk of bias in nonrandomized studies should be used to rate the certainty of a body of evidence

To provide guidance on how systematic review authors, guideline developers, and health technology assessment practitioners should approach the use of the risk of bias in non-randomized studies of interventions (ROBINS-I) tool as part of GRADE's certainty rating process. Iterative discussions, testing in systematic reviews, presentation at GRADE working group meetings with feedback from the GRADE Working Group. We describe where to start the initial assessment of a body of evidence with the use of ROBINS-I, and where one would anticipate the final rating would end up. GRADE accounted for issues that mitigate concerns about confounding and selection bias by introducing the upgrading domains: large effects, dose-effect relations, and when plausible residual confounders or other biases increase certainty. They will need to be considered in an assessment of a body of evidence when using ROBINS-I. The use of ROBINS-I in GRADE assessments may allow for a better comparison of evidence from RCTs and NRS because they are placed on a common metric for risk of bias. Challenges remain that include appropriate presentation of evidence from RCTs and NRS for decision-making and how to optimally integrate RCTs and NRS in an evidence assessment

Protocol: Barriers and facilitators to stakeholder engagement in health guideline development: a qualitative evidence synthesis

Background There is a need for the development of comprehensive, global, evidence-based guidance for stakeholder engagement in guideline development. Stakeholders are any individual or group who is responsible for or affected by health- and healthcare-related decisions. This includes patients, the public, providers of health care and policymakers for example. As part of the guidance development process, Multi-Stakeholder Engagement (MuSE) Consortium set out to conduct four concurrent systematic reviews to summarise the evidence on: (1) existing guidance for stakeholder engagement in guideline development, (2) barriers and facilitators to stakeholder engagement in guideline development, (3) managing conflicts of interest in stakeholder engagement in guideline development and (4) measuring the impact of stakeholder engagement in guideline development. This protocol addresses the second systematic review in the series. Objectives The objective of this review is to identify and synthesise the existing evidence on barriers and facilitators to stakeholder engagement in health guideline development. We will address this objective through two research questions: (1) What are the barriers to multi-stakeholder engagement in health guideline development across any of the 18 steps of the GIN-McMaster checklist? (2) What are the facilitators to multi-stakeholder engagement in health guideline development across any of the 18 steps of the GIN-McMaster checklist? Search Methods A comprehensive search strategy will be developed and peer-reviewed in consultation with a medical librarian. We will search the following databases: MEDLINE, Cumulative Index to Nursing & Allied Health Literature (CINAHL), EMBASE, PsycInfo, Scopus, and Sociological Abstracts. To identify grey literature, we will search the websites of agencies who actively engage stakeholder groups such as the AHRQ, Canadian Institutes of Health Research (CIHR) Strategy for Patient-Oriented Research (SPOR), INVOLVE, the National Institute for Health and Care Excellence (NICE) and the PCORI. We will also search the websites of guideline-producing agencies, such as the American Academy of Pediatrics, Australia's National Health Medical Research Council (NHMRC) and the WHO. We will invite members of the team to suggest grey literature sources and we plan to broaden the search by soliciting suggestions via social media, such as Twitter. Selection Criteria We will include empirical qualitative and mixed-method primary research studies which qualitatively report on the barriers or facilitators to stakeholder engagement in health guideline development. The population of interest is stakeholders in health guideline development. Building on previous work, we have identified 13 types of stakeholders whose input can enhance the relevance and uptake of guidelines: Patients, caregivers and patient advocates; Public; Providers of health care; Payers of health services; Payers of research; Policy makers; Program managers; Product makers; Purchasers; Principal investigators and their research teams; and Peer-review editors/publishers. Eligible studies must describe stakeholder engagement at any of the following steps of the GIN-McMaster Checklist for Guideline Development. Data Collection and Analysis All identified citations from electronic databases will be imported into Covidence software for screening and selection. Documents identified through our grey literature search will be managed and screened using an Excel spreadsheet. A two-part study selection process will be used for all identified citations: (1) a title and abstract review and (2) full-text review. At each stage, teams of two review authors will independently assess all potential studies in duplicate using a priori inclusion and exclusion criteria. Data will be extracted by two review authors independently and in duplicate according to a standardised data extraction form. Main Results The results of this review will be used to inform the development of guidance for multi-stakeholder engagement in guideline development and implementation. This guidance will be official GRADE (Grading of Recommendations Assessment, Development and Evaluation) Working Group guidance. The GRADE system is internationally recognised as a standard for guideline development. The findings of this review will assist organisations who develop healthcare, public health and health policy guidelines, such as the World Health Organization, to involve multiple stakeholders in the guideline development process to ensure the development of relevant, high quality and transparent guidelines

GRADE guidelines 17:Assessing the risk of bias associated with missing participant outcome data in a body of evidence

Objective: To provide GRADE guidance for assessing risk of bias across an entire body of evidence consequent on missing data for systematic reviews of both binary and continuous outcomes. Study Design and Setting: Systematic survey of published methodological research, iterative discussions, testing in systematic reviews, and feedback from the GRADE Working Group. Results: Approaches begin with a primary meta-analysis using a complete case analysis followed by sensitivity meta-analyses imputing, in each study, data for those with missing data, and then pooling across studies. For binary outcomes, we suggest use of "plausible worst case" in which review authors assume that those with missing data in treatment arms have proportionally higher event rates than those followed successfully. For continuous outcomes, imputed mean values come from other studies within the systematic review and the standard deviation (SD) from the median SDs of the control arms of all studies. Conclusions: If the results of the primary meta-analysis are robust to the most extreme assumptions viewed as plausible, one does not rate down certainty in the evidence for risk of bias due to missing participant outcome data. If the results prove not robust to plausible assumptions, one would rate down certainty in the evidence for risk of bias.</p

Systematic survey of randomized trials evaluating the impact of alternative diagnostic strategies on patient-important outcomes

Objectives: To provide a perspective on the current practice of randomized clinical trials (RCTs) of diagnostic strategies focusing on patient-important outcomes. Study Design and Setting: We conducted a comprehensive search of MEDLINE and included RCTs published in full-text reports that evaluated alternative diagnostic strategies. Results: Of 56,912 unique citations, we sampled 7,500 and included 103 eligible RCTs, therefore suggesting that MEDLINE includes approximately 781 diagnostic RCTs. The 103 eligible trials reported on: mortality (n = 41; 39.8%); morbidities (n = 63; 61.2%); symptoms/quality of life/functional status (n = 14; 13.6%); and on composite end points (n = 10; 9.7%). Of the studies that reported statistically significant results (n = 12; 11.6%), we judged 7 (58.3%) as at low risk of bias with respect to missing outcome data and 4 (33.3%) as at low risk of bias regarding blinding. Of the 41 RCTs that reported on mortality, only one (2.4%) reported statistically significant results. Of 63 RCTs addressing morbidity outcomes, 11 (17.5%) reported statistically significant results, all of which reported relative effects of greater than 20%. Conclusion: RCTs of diagnostic tests are not uncommon, and sometimes suggest benefits on patient-important outcomes but often suffer from limitations in sample size and conduct. (C) 2017 Elsevier Inc. All rights reserved.Peer reviewe

Guidance was developed on how to write a plain language summary for diagnostic test accuracy reviews

Objective: To develop guidance for authors of diagnostic test accuracy (DTA) reviews to help them write a plain language summary of the results of their review. Study Design and Setting: We used a combination of focus groups, user testing, and a web-based survey. Participants included patient representatives, media representatives, and health professionals. Results: We present step-by-step guidance for authors of DTA reviews for writing a plain language summary. This guidance is illustrated with examples of reader-tested sentences, explanations, and a figure. Conclusion: We hope this guidance will allow reviewers to present the findings of DTA reviews so that it is easier for readers to understand the results and conclusions. This will increase the accessibility of these reviews for various audiences

Protocol for the development of guidance for stakeholder engagement in health and healthcare guideline development and implementation

Stakeholder engagement has become widely accepted as a necessary component of guideline development and implementation. While frameworks for developing guidelines express the need for those potentially affected by guideline recommendations to be involved in their development, there is a lack of consensus on how this should be done in practice. Further, there is a lack of guidance on how to equitably and meaningfully engage multiple stakeholders. We aim to develop guidance for the meaningful and equitable engagement of multiple stakeholders in guideline development and implementation. METHODS: This will be a multi-stage project. The first stage is to conduct a series of four systematic reviews. These will (1) describe existing guidance and methods for stakeholder engagement in guideline development and implementation, (2) characterize barriers and facilitators to stakeholder engagement in guideline development and implementation, (3) explore the impact of stakeholder engagement on guideline development and implementation, and (4) identify issues related to conflicts of interest when engaging multiple stakeholders in guideline development and implementation. DISCUSSION: We will collaborate with our multiple and diverse stakeholders to develop guidance for multi-stakeholder engagement in guideline development and implementation. We will use the results of the systematic reviews to develop a candidate list of draft guidance recommendations and will seek broad feedback on the draft guidance via an online survey of guideline developers and external stakeholders. An invited group of representatives from all stakeholder groups will discuss the results of the survey at a consensus meeting which will inform the development of the final guidance papers. Our overall goal is to improve the development of guidelines through meaningful and equitable multi-stakeholder engagement, and subsequently to improve health outcomes and reduce inequities in health

The GRADE Working Group clarifies the construct of certainty of evidence

The authors would like to thank colleagues at the Swedish Agency for Health Technology Assessment and Assessment of Social Services (SBU) who, through participating in a series of seminars, contributed with valuable input on the initial draft of the presented approaches. The authors also thank all GRADE Working Group members who have contributed to the paper during group discussions at Grade Working Group meetings. The Health Services Research Unit, University of Aberdeen, receives core funding from the Chief Scientist Office of the Scottish Government Health Directorates. The Parker Institute, Bispebjerg and Frederiksberg Hospital is supported by a core grant from the Oak Foundation (OCAY-13-309). SVK is funded by a NRS Scottish Senior Clinical Fellowship (SCAF/15/02), the Medical Research Council (MC_UU_12017/13 & MC_UU_12017/15) and Chief Scientist’s Office (SPHSU13 & SPHSU15).Peer reviewedPublisher PD

GRADE guidelines: 29. Rating the certainty in time-to-event outcomes – Study limitations due to censoring of participants with missing data in intervention studies

Objectives: To provide Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) guidance for the consideration of study limitations (risk of bias) due to missing participant outcome data for time-to-event outcomes in intervention studies. Study Design and Setting: We developed this guidance through an iterative process that included membership consultation, feedback, presentation, and iterative discussion at meetings of the GRADE working group. Results: The GRADE working group has published guidance on how to account for missing participant outcome data in binary and continuous outcomes. When analyzing time-to-event outcomes (e.g., overall survival and time-to-treatment failure) data of participants for whom the outcome of interest (e.g., death and relapse) has not been observed are dealt with through censoring. To do so, standard methods require that censored individuals are representative for those remaining in the study. Two types of censoring can be distinguished, end of study censoring and censoring because of missing data, commonly named loss to follow-up censoring. However, both types are not distinguishable with the usual information on censoring available to review authors. Dealing with individuals for whom data are missing during follow-up in the same way as individuals for whom full follow-up is available at the end of the study increases the risk of bias. Considerable differences in the treatment arms in the distribution of censoring over time (early versus late censoring), the overall degree of missing follow-up data, and the reasons why individuals were lost to follow-up may reduce the certainty in the study results. With often only very limited data available, review and guideline authors are required to make transparent and well-considered judgments when judging risk of bias of individual studies and then come to an overall grading decision for the entire body of evidence. Conclusion: Concern for risk of bias resulting from censoring of participants for whom follow-up data are missing in the underlying studies of a body of evidence can be expressed in the study limitations (risk of bias) domain of the GRADE approach. (C) 2020 Elsevier Inc. All rights reserved