Improving the use of research evidence in guideline development: 1. Guidelines for guidelines

BACKGROUND: The World Health Organization (WHO), like many other organisations around the world, has recognised the need to use more rigorous processes to ensure that health care recommendations are informed by the best available research evidence. This is the first of a series of 16 reviews that have been prepared as background for advice from the WHO Advisory Committee on Health Research to WHO on how to achieve this. OBJECTIVES: We reviewed the literature on guidelines for the development of guidelines. METHODS: We searched PubMed and three databases of methodological studies for existing systematic reviews and relevant methodological research. We did not conduct systematic reviews ourselves. Our conclusions are based on the available evidence, consideration of what WHO and other organisations are doing and logical arguments. KEY QUESTIONS AND ANSWERS: We found no experimental research that compared different formats of guidelines for guidelines or studies that compared different components of guidelines for guidelines. However, there are many examples, surveys and other observational studies that compared the impact of different guideline development documents on guideline quality. WHAT HAVE OTHER ORGANIZATIONS DONE TO DEVELOP GUIDELINES FOR GUIDELINES FROM WHICH WHO CAN LEARN? • Establish a credible, independent committee that evaluates existing methods for developing guidelines or that updates existing ones. • Obtain feedback and approval from various stakeholders during the development process of guidelines for guidelines. • Develop a detailed source document (manual) that guideline developers can use as reference material. WHAT SHOULD BE THE KEY COMPONENTS OF WHO GUIDELINES FOR GUIDELINES? • Guidelines for guidelines should include information and instructions about the following components: 1) Priority setting; 2) Group composition and consultations; 3) Declaration and avoidance of conflicts of interest; 4) Group processes; 5) Identification of important outcomes; 6) Explicit definition of the questions and eligibility criteria ; 7) Type of study designs for different questions; 8) Identification of evidence; 9) Synthesis and presentation of evidence; 10) Specification and integration of values; 11) Making judgments about desirable and undesirable effects; 12) Taking account of equity; 13) Grading evidence and recommendations; 14) Taking account of costs; 15) Adaptation, applicability, transferability of guidelines; 16) Structure of reports; 17) Methods of peer review; 18) Planned methods of dissemination & implementation; 19) Evaluation of the guidelines. WHAT HAVE OTHER ORGANIZATIONS DONE TO IMPLEMENT GUIDELINES FOR GUIDELINES FROM WHICH WHO CAN LEARN? • Obtain buy-in from regions and country level representatives for guidelines for guidelines before dissemination of a revised version. • Disseminate the guidelines for guidelines widely and make them available (e.g. on the Internet). • Develop examples of guidelines that guideline developers can use as models when applying the guidelines for guidelines. • Ensure training sessions for those responsible for developing guidelines. • Continue to monitor the methodological literature on guideline development

Improving the use of research evidence in guideline development: introduction

In 2005 the World Health Organisation (WHO) asked its Advisory Committee on Health Research (ACHR) for advice on ways in which WHO can improve the use of research evidence in the development of recommendations, including guidelines and policies. The ACHR established the Subcommittee on the Use of Research Evidence (SURE) to collect background documentation and consult widely among WHO staff, international experts and end users of WHO recommendations to inform its advice to WHO. We have prepared a series of reviews of methods that are used in the development of guidelines as part of this background documentation. We describe here the background and methods of these reviews, which are being published in Health Research Policy and Systems together with this introduction

Improving the use of research evidence in guideline development: 13. Applicability, transferability and adaptation

BACKGROUND: The World Health Organization (WHO), like many other organisations around the world, has recognised the need to use more rigorous processes to ensure that health care recommendations are informed by the best available research evidence. This is the thirteenth of a series of 16 reviews that have been prepared as background for advice from the WHO Advisory Committee on Health Research to WHO on how to achieve this. OBJECTIVES: We reviewed the literature on applicability, transferability, and adaptation of guidelines. METHODS: We searched five databases for existing systematic reviews and relevant primary methodological research. We reviewed the titles of all citations and retrieved abstracts and full text articles if the citations appeared relevant to the topic. We checked the reference lists of articles relevant to the questions and used snowballing as a technique to obtain additional information. We used the definition "coming from, concerning or belonging to at least two or all nations" for the term international. Our conclusions are based on the available evidence, consideration of what WHO and other organisations are doing and logical arguments. KEY QUESTIONS AND ANSWERS: We did not identify systematic reviews addressing the key questions. We found individual studies and projects published in the peer reviewed literature and on the Internet. Should WHO develop international recommendations? • Resources for developing high quality recommendations are limited. Internationally developed recommendations can facilitate access to and pooling of resources, reduce unnecessary duplication, and involve international scientists. • Priority should be given to international health problems and problems that are important in low and middle-income countries, where these advantages are likely to be greatest. • Factors that influence the transferability of recommendations across different settings should be considered systematically and flagged, including modifying factors, important variation in needs, values, costs and the availability of resources. What should be done centrally and locally? • The preparation of systematic reviews and evidence profiles should be coordinated centrally, in collaboration with organizations that produce systematic reviews. Centrally developed evidence profiles should be adaptable to specific local circumstances. • Consideration should be given to models that involve central coordination with work being undertaken by centres located throughout the world. • While needs, availability of resources, costs, the presence of modifying factors and values need to be assessed locally, support for undertaking these assessments may be needed to make guidelines applicable. • WHO should provide local support for adapting and implementing recommendations by developing tools, building capacity, learning from international experience, and through international networks that support evidence-informed health policies, such as the Evidence-informed Policy Network (EVIPNet). How should recommendations be adapted? • WHO should provide detailed guidance for adaptation of international recommendations. • Local adaptation processes should be systematic and transparent, they should involve stakeholders, and they should report the key factors that influence decisions, including those flagged in international guidelines, and the reasons for any modifications that are made

Improving the use of research evidence in guideline development: 9. Grading evidence and recommendations

BACKGROUND: The World Health Organization (WHO), like many other organisations around the world, has recognised the need to use more rigorous processes to ensure that health care recommendations are informed by the best available research evidence. This is the ninth of a series of 16 reviews that have been prepared as background for advice from the WHO Advisory Committee on Health Research to WHO on how to achieve this. OBJECTIVES: We reviewed the literature on grading evidence and recommendations in guidelines. METHODS: We searched PubMed and three databases of methodological studies for existing systematic reviews and relevant methodological research. We did not conduct a full systematic review ourselves. Our conclusions are based on the available evidence, consideration of what WHO and other organisations are doing and logical arguments. KEY QUESTIONS AND ANSWERS: Should WHO grade the quality of evidence and the strength of recommendations? • Users of recommendations need to know how much confidence they can place in the underlying evidence and the recommendations. The degree of confidence depends on a number of factors and requires complex judgments. These judgments should be made explicitly in WHO recommendations. A systematic and explicit approach to making judgments about the quality of evidence and the strength of recommendations can help to prevent errors, facilitate critical appraisal of these judgments, and can help to improve communication of this information. What criteria should be used to grade evidence and recommendations? • Both the quality of evidence and the strength of recommendations should be graded. The criteria used to grade the strength of recommendations should include the quality of the underlying evidence, but should not be limited to that. • The approach to grading should be one that has wide international support and is suitable for a wide range of different types of recommendations. The Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach, which is currently suggested in the Guidelines for WHO Guidelines, is being used by an increasing number of other organizations internationally. It should be used more consistently by WHO. Further developments of this approach should ensure its wide applicability. Should WHO use the same grading system for all of its recommendations? • Although there are arguments for and against using the same grading system across a wide range of different types of recommendations, WHO should use a uniform grading system to prevent confusion for developers and users of recommendations

Global access to affordable direct oral anticoagulants

Poor control of cardiovascular disease accounts for a substantial proportion of the disease burden in developing countries, but often essential anticoagulant medicines for preventing strokes and embolisms are not widely available. In 2019, direct oral anticoagulants were added to the World Health Organization's WHO Model list of essential medicines. The aims of this paper are to summarize the benefits of direct oral anticoagulants for patients with cardiovascular disease and to discuss ways of increasing their usage internationally. Although the cost of direct oral anticoagulants has provoked debate, the affordability of introducing these drugs into clinical practice could be increased by: price negotiation; pooled procurement; competitive tendering; the use of patent pools; and expanded use of generics. In 2017, only 14 of 137 countries that had adopted national essential medicines lists included a direct oral anticoagulant on their lists. This number could increase rapidly if problems with availability and affordability can be tackled. Once the types of patient likely to benefit from direct oral anticoagulants have been clearly defined in clinical practice guidelines, coverage can be more accurately determined and associated costs can be better managed. Government action is required to ensure that direct oral anticoagulants are covered by national budgets because the absence of reimbursement remains an impediment to achieving universal coverage. Tackling cardiovascular disease with the aid of direct oral anticoagulants is an essential component of efforts to achieve the World Health Organization's target of reducing premature deaths due to noncommunicable disease by 25% by 2025

Interpreting the results of patient reported outcome measures in clinical trials: The clinician's perspective

This article deals with the problem of interpreting health-related quality of life (HRQL) outcomes in clinical trials. First, we will briefly describe how dichotomization and item response theory can facilitate interpretation. Based on examples from the medical literature for the interpretation of HRQL scores we will show that dichotomies may help clinicians understand information provided by HRQL instruments in RCTs. They can choose thresholds to calculate proportions of patients benefiting based on absolute scores or change scores. For example, clinicians interpreting clinical trial results could consider the difference in the proportion of patients who achieve a mean score of 50 before and after an intervention on a scale from 1 to 100. For the change score approach, they could consider the proportion of patients who have changed by a score of 5 or more. Finally, they can calculate the proportion of patients benefiting and transform these numbers into a number needed to treat or natural frequencies. Second, we will describe in more detail an approach to the interpretation of HRQL scores based on the minimal important difference (MID) and proportions. The MID is the smallest difference in score in the outcome of interest that informed patients or informed proxies perceive as important, either beneficial or harmful, and that would lead the patient or clinician to consider a change in the management. Any change in management will depend on the downsides, including cost and inconvenience, associated with the intervention. Investigators can help with the interpretation of HRQL scores by determining the MID of an HRQL instrument and provide mean differences in relation to the MID. For instance, for an MID of 0.5 on a seven point scale investigators could provide the mean change on the instrument as well as the proportion of patients with scores greater than the MID. Thus, there are several steps investigators can take to facilitate this process to help bringing HRQL information closer to the bedside

Beverage specific alcohol intake in a population-based study: Evidence for a positive association between pulmonary function and wine intake

BACKGROUND: Lung function is a strong predictor of cardiovascular and all-cause mortality. Previous studies suggest that alcohol exposure may be linked to impaired pulmonary function through oxidant-antioxidant mechanisms. Alcohol may be an important source of oxidants; however, wine contains several antioxidants. In this study we analyzed the relation of beverage specific alcohol intake with forced expiratory volume in one second (FEV(1)) and forced vital capacity (FVC) in a random sample of 1555 residents of Western New York, USA. METHODS: We expressed pulmonary function as percent of predicted normal FEV(1) (FEV(1)%) and FVC (FVC%) after adjustment for height, age, gender and race. To obtain information on alcohol intake we used a questionnaire that reliably queries total alcohol and beverage specific recent (past 30 days) and lifetime alcohol consumption. Results: Using multiple linear regression analysis after adjustment for covariates (pack-years of smoking, weight, smoking status, education, nutritional factors and for FEV(1)%, in addition, eosinophil count), we observed no significant correlation between total alcohol intake and lung function. However, we found positive associations of recent and lifetime wine intake with FEV(1)% and FVC%. When we analyzed white and red wine intake separately, the association of lung function with red wine was weaker than for white wine. CONCLUSION: While total alcohol intake was not related to lung function, wine intake showed a positive association with lung function. Although we cannot exclude residual confounding by healthier lifestyle in wine drinkers, differential effects of alcoholic beverages on lung health may exist

Improving the use of research evidence in guideline development: 14. Reporting guidelines

BACKGROUND: The World Health Organization (WHO), like many other organisations around the world, has recognised the need to use more rigorous processes to ensure that health care recommendations are informed by the best available research evidence. This is the 14(th )of a series of 16 reviews that have been prepared as background for advice from the WHO Advisory Committee on Health Research to WHO on how to achieve this. OBJECTIVES: We reviewed the literature on reporting guidelines and recommendations. METHODS: We searched PubMed and three databases of methodological studies for existing systematic reviews and relevant methodological research. We did not conduct systematic reviews ourselves. Our conclusions are based on the available evidence, consideration of what WHO and other organisations are doing and logical arguments. KEY QUESTIONS AND ANSWERS: There is little empirical evidence that addresses these questions. Our answers are based on logical arguments and standards put forward by other groups. What standard types of recommendations or reports should WHO use? • WHO should develop standard formats for reporting recommendations to facilitate recognition and use by decision makers for whom the recommendations are intended, and to ensure that all the information needed to judge the quality of a guideline, determine its applicability and, if needed, adapt it, is reported. • WHO should develop standard formats for full systematically developed guidelines that are sponsored by WHO, rapid assessments, and guidelines that are endorsed by WHO. • All three formats should include the same information as full guidelines, indicating explicitly what the group preparing the guideline did not do, as well as the methods that were used. • These formats should be used across clinical, public health and health systems recommendations. How should recommendations be formulated and reported? • Reports should be structured, using headings that correspond to those suggested by the Conference on Guideline Standardization or similar headings. • The quality of evidence and strength of recommendations should be reported explicitly using a standard approach. • The way in which recommendations are formulated should be adapted to the specific characteristics of a specific guideline. • Urgent attention should be given to developing a template that provides decision makers with the relevant global evidence that is needed to inform a decision and offers practical methods for incorporating the context specific evidence and judgements that are needed

Living systematic reviews: 4. Living guideline recommendations

While it is important for the evidence supporting practice guidelines to be current, that is often not the case. The advent of living systematic reviews has made the concept of "living guidelines" realistic, with the promise to provide timely, up-to-date and high-quality guidance to target users. We define living guidelines as an optimization of the guideline development process to allow updating individual recommendations as soon as new relevant evidence becomes available. A major implication of that definition is that the unit of update is the individual recommendation and not the whole guideline. We then discuss when living guidelines are appropriate, the workflows required to support them, the collaboration between living systematic reviews and living guideline teams, the thresholds for changing recommendations, and potential approaches to publication and dissemination. The success and sustainability of the concept of living guideline will depend on those of its major pillar, the living systematic review. We conclude that guideline developers should both experiment with and research the process of living guidelines

Application of GRADE: Making evidence-based recommendations about diagnostic tests in clinical practice guidelines

<p>Abstract</p> <p>Background</p> <p>Accurate diagnosis is a fundamental aspect of appropriate healthcare. However, clinicians need guidance when implementing diagnostic tests given the number of tests available and resource constraints in healthcare. Practitioners of health often feel compelled to implement recommendations in guidelines, including recommendations about the use of diagnostic tests. However, the understanding about diagnostic tests by guideline panels and the methodology for developing recommendations is far from completely explored. Therefore, we evaluated the factors that guideline developers and users need to consider for the development of implementable recommendations about diagnostic tests.</p> <p>Methods</p> <p>Using a critical analysis of the process, we present the results of a case study using the Grading of Recommendations Applicability, Development and Evaluation (GRADE) approach to develop a clinical practice guideline for the diagnosis of Cow Milk Allergy with the World Allergy Organization.</p> <p>Results</p> <p>To ensure that guideline panels can develop informed recommendations about diagnostic tests, it appears that more emphasis needs to be placed on group processes, including question formulation, defining patient-important outcomes for diagnostic tests, and summarizing evidence. Explicit consideration of concepts of diagnosis from evidence-based medicine, such as pre-test probability and treatment threshold, is required to facilitate the work of a guideline panel and to formulate implementable recommendations.</p> <p>Discussion</p> <p>This case study provides useful guidance for guideline developers and clinicians about what they ought to demand from clinical practice guidelines to facilitate implementation and strengthen confidence in recommendations about diagnostic tests. Applying a structured framework like the GRADE approach with its requirement for transparency in the description of the evidence and factors that influence recommendations facilitates laying out the process and decision factors that are required for the development, interpretation, and implementation of recommendations about diagnostic tests.</p