SUPPORT Tools for evidence-informed health Policymaking (STP)

This article is the Introduction to a series written for people responsible for making decisions about health policies and programmes and for those who support these decision makers

Improving the use of research evidence in guideline development: 1. Guidelines for guidelines

BACKGROUND: The World Health Organization (WHO), like many other organisations around the world, has recognised the need to use more rigorous processes to ensure that health care recommendations are informed by the best available research evidence. This is the first of a series of 16 reviews that have been prepared as background for advice from the WHO Advisory Committee on Health Research to WHO on how to achieve this. OBJECTIVES: We reviewed the literature on guidelines for the development of guidelines. METHODS: We searched PubMed and three databases of methodological studies for existing systematic reviews and relevant methodological research. We did not conduct systematic reviews ourselves. Our conclusions are based on the available evidence, consideration of what WHO and other organisations are doing and logical arguments. KEY QUESTIONS AND ANSWERS: We found no experimental research that compared different formats of guidelines for guidelines or studies that compared different components of guidelines for guidelines. However, there are many examples, surveys and other observational studies that compared the impact of different guideline development documents on guideline quality. WHAT HAVE OTHER ORGANIZATIONS DONE TO DEVELOP GUIDELINES FOR GUIDELINES FROM WHICH WHO CAN LEARN? • Establish a credible, independent committee that evaluates existing methods for developing guidelines or that updates existing ones. • Obtain feedback and approval from various stakeholders during the development process of guidelines for guidelines. • Develop a detailed source document (manual) that guideline developers can use as reference material. WHAT SHOULD BE THE KEY COMPONENTS OF WHO GUIDELINES FOR GUIDELINES? • Guidelines for guidelines should include information and instructions about the following components: 1) Priority setting; 2) Group composition and consultations; 3) Declaration and avoidance of conflicts of interest; 4) Group processes; 5) Identification of important outcomes; 6) Explicit definition of the questions and eligibility criteria ; 7) Type of study designs for different questions; 8) Identification of evidence; 9) Synthesis and presentation of evidence; 10) Specification and integration of values; 11) Making judgments about desirable and undesirable effects; 12) Taking account of equity; 13) Grading evidence and recommendations; 14) Taking account of costs; 15) Adaptation, applicability, transferability of guidelines; 16) Structure of reports; 17) Methods of peer review; 18) Planned methods of dissemination & implementation; 19) Evaluation of the guidelines. WHAT HAVE OTHER ORGANIZATIONS DONE TO IMPLEMENT GUIDELINES FOR GUIDELINES FROM WHICH WHO CAN LEARN? • Obtain buy-in from regions and country level representatives for guidelines for guidelines before dissemination of a revised version. • Disseminate the guidelines for guidelines widely and make them available (e.g. on the Internet). • Develop examples of guidelines that guideline developers can use as models when applying the guidelines for guidelines. • Ensure training sessions for those responsible for developing guidelines. • Continue to monitor the methodological literature on guideline development

SUPPORT Tools for evidence-informed health Policymaking (STP) 5: Using research evidence to frame options to address a problem

This article is part of a series written for people responsible for making decisions about health policies and programmes and for those who support these decision makers

Improving the use of research evidence in guideline development: 13. Applicability, transferability and adaptation

BACKGROUND: The World Health Organization (WHO), like many other organisations around the world, has recognised the need to use more rigorous processes to ensure that health care recommendations are informed by the best available research evidence. This is the thirteenth of a series of 16 reviews that have been prepared as background for advice from the WHO Advisory Committee on Health Research to WHO on how to achieve this. OBJECTIVES: We reviewed the literature on applicability, transferability, and adaptation of guidelines. METHODS: We searched five databases for existing systematic reviews and relevant primary methodological research. We reviewed the titles of all citations and retrieved abstracts and full text articles if the citations appeared relevant to the topic. We checked the reference lists of articles relevant to the questions and used snowballing as a technique to obtain additional information. We used the definition "coming from, concerning or belonging to at least two or all nations" for the term international. Our conclusions are based on the available evidence, consideration of what WHO and other organisations are doing and logical arguments. KEY QUESTIONS AND ANSWERS: We did not identify systematic reviews addressing the key questions. We found individual studies and projects published in the peer reviewed literature and on the Internet. Should WHO develop international recommendations? • Resources for developing high quality recommendations are limited. Internationally developed recommendations can facilitate access to and pooling of resources, reduce unnecessary duplication, and involve international scientists. • Priority should be given to international health problems and problems that are important in low and middle-income countries, where these advantages are likely to be greatest. • Factors that influence the transferability of recommendations across different settings should be considered systematically and flagged, including modifying factors, important variation in needs, values, costs and the availability of resources. What should be done centrally and locally? • The preparation of systematic reviews and evidence profiles should be coordinated centrally, in collaboration with organizations that produce systematic reviews. Centrally developed evidence profiles should be adaptable to specific local circumstances. • Consideration should be given to models that involve central coordination with work being undertaken by centres located throughout the world. • While needs, availability of resources, costs, the presence of modifying factors and values need to be assessed locally, support for undertaking these assessments may be needed to make guidelines applicable. • WHO should provide local support for adapting and implementing recommendations by developing tools, building capacity, learning from international experience, and through international networks that support evidence-informed health policies, such as the Evidence-informed Policy Network (EVIPNet). How should recommendations be adapted? • WHO should provide detailed guidance for adaptation of international recommendations. • Local adaptation processes should be systematic and transparent, they should involve stakeholders, and they should report the key factors that influence decisions, including those flagged in international guidelines, and the reasons for any modifications that are made

Improving the use of research evidence in guideline development: 9. Grading evidence and recommendations

BACKGROUND: The World Health Organization (WHO), like many other organisations around the world, has recognised the need to use more rigorous processes to ensure that health care recommendations are informed by the best available research evidence. This is the ninth of a series of 16 reviews that have been prepared as background for advice from the WHO Advisory Committee on Health Research to WHO on how to achieve this. OBJECTIVES: We reviewed the literature on grading evidence and recommendations in guidelines. METHODS: We searched PubMed and three databases of methodological studies for existing systematic reviews and relevant methodological research. We did not conduct a full systematic review ourselves. Our conclusions are based on the available evidence, consideration of what WHO and other organisations are doing and logical arguments. KEY QUESTIONS AND ANSWERS: Should WHO grade the quality of evidence and the strength of recommendations? • Users of recommendations need to know how much confidence they can place in the underlying evidence and the recommendations. The degree of confidence depends on a number of factors and requires complex judgments. These judgments should be made explicitly in WHO recommendations. A systematic and explicit approach to making judgments about the quality of evidence and the strength of recommendations can help to prevent errors, facilitate critical appraisal of these judgments, and can help to improve communication of this information. What criteria should be used to grade evidence and recommendations? • Both the quality of evidence and the strength of recommendations should be graded. The criteria used to grade the strength of recommendations should include the quality of the underlying evidence, but should not be limited to that. • The approach to grading should be one that has wide international support and is suitable for a wide range of different types of recommendations. The Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach, which is currently suggested in the Guidelines for WHO Guidelines, is being used by an increasing number of other organizations internationally. It should be used more consistently by WHO. Further developments of this approach should ensure its wide applicability. Should WHO use the same grading system for all of its recommendations? • Although there are arguments for and against using the same grading system across a wide range of different types of recommendations, WHO should use a uniform grading system to prevent confusion for developers and users of recommendations

System and market failures: the unavailability of magnesium sulphate for the treatment of eclampsia and pre-eclampsia in Mozambique and Zimbabwe.

Low cost and effective drugs, such as magnesium sulphate, need to be included in initiatives to improve access to essential medicines in Afric

SUPPORT Tools for evidence-informed health Policymaking (STP) 8: Deciding how much confidence to place in a systematic review

This article is part of a series written for people responsible for making decisions about health policies and programmes and for those who support these decision makers

SUPPORT Tools for evidence-informed health Policymaking (STP) 12: Finding and using research evidence about resource use and costs

This article is part of a series written for people responsible for making decisions about health policies and programmes and for those who support these decision makers

Improving the use of research evidence in guideline development: 15. Disseminating and implementing guidelines

BACKGROUND: The World Health Organization (WHO), like many other organisations around the world, has recognised the need to use more rigorous processes to ensure that health care recommendations are informed by the best available research evidence. This is the 15(th )of a series of 16 reviews that have been prepared as background for advice from the WHO Advisory Committee on Health Research to WHO on how to achieve this. OBJECTIVES: In this review we address strategies for the implementation of recommendations in health care. METHODS: We examined overviews of systematic reviews of interventions to improve health care delivery and health care systems prepared by the Cochrane Effective Practice and Organisation of Care (EPOC) group. We also conducted searches using PubMed and three databases of methodological studies for existing systematic reviews and relevant methodological research. We did not conduct systematic reviews ourselves. Our conclusions are based on the available evidence, consideration of what WHO and other organisations are doing and logical arguments. KEY QUESTIONS AND ANSWERS: What should WHO do to disseminate and facilitate the uptake of recommendations? • WHO should choose strategies to implement their guidelines from among those which have been evaluated positively in the published literature on implementation research • Because the evidence base is weak and modest to moderate effects, at best, can be anticipated, WHO should promote rigorous evaluations of implementation strategies. What should be done at headquarters, by regional offices and in countries? • Adaptation and implementation of WHO guidelines should be done locally, at the national or sub-national level. • WHO headquarters and regional offices should support the development and evaluation of implementation strategies by local authorities

Improving the use of research evidence in guideline development: 16. Evaluation

BACKGROUND: The World Health Organization (WHO), like many other organisations around the world, has recognised the need to use more rigorous processes to ensure that health care recommendations are informed by the best available research evidence. This is the last of a series of 16 reviews that have been prepared as background for advice from the WHO Advisory Committee on Health Research to WHO on how to achieve this. OBJECTIVES: We reviewed the literature on evaluating guidelines and recommendations, including their quality, whether they are likely to be up-to-date, and their implementation. We also considered the role of guideline developers in undertaking evaluations that are needed to inform recommendations. METHODS: We searched PubMed and three databases of methodological studies for existing systematic reviews and relevant methodological research. We did not conduct systematic reviews ourselves. Our conclusions are based on the available evidence, consideration of what WHO and other organisations are doing and logical arguments. KEY QUESTIONS AND ANSWERS: Our answers to these questions were informed by a review of instruments for evaluating guidelines, several studies of the need for updating guidelines, discussions of the pros and cons of different research designs for evaluating the implementation of guidelines, and consideration of the use of uncertainties identified in systematic reviews to set research priorities. How should the quality of guidelines or recommendations be appraised? • WHO should put into place processes to ensure that both internal and external review of guidelines is undertaken routinely. • A checklist, such as the AGREE instrument, should be used. • The checklist should be adapted and tested to ensure that it is suitable to the broad range of recommendations that WHO produces, including public health and health policy recommendations, and that it includes questions about equity and other items that are particularly important for WHO guidelines. When should guidelines or recommendations be updated? • Processes should be put into place to ensure that guidelines are monitored routinely to determine if they are in need of updating. • People who are familiar with the topic, such as Cochrane review groups, should do focused, routine searches for new research that would require revision of the guideline. • Periodic review of guidelines by experts not involved in developing the guidelines should also be considered. • Consideration should be given to establishing guideline panels that are ongoing, to facilitate routine updating, with members serving fixed periods with a rotating membership. How should the impact of guidelines or recommendations be evaluated? • WHO headquarters and regional offices should support member states and those responsible for policy decisions and implementation to evaluate the impact of their decisions and actions by providing advice regarding impact assessment, practical support and coordination of efforts. • Before-after evaluations should be used cautiously and when there are important uncertainties regarding the effects of a policy or its implementation, randomised evaluations should be used when possible. What responsibility should WHO take for ensuring that important uncertainties are addressed by future research when the evidence needed to inform recommendations is lacking? • Guideline panels should routinely identify important uncertainties and research priorities. This source of potential priorities for research should be used systematically to inform priority-setting processes for global research