Systems for grading the quality of evidence and the strength of recommendations I: Critical appraisal of existing approaches The GRADE Working Group

BACKGROUND: A number of approaches have been used to grade levels of evidence and the strength of recommendations. The use of many different approaches detracts from one of the main reasons for having explicit approaches: to concisely characterise and communicate this information so that it can easily be understood and thereby help people make well-informed decisions. Our objective was to critically appraise six prominent systems for grading levels of evidence and the strength of recommendations as a basis for agreeing on characteristics of a common, sensible approach to grading levels of evidence and the strength of recommendations. METHODS: Six prominent systems for grading levels of evidence and strength of recommendations were selected and someone familiar with each system prepared a description of each of these. Twelve assessors independently evaluated each system based on twelve criteria to assess the sensibility of the different approaches. Systems used by 51 organisations were compared with these six approaches. RESULTS: There was poor agreement about the sensibility of the six systems. Only one of the systems was suitable for all four types of questions we considered (effectiveness, harm, diagnosis and prognosis). None of the systems was considered usable for all of the target groups we considered (professionals, patients and policy makers). The raters found low reproducibility of judgements made using all six systems. Systems used by 51 organisations that sponsor clinical practice guidelines included a number of minor variations of the six systems that we critically appraised. CONCLUSIONS: All of the currently used approaches to grading levels of evidence and the strength of recommendations have important shortcomings

Transparent Development of the WHO Rapid Advice Guidelines

Emerging health problems require rapid advice. We describe the development and pilot testing of a systematic, transparent approach used by the World Health Organization (WHO) to develop rapid advice guidelines in response to requests from member states confronted with uncertainty about the pharmacological management of avian influenza A (H5N1) virus infection. We first searched for systematic reviews of randomized trials of treatment and prevention of seasonal influenza and for non-trial evidence on H5N1 infection, including case reports and animal and in vitro studies. A panel of clinical experts, clinicians with experience in treating patients with H5N1, influenza researchers, and methodologists was convened for a two-day meeting. Panel members reviewed the evidence prior to the meeting and agreed on the process. It took one month to put together a team to prepare the evidence profiles (i.e., summaries of the evidence on important clinical and policy questions), and it took the team only five weeks to prepare and revise the evidence profiles and to prepare draft guidelines prior to the panel meeting. A draft manuscript for publication was prepared within 10 days following the panel meeting. Strengths of the process include its transparency and the short amount of time used to prepare these WHO guidelines. The process could be improved by shortening the time required to commission evidence profiles. Further development is needed to facilitate stakeholder involvement, and evaluate and ensure the guideline's usefulness

Effect of interventions to reduce potentially inappropriate use of drugs in nursing homes: a systematic review of randomised controlled trials

Background Studies have shown that residents in nursing homes often are exposed to inappropriate medication. Particular concern has been raised about the consumption of psychoactive drugs, which are commonly prescribed for nursing home residents suffering from dementia. This review is an update of a Norwegian systematic review commissioned by the Norwegian Directorate of Health. The purpose of the review was to identify and summarise the effect of interventions aimed at reducing potentially inappropriate use or prescribing of drugs in nursing homes. Methods We searched for systematic reviews and randomised controlled trials in the Cochrane Library, MEDLINE, EMBASE, ISI Web of Knowledge, DARE and HTA, with the last update in April 2010. Two of the authors independently screened titles and abstracts for inclusion or exclusion. Data on interventions, participants, comparison intervention, and outcomes were extracted from the included studies. Risk of bias and quality of evidence were assessed using the Cochrane Risk of Bias Table and GRADE, respectively. Outcomes assessed were use of or prescribing of drugs (primary) and the health-related outcomes falls, physical limitation, hospitalisation and mortality (secondary). Results Due to heterogeneity in interventions and outcomes, we employed a narrative approach. Twenty randomised controlled trials were included from 1631 evaluated references. Ten studies tested different kinds of educational interventions while seven studies tested medication reviews by pharmacists. Only one study was found for each of the interventions geriatric care teams, early psychiatric intervening or activities for the residents combined with education of health care personnel. Several reviews were identified, but these either concerned elderly in general or did not satisfy all the requirements for systematic reviews. Conclusions Interventions using educational outreach, on-site education given alone or as part of an intervention package and pharmacist medication review may under certain circumstances reduce inappropriate drug use, but the evidence is of low quality. Due to poor quality of the evidence, no conclusions may be drawn about the effect of the other three interventions on drug use, or of either intervention on health-related outcomes

Guidance for Evidence-Informed Policies about Health Systems: Assessing How Much Confidence to Place in the Research Evidence

In the third paper in a three-part series on health systems guidance, Simon Lewin and colleagues explore the challenge of assessing how much confidence to place in evidence on health systems interventions

Emergency cricothyrotomy – a systematic review

BACKGROUND: An emergency cricothyrotomy is the last-resort in most airway management protocols and is performed when it is not possible to intubate or ventilate a patient. This situation can rapidly prove fatal, making it important to identify the best method to establish a secure airway. We conducted a systematic review to identify whether there exists superiority between available commercial kits versus traditional surgical and needle techniques. METHODS: Medline, EMBASE and other databases were searched for pertinent studies. The inclusion criteria included manikin, animal and human studies and there were no restrictions regarding the professional background of the person performing the procedure. RESULTS: In total, 1,405 unique references were identified; 108 full text articles were retrieved; and 24 studies were included in the review. Studies comparing kits with one another or with various surgical and needle techniques were identified. The outcome measures included in this systematic review were success rate and time consumption. The investigators performing the studies had chosen unique combinations of starting and stopping points for time measurements, making comparisons between studies difficult and leading to many conflicting results. No single method was shown to be better than the others, but the size of the studies makes it impossible to draw firm conclusions. CONCLUSIONS: The large majority of the studies were too small to demonstrate statistically significant differences, and the limited available evidence was of low or very low quality. That none of the techniques in these studies demonstrated better results than the others does not necessarily indicate that each is equally good, and these conclusions will likely change as new evidence becomes available

Effect of interventions to reduce potentially inappropriate use of drugs in nursing homes: a systematic review of randomised controlled trials

Abstract Background Studies have shown that residents in nursing homes often are exposed to inappropriate medication. Particular concern has been raised about the consumption of psychoactive drugs, which are commonly prescribed for nursing home residents suffering from dementia. This review is an update of a Norwegian systematic review commissioned by the Norwegian Directorate of Health. The purpose of the review was to identify and summarise the effect of interventions aimed at reducing potentially inappropriate use or prescribing of drugs in nursing homes. Methods We searched for systematic reviews and randomised controlled trials in the Cochrane Library, MEDLINE, EMBASE, ISI Web of Knowledge, DARE and HTA, with the last update in April 2010. Two of the authors independently screened titles and abstracts for inclusion or exclusion. Data on interventions, participants, comparison intervention, and outcomes were extracted from the included studies. Risk of bias and quality of evidence were assessed using the Cochrane Risk of Bias Table and GRADE, respectively. Outcomes assessed were use of or prescribing of drugs (primary) and the health-related outcomes falls, physical limitation, hospitalisation and mortality (secondary). Results Due to heterogeneity in interventions and outcomes, we employed a narrative approach. Twenty randomised controlled trials were included from 1631 evaluated references. Ten studies tested different kinds of educational interventions while seven studies tested medication reviews by pharmacists. Only one study was found for each of the interventions geriatric care teams, early psychiatric intervening or activities for the residents combined with education of health care personnel. Several reviews were identified, but these either concerned elderly in general or did not satisfy all the requirements for systematic reviews. Conclusions Interventions using educational outreach, on-site education given alone or as part of an intervention package and pharmacist medication review may under certain circumstances reduce inappropriate drug use, but the evidence is of low quality. Due to poor quality of the evidence, no conclusions may be drawn about the effect of the other three interventions on drug use, or of either intervention on health-related outcomes.</p

Is the supine position associated with loss of airway patency in unconscious trauma patients? A systematic review and meta-analysis

Background Airway compromise is a leading cause of death in unconscious trauma patients. Although endotracheal intubation is regarded as the gold standard treatment, most prehospital providers are not trained to perform ETI in such patients. Therefore, various lateral positions are advocated for unconscious patients, but their use remains controversial in trauma patients. We conducted a systematic review to investigate whether the supine position is associated with loss of airway patency compared to the lateral position. Methods The review protocol was published in the PROSPERO database (Reg. no. CRD42012001190). We performed literature searches in PubMed, Medline, EMBASE, Cochrane Library, CINAHL and British Nursing Index and included studies related to airway patency, reduced level of consciousness and patient position. We conducted meta-analyses, where appropriate. We graded the quality of evidence with the GRADE methodology. The search was updated in June 2014. Results We identified 1,306 publications, 39 of which were included for further analysis. Sixteen of these publications were included in meta-analysis. We did not identify any studies reporting direct outcome measures (mortality or morbidity) related to airway compromise caused by the patient position (lateral vs. supine position) in trauma patients or in any other patient group. In studies reporting only indirect outcome measures, we found moderate evidence of reduced airway patency in the supine vs. the lateral position, which was measured by the apnea/hypopnea index (AHI). For other indirect outcomes, we only found low or very low quality evidence. Conclusions Although concerns other than airway patency may influence how a trauma patient is positioned, our systematic review provides evidence supporting the long held recommendation that unconscious trauma patients should be placed in a lateral position