When and how to update systematic reviews

Governments, funding agencies, academic institutions, and health care policy makers are increasingly investing in the design, development, and dissemination of systematic reviews (SRs) to inform clinical practice guidelines, ethical guidance of clinical research, and health care practice and policy. SRs need to be sensitive to the dynamic nature of new evidence, such as published papers. The emergence of new evidence over time may undermine the validity of conclusions and recommendations in any given SR and subsequent practice guideline. This issue has only started to be more seriously considered during the last decade or so. Now it is clear that the use of out-dated evidence can lead to a waste of resources, provision of redundant, ineffective or even harmful health care. The author of this dissertation and his colleagues conducted and published three empirical studies and two conceptual articles (in six peer-reviewed journal publications), which addressed the methodologic aspects of when and how to update SRs. This PhD project provides a summary of these publications. The work described herein has had a significant impact on raising awareness and initiating new research efforts for keeping SRs up-to-date. Publication 1 proposed the first formal definition of what constitutes an update of a SR. The article presented distinguishing features of an updated vs. not updated or a new review. Publication 2 (or Publication 3) systematically reviewed methods, techniques, and strategies describing when and how to update SRs (Study #1). Publication 4, an international survey (Study #2), identified and described updating practices and policies of organisations involved in the production and commission of SRs. Publication 5 reviewed the knowledge and efforts in updating SRs and provided guidance for authors and SR groups as to when and how to update comparative effectiveness reviews produced by the Agency for Healthcare Research and Quality’s (AHRQ) Evidence-based Practice Centres (EPCs) throughout North America. Publication 6 (Study #3) described the development, piloting, and feasibility of a surveillance system to assess the need for updating comparative effectiveness reviews produced by the AHRQ’s EPC Program. This surveillance method has proved to be an efficient approach for prioritising SRs with respect to updating need

Community-onset sepsis and its public health burden : protocol of a systematic review

Background: Sepsis is a life-threatening condition and major contributor of public health and economic burden in the industrialised world. The heterogeneity, absence of more specific definition, and difficulties in accurate diagnosis lead to great variability in the estimates of sepsis incidence. There has been uncertainty regarding the incidence and risk factors attributable to community-onset as opposed to hospital-acquired sepsis. Community-onset sepsis has distinct host characteristics, risk factors, pathogens, and prognosis. A systematic assessment of recent evidence is warranted in light of secular changes in epidemiology, pathogens, and the uncertainties around the incidence and risk factors of community-onset sepsis. This protocol describes a systematic review which aims to synthesise the recent empirical evidence on the incidence and risk factors of community-onset sepsis, severe sepsis, and septic shock in high-income countries. Methods/design: English-language publications of cohort and case-control studies reporting incidence and risk factors of community-onset sepsis will be eligible for inclusion. MEDLINE and Embase databases will be searched from 2002 and onwards. References of relevant publications will be hand-searched. Two reviewers will independently screen titles/abstracts and full texts as well as extract data and appraise the risk of bias of included studies. The data extractions and risk of bias assessments will be cross-checked. Any disagreements will be resolved via consensus. The data on incidence and risk factors of sepsis will be organised and synthesised in text, tables, and forest plots. The evidence will be pooled given sufficient data and degree of similarity across study populations, exposures, and outcomes. The heterogeneity will be assessed through visual inspection of forest plots, Chi-square-based p value, and I (2) statistic. The sources of heterogeneity will be explored via subgroup analysis. Discussion: Timeliness and accuracy of diagnosis of sepsis are both crucial aspects for improving the patient's outcome. The findings of this review will be discussed with a view to better inform future recommendations on improving public-facing campaigns, timely presentation, and diagnosis of sepsis in the community. The review will also discuss gaps in evidence and highlight future research and policy-making avenues for improving public health planning. Systematic review registration: PROSPERO CRD42015023484

A systematic review of economic models used to assess the cost-effectiveness of strategies for identifying latent tuberculosis in high-risk groups

Timely diagnosis and treatment of latent tuberculosis infection (LTBI) through screening remains a key public health priority. Although globally it is recommended to screen people at high risk of developing TB, the economic evidence underpinning these recommendations is limited. This review critically appraised studies that had used a decision-analytical modelling framework to estimate the cost-effectiveness of interferon gamma release assays (IGRAs) compared to tuberculin skin test (TST) for detecting LTBI in high risk populations. A systematic review of economic models used to assess the cost-effectiveness of strategies for identifying latent tuberculosis in high-risk groups (PDF Download Available). Available from: https://www.researchgate.net/publication/301759613_A_systematic_review_of_economic_models_used_to_assess_the_cost-effectiveness_of_strategies_for_identifying_latent_tuberculosis_in_high-risk_groups [accessed May 27, 2016]

Total hip replacement for the treatment of end stage arthritis of the hip : a systematic review and meta-analysis

Background: Evolvements in the design, fixation methods, size, and bearing surface of implants for total hip replacement (THR) have led to a variety of options for healthcare professionals to consider. The need to determine the most optimal combinations of THR implant is warranted. This systematic review evaluated the clinical effectiveness of different types of THR used for the treatment of end stage arthritis of the hip. Methods: A comprehensive literature search was undertaken in major health databases. Randomised controlled trials (RCTs) and systematic reviews published from 2008 onwards comparing different types of primary THR in patients with end stage arthritis of the hip were included. Results: Fourteen RCTs and five systematic reviews were included. Patients experienced significant post-THR improvements in Harris Hip scores, but this did not differ between impact types. There was a reduced risk of implant dislocation after receiving a larger femoral head size (36 mm vs. 28 mm; RR = 0.17, 95% CI: 0.04, 0.78) or cemented cup (vs. cementless cup; pooled odds ratio: 0.34, 95% CI: 0.13, 0.89). Recipients of cross-linked vs. conventional polyethylene cup liners experienced reduced femoral head penetration and revision. There was no impact of femoral stem fixation and cup shell design on implant survival rates. Evidence on mortality and complications (aseptic loosening, femoral fracture) was inconclusive. Conclusions: The majority of evidence was inconclusive due to poor reporting, missing data, or uncertainty in treatment estimates. The findings warrant cautious interpretation given the risk of bias (blinding, attrition), methodological limitations (small sample size, low event counts, short follow-up), and poor reporting. Long-term pragmatic RCTs are needed to allow for more definitive conclusions. Authors are encouraged to specify the minimal clinically important difference and power calculation for their primary outcome(s) as well CONSORT, PRISMA and STROBE guidelines to ensure better reporting and more reliable production and assessment of evidence

Cost-Effectiveness of Manual Therapy for the Management of Musculoskeletal Conditions: A Systematic Review and Narrative Synthesis of Evidence From Randomized Controlled Trials

AbstractObjectivesThe purpose of this study was to systematically review trial-based economic evaluations of manual therapy relative to other alternative interventions used for the management of musculoskeletal conditions.MethodsA comprehensive literature search was undertaken in major medical, health-related, science and health economic electronic databases.ResultsTwenty-five publications were included (11 trial-based economic evaluations). The studies compared cost-effectiveness and/or cost-utility of manual therapy interventions to other treatment alternatives in reducing pain (spinal, shoulder, ankle). Manual therapy techniques (eg, osteopathic spinal manipulation, physiotherapy manipulation and mobilization techniques, and chiropractic manipulation with or without other treatments) were more cost-effective than usual general practitioner (GP) care alone or with exercise, spinal stabilization, GP advice, advice to remain active, or brief pain management for improving low back and shoulder pain/disability. Chiropractic manipulation was found to be less costly and more effective than alternative treatment compared with either physiotherapy or GP care in improving neck pain.ConclusionsPreliminary evidence from this review shows some economic advantage of manual therapy relative to other interventions used for the management of musculoskeletal conditions, indicating that some manual therapy techniques may be more cost-effective than usual GP care, spinal stabilization, GP advice, advice to remain active, or brief pain management for improving low back and shoulder pain/disability. However, at present, there is a paucity of evidence on the cost-effectiveness and/or cost-utility evaluations for manual therapy interventions. Further improvements in the methodological conduct and reporting quality of economic evaluations of manual therapy are warranted in order to facilitate adequate evidence-based decisions among policy makers, health care practitioners, and patients

Extending the diabetic retinopathy screening interval beyond 1 year : systematic review

To determine whether the recommended screening interval for diabetic retinopathy (DR) in the UK can safely be extended beyond 1 year. Systematic review of clinical and cost-effectiveness studies. Nine databases were searched with no date restrictions. Randomised controlled trials (RCTs), cohort studies, prognostic or economic modelling studies which described the incidence and progression of DR in populations with type 1 diabetes mellitus or type 2 diabetes mellitus of either sex and of any age reporting incidence and progression of DR in relation to screening interval (vs annual screening interval) and/or prognostic factors were included. Narrative synthesis was undertaken. 14 013 papers were identified, of which 11 observational studies, 5 risk stratification modelling studies and 9 economic studies were included. Data were available for 262 541 patients of whom at least 228 649 (87%) had type 2 diabetes. There were no RCTs. Studies concluded that there is little difference between clinical outcomes from screening 1 yearly or 2 yearly in low-risk patients. However there was high loss to follow-up (13–31%), heterogeneity in definitions of low risk and variation in screening and grading protocols for prior retinopathy results. Observational and economic modelling studies in low-risk patients show little difference in clinical outcomes between 1-year and 2-year screening intervals. The lack of experimental research designs and heterogeneity in definition of low risk considerably limits the reliability and validity of this conclusion. Cost-effectiveness findings were mixed. There is insufficient evidence to recommend a move to extend the screening interval beyond 1 year

Community-onset sepsis and its public health burden : a systematic review

Background: Sepsis is a life-threatening condition and major contributor to public health and economic burden in the industrialised world. The difficulties in accurate diagnosis lead to great variability in estimates of sepsis incidence. There has been even a greater uncertainty regarding the incidence of and risk factors for community-onset sepsis (COS). We systematically reviewed the recent evidence on the incidence and risk factors of COS in high income countries. Methods: Cohort and case-control studies were eligible for inclusion. Medline and Embase databases were searched from 2002 onwards. References of relevant publications were hand searched. Two reviewers screened titles/abstracts and full-texts independently. One reviewer extracted data and appraised studies which were cross-checked by independent reviewers. Disagreements were resolved via consensus. Odds ratios (ORs) and 95 percent confidence intervals (95% CIs) were ascertained by type of sepsis (non-severe, severe, and septic shock). Results: 10 cohort and 4 case-control studies were included. There was a wide variation in the incidence (# cases per 100,000 per year) of non-severe sepsis (range: 64 - 514), severe sepsis (range: 40 - 455), and septic shock (range: 9 - 31). Heterogeneity precluded statistical pooling. Two cohort and 4 case-control studies reported risk factors for sepsis. In one casecontrol and one cohort study, older age and diabetes were associated with increased risk of sepsis. The same case-control study showed an excess risk for sepsis in participants with clinical conditions (e.g., immunosuppression, lung disease, and peripheral artery disease). In one cohort study, higher risk of sepsis was associated with being a nursing home resident (OR=2.60, 95% CI: 1.20, 5.60) and in the other cohort study with being physically inactive (OR=1.33, 95% CI: 1.13, 1.56) and smoking tobacco (OR=1.85, 95% CI: 1.54, 2.22). The evidence on sex, ethnicity, statin use, and body mass index as risk factors was inconclusive. Conclusions: The lack of a valid standard approach for defining sepsis makes it difficult to determine the true incidence of COS. Differences in case ascertainment contribute to the variation in incidence of COS. The evidence on COS is limited in terms of the number and quality of studies. This review highlights the urgent need for an accurate and standard method for identifying sepsis. Future studies need to improve the methodological shortcomings of previous research in terms of case definition, identification, and surveillance practice

Updating Systematic Reviews: An International Survey

BACKGROUND: Systematic reviews (SRs) should be up to date to maintain their importance in informing healthcare policy and practice. However, little guidance is available about when and how to update SRs. Moreover, the updating policies and practices of organizations that commission or produce SRs are unclear. METHODOLOGY/PRINCIPAL FINDINGS: The objective was to describe the updating practices and policies of agencies that sponsor or conduct SRs. An Internet-based survey was administered to a purposive non-random sample of 195 healthcare organizations within the international SR community. Survey results were analyzed using descriptive statistics. The completed response rate was 58% (n = 114) from across 26 countries with 70% (75/107) of participants identified as producers of SRs. Among responders, 79% (84/107) characterized the importance of updating as high or very-high and 57% (60/106) of organizations reported to have a formal policy for updating. However, only 29% (35/106) of organizations made reference to a written policy document. Several groups (62/105; 59%) reported updating practices as irregular, and over half (53/103) of organizational respondents estimated that more than 50% of their respective SRs were likely out of date. Authors of the original SR (42/106; 40%) were most often deemed responsible for ensuring SRs were current. Barriers to updating included resource constraints, reviewer motivation, lack of academic credit, and limited publishing formats. Most respondents (70/100; 70%) indicated that they supported centralization of updating efforts across institutions or agencies. Furthermore, 84% (83/99) of respondents indicated they favoured the development of a central registry of SRs, analogous to efforts within the clinical trials community. CONCLUSIONS/SIGNIFICANCE: Most organizations that sponsor and/or carry out SRs consider updating important. Despite this recognition, updating practices are not regular, and many organizations lack a formal written policy for updating SRs. This research marks the first baseline data available on updating from an organizational perspective

Agreement between gastrointestinal panel testing and standard microbiology methods for detecting pathogens in suspected infectious gastroenteritis : test evaluation and meta-analysis in the absence of a reference standard

Objective: Multiplex gastrointestinal pathogen panel (GPP) tests simultaneously identify bacterial, viral and parasitic pathogens from the stool samples of patients with suspected infectious gastroenteritis presenting in hospital or the community. We undertook a systematic review to compare the accuracy of GPP tests with standard microbiology techniques. Review methods: Searches in Medline, Embase, Web of Science and the Cochrane library were undertaken from inception to January 2016. Eligible studies compared GPP tests with standard microbiology techniques in patients with suspected gastroenteritis. Quality assessment of included studies used tailored QUADAS-2. In the absence of a reference standard we analysed test performance taking GPP tests and standard microbiology techniques in turn as the benchmark test, using random effects meta-analysis of proportions. Results: No study provided an adequate reference standard with which to compare the test accuracy of GPP and conventional tests. Ten studies informed a meta-analysis of positive and negative agreement. Positive agreement across all pathogens was 0.93 (95% CI 0.90 to 0.96) when conventional methods were the benchmark and 0.68 (95% CI: 0.58 to 0.77) when GPP provided the benchmark. Negative agreement was high in both instances due to the high proportion of negative cases. GPP testing produced a greater number of pathogen-positive findings than conventional testing. It is unclear whether these additional ‘positives’ are clinically important. Conclusions: GPP testing has the potential to simplify testing and accelerate reporting when compared to conventional microbiology methods. However the impact of GPP testing upon the management, treatment and outcome of patients is poorly understood and further studies are needed to evaluate the health economic impact of GPP testing compared with standard methods

Ixazomib for relapsed or refractory multiple myeloma : review from an evidence review group on a NICE single technology appraisal

Ixazomib is an oral proteasome inhibitor used in combination with lenalidomide plus dexamethasone (IXA-LEN-DEX) and licensed for relapsed or refractory multiple myeloma. As part of a single technology appraisal (ID807) undertaken by the National Institute of Health and Care Excellence, the Evidence Review Group, Warwick Evidence was invited to independently review the evidence submitted by the manufacturer of ixazomib, Takeda UK Ltd. The main source of clinical effectiveness data about IXA-LEN-DEX came from the Tourmaline-MM1 randomized controlled trial in which 771 patients with relapsed or refractory multiple myeloma received either IXA-LEN-DEX or placebo-LEN-DEX as their second-, third-, or fourth-line treatment. Takeda estimated the cost effectiveness of IXA-LEN-DEX using a de-novo partitioned-survival model with three health states (pre-progression, post-progression, and dead). In their first submission, this model was used to estimate the cost effectiveness of IXA-LEN-DEX vs. bortezomib plus dexamethasone (BORT-DEX) in second-line treatment, and of IXA-LEN-DEX vs. LEN-DEX in third-line treatment. To estimate the relative clinical performance of IXA-LEN-DEX vs. BORT-DEX, Takeda conducted network meta-analyses for important outcomes. The network meta-analysis for overall survival was found to be flawed in several respects, but mainly because a hazard ratio input for one of the studies in the network had been inverted, resulting in a large inflation of the claimed superiority of IXA-LEN-DEX over BORT-DEX and a considerable overestimation of its cost effectiveness. In subsequent submissions, Takeda withdrew second-line treatment as an option for IXA-LEN-DEX. The manufacturer’s first submission comparing IXA-LEN-DEX with LEN-DEX for third-line therapy employed Tourmaline-MM1 data from third- and fourth-line patients as proxy for a third-line population. The appraisal committee did not consider this reasonable because randomization in Tourmaline-MM1 was stratified according to one previous treatment and two or more previous treatments. A further deficiency was considered to be the manufacturer’s use of interim survival data rather than the most mature data available. A second submission from the company focussed on IXA-LEN-DEX vs. LEN-DEX as third- or fourth-line treatment (the two or more previous lines population) and a new patient access scheme was introduced. Covariate modeling of survival outcomes was proposed using the most mature survival data. The Evidence Review Group’s main criticisms of the new evidence included: the utility associated with the pre-progression health state was overestimated, treatment costs of ixazomib were underestimated, survival models were still associated with great uncertainty, leading to clinically implausible anomalies and highly variable incremental cost-effectiveness ratio estimates, and the company had not explored a strong assumption that the survival benefit of IXA-LEN-DEX over LEN-DEX would be fully maintained for a further 22 years beyond the observed data, which encompassed only approximately 2.5 years of observation. The appraisal committee remained unconvinced that ixazomib represented a cost-effective use of National Health Service resources. Takeda’s third submission offered new base-case parametric models for survival outcomes, a new analysis of utilities, and proposed a commercial access agreement. In a brief critique of the third submission, the Evidence Review Group agreed that the selection of appropriate survival models was problematic and at the request of the National Institute for Health Care and Excellence investigated external sources of evidence regarding survival outcomes. The Evidence Review Group considered that some cost and utility estimates in the submission may have remained biased in favor of ixazomib. As a result of their third appraisal meeting, the committee judged that for the two to three prior therapies population, and at the price agreed in a commercial access agreement, ixazomib had the potential to be cost effective. It was referred to the Cancer Drugs Fund so that further data could accrue with the aim of diminishing the clinical uncertainties