When and how to update systematic reviews

Governments, funding agencies, academic institutions, and health care policy makers are increasingly investing in the design, development, and dissemination of systematic reviews (SRs) to inform clinical practice guidelines, ethical guidance of clinical research, and health care practice and policy. SRs need to be sensitive to the dynamic nature of new evidence, such as published papers. The emergence of new evidence over time may undermine the validity of conclusions and recommendations in any given SR and subsequent practice guideline. This issue has only started to be more seriously considered during the last decade or so. Now it is clear that the use of out-dated evidence can lead to a waste of resources, provision of redundant, ineffective or even harmful health care. The author of this dissertation and his colleagues conducted and published three empirical studies and two conceptual articles (in six peer-reviewed journal publications), which addressed the methodologic aspects of when and how to update SRs. This PhD project provides a summary of these publications. The work described herein has had a significant impact on raising awareness and initiating new research efforts for keeping SRs up-to-date. Publication 1 proposed the first formal definition of what constitutes an update of a SR. The article presented distinguishing features of an updated vs. not updated or a new review. Publication 2 (or Publication 3) systematically reviewed methods, techniques, and strategies describing when and how to update SRs (Study #1). Publication 4, an international survey (Study #2), identified and described updating practices and policies of organisations involved in the production and commission of SRs. Publication 5 reviewed the knowledge and efforts in updating SRs and provided guidance for authors and SR groups as to when and how to update comparative effectiveness reviews produced by the Agency for Healthcare Research and Quality’s (AHRQ) Evidence-based Practice Centres (EPCs) throughout North America. Publication 6 (Study #3) described the development, piloting, and feasibility of a surveillance system to assess the need for updating comparative effectiveness reviews produced by the AHRQ’s EPC Program. This surveillance method has proved to be an efficient approach for prioritising SRs with respect to updating need

Community-onset sepsis and its public health burden : protocol of a systematic review

Background: Sepsis is a life-threatening condition and major contributor of public health and economic burden in the industrialised world. The heterogeneity, absence of more specific definition, and difficulties in accurate diagnosis lead to great variability in the estimates of sepsis incidence. There has been uncertainty regarding the incidence and risk factors attributable to community-onset as opposed to hospital-acquired sepsis. Community-onset sepsis has distinct host characteristics, risk factors, pathogens, and prognosis. A systematic assessment of recent evidence is warranted in light of secular changes in epidemiology, pathogens, and the uncertainties around the incidence and risk factors of community-onset sepsis. This protocol describes a systematic review which aims to synthesise the recent empirical evidence on the incidence and risk factors of community-onset sepsis, severe sepsis, and septic shock in high-income countries. Methods/design: English-language publications of cohort and case-control studies reporting incidence and risk factors of community-onset sepsis will be eligible for inclusion. MEDLINE and Embase databases will be searched from 2002 and onwards. References of relevant publications will be hand-searched. Two reviewers will independently screen titles/abstracts and full texts as well as extract data and appraise the risk of bias of included studies. The data extractions and risk of bias assessments will be cross-checked. Any disagreements will be resolved via consensus. The data on incidence and risk factors of sepsis will be organised and synthesised in text, tables, and forest plots. The evidence will be pooled given sufficient data and degree of similarity across study populations, exposures, and outcomes. The heterogeneity will be assessed through visual inspection of forest plots, Chi-square-based p value, and I (2) statistic. The sources of heterogeneity will be explored via subgroup analysis. Discussion: Timeliness and accuracy of diagnosis of sepsis are both crucial aspects for improving the patient's outcome. The findings of this review will be discussed with a view to better inform future recommendations on improving public-facing campaigns, timely presentation, and diagnosis of sepsis in the community. The review will also discuss gaps in evidence and highlight future research and policy-making avenues for improving public health planning. Systematic review registration: PROSPERO CRD42015023484

A systematic review of economic models used to assess the cost-effectiveness of strategies for identifying latent tuberculosis in high-risk groups

Timely diagnosis and treatment of latent tuberculosis infection (LTBI) through screening remains a key public health priority. Although globally it is recommended to screen people at high risk of developing TB, the economic evidence underpinning these recommendations is limited. This review critically appraised studies that had used a decision-analytical modelling framework to estimate the cost-effectiveness of interferon gamma release assays (IGRAs) compared to tuberculin skin test (TST) for detecting LTBI in high risk populations. A systematic review of economic models used to assess the cost-effectiveness of strategies for identifying latent tuberculosis in high-risk groups (PDF Download Available). Available from: https://www.researchgate.net/publication/301759613_A_systematic_review_of_economic_models_used_to_assess_the_cost-effectiveness_of_strategies_for_identifying_latent_tuberculosis_in_high-risk_groups [accessed May 27, 2016]

Cost-Effectiveness of Manual Therapy for the Management of Musculoskeletal Conditions: A Systematic Review and Narrative Synthesis of Evidence From Randomized Controlled Trials

AbstractObjectivesThe purpose of this study was to systematically review trial-based economic evaluations of manual therapy relative to other alternative interventions used for the management of musculoskeletal conditions.MethodsA comprehensive literature search was undertaken in major medical, health-related, science and health economic electronic databases.ResultsTwenty-five publications were included (11 trial-based economic evaluations). The studies compared cost-effectiveness and/or cost-utility of manual therapy interventions to other treatment alternatives in reducing pain (spinal, shoulder, ankle). Manual therapy techniques (eg, osteopathic spinal manipulation, physiotherapy manipulation and mobilization techniques, and chiropractic manipulation with or without other treatments) were more cost-effective than usual general practitioner (GP) care alone or with exercise, spinal stabilization, GP advice, advice to remain active, or brief pain management for improving low back and shoulder pain/disability. Chiropractic manipulation was found to be less costly and more effective than alternative treatment compared with either physiotherapy or GP care in improving neck pain.ConclusionsPreliminary evidence from this review shows some economic advantage of manual therapy relative to other interventions used for the management of musculoskeletal conditions, indicating that some manual therapy techniques may be more cost-effective than usual GP care, spinal stabilization, GP advice, advice to remain active, or brief pain management for improving low back and shoulder pain/disability. However, at present, there is a paucity of evidence on the cost-effectiveness and/or cost-utility evaluations for manual therapy interventions. Further improvements in the methodological conduct and reporting quality of economic evaluations of manual therapy are warranted in order to facilitate adequate evidence-based decisions among policy makers, health care practitioners, and patients

Extending the diabetic retinopathy screening interval beyond 1 year : systematic review

To determine whether the recommended screening interval for diabetic retinopathy (DR) in the UK can safely be extended beyond 1 year. Systematic review of clinical and cost-effectiveness studies. Nine databases were searched with no date restrictions. Randomised controlled trials (RCTs), cohort studies, prognostic or economic modelling studies which described the incidence and progression of DR in populations with type 1 diabetes mellitus or type 2 diabetes mellitus of either sex and of any age reporting incidence and progression of DR in relation to screening interval (vs annual screening interval) and/or prognostic factors were included. Narrative synthesis was undertaken. 14 013 papers were identified, of which 11 observational studies, 5 risk stratification modelling studies and 9 economic studies were included. Data were available for 262 541 patients of whom at least 228 649 (87%) had type 2 diabetes. There were no RCTs. Studies concluded that there is little difference between clinical outcomes from screening 1 yearly or 2 yearly in low-risk patients. However there was high loss to follow-up (13–31%), heterogeneity in definitions of low risk and variation in screening and grading protocols for prior retinopathy results. Observational and economic modelling studies in low-risk patients show little difference in clinical outcomes between 1-year and 2-year screening intervals. The lack of experimental research designs and heterogeneity in definition of low risk considerably limits the reliability and validity of this conclusion. Cost-effectiveness findings were mixed. There is insufficient evidence to recommend a move to extend the screening interval beyond 1 year

Updating Systematic Reviews: An International Survey

BACKGROUND: Systematic reviews (SRs) should be up to date to maintain their importance in informing healthcare policy and practice. However, little guidance is available about when and how to update SRs. Moreover, the updating policies and practices of organizations that commission or produce SRs are unclear. METHODOLOGY/PRINCIPAL FINDINGS: The objective was to describe the updating practices and policies of agencies that sponsor or conduct SRs. An Internet-based survey was administered to a purposive non-random sample of 195 healthcare organizations within the international SR community. Survey results were analyzed using descriptive statistics. The completed response rate was 58% (n = 114) from across 26 countries with 70% (75/107) of participants identified as producers of SRs. Among responders, 79% (84/107) characterized the importance of updating as high or very-high and 57% (60/106) of organizations reported to have a formal policy for updating. However, only 29% (35/106) of organizations made reference to a written policy document. Several groups (62/105; 59%) reported updating practices as irregular, and over half (53/103) of organizational respondents estimated that more than 50% of their respective SRs were likely out of date. Authors of the original SR (42/106; 40%) were most often deemed responsible for ensuring SRs were current. Barriers to updating included resource constraints, reviewer motivation, lack of academic credit, and limited publishing formats. Most respondents (70/100; 70%) indicated that they supported centralization of updating efforts across institutions or agencies. Furthermore, 84% (83/99) of respondents indicated they favoured the development of a central registry of SRs, analogous to efforts within the clinical trials community. CONCLUSIONS/SIGNIFICANCE: Most organizations that sponsor and/or carry out SRs consider updating important. Despite this recognition, updating practices are not regular, and many organizations lack a formal written policy for updating SRs. This research marks the first baseline data available on updating from an organizational perspective

Agreement between gastrointestinal panel testing and standard microbiology methods for detecting pathogens in suspected infectious gastroenteritis : test evaluation and meta-analysis in the absence of a reference standard

Objective: Multiplex gastrointestinal pathogen panel (GPP) tests simultaneously identify bacterial, viral and parasitic pathogens from the stool samples of patients with suspected infectious gastroenteritis presenting in hospital or the community. We undertook a systematic review to compare the accuracy of GPP tests with standard microbiology techniques. Review methods: Searches in Medline, Embase, Web of Science and the Cochrane library were undertaken from inception to January 2016. Eligible studies compared GPP tests with standard microbiology techniques in patients with suspected gastroenteritis. Quality assessment of included studies used tailored QUADAS-2. In the absence of a reference standard we analysed test performance taking GPP tests and standard microbiology techniques in turn as the benchmark test, using random effects meta-analysis of proportions. Results: No study provided an adequate reference standard with which to compare the test accuracy of GPP and conventional tests. Ten studies informed a meta-analysis of positive and negative agreement. Positive agreement across all pathogens was 0.93 (95% CI 0.90 to 0.96) when conventional methods were the benchmark and 0.68 (95% CI: 0.58 to 0.77) when GPP provided the benchmark. Negative agreement was high in both instances due to the high proportion of negative cases. GPP testing produced a greater number of pathogen-positive findings than conventional testing. It is unclear whether these additional ‘positives’ are clinically important. Conclusions: GPP testing has the potential to simplify testing and accelerate reporting when compared to conventional microbiology methods. However the impact of GPP testing upon the management, treatment and outcome of patients is poorly understood and further studies are needed to evaluate the health economic impact of GPP testing compared with standard methods

Ixazomib for relapsed or refractory multiple myeloma : review from an evidence review group on a NICE single technology appraisal

Ixazomib is an oral proteasome inhibitor used in combination with lenalidomide plus dexamethasone (IXA-LEN-DEX) and licensed for relapsed or refractory multiple myeloma. As part of a single technology appraisal (ID807) undertaken by the National Institute of Health and Care Excellence, the Evidence Review Group, Warwick Evidence was invited to independently review the evidence submitted by the manufacturer of ixazomib, Takeda UK Ltd. The main source of clinical effectiveness data about IXA-LEN-DEX came from the Tourmaline-MM1 randomized controlled trial in which 771 patients with relapsed or refractory multiple myeloma received either IXA-LEN-DEX or placebo-LEN-DEX as their second-, third-, or fourth-line treatment. Takeda estimated the cost effectiveness of IXA-LEN-DEX using a de-novo partitioned-survival model with three health states (pre-progression, post-progression, and dead). In their first submission, this model was used to estimate the cost effectiveness of IXA-LEN-DEX vs. bortezomib plus dexamethasone (BORT-DEX) in second-line treatment, and of IXA-LEN-DEX vs. LEN-DEX in third-line treatment. To estimate the relative clinical performance of IXA-LEN-DEX vs. BORT-DEX, Takeda conducted network meta-analyses for important outcomes. The network meta-analysis for overall survival was found to be flawed in several respects, but mainly because a hazard ratio input for one of the studies in the network had been inverted, resulting in a large inflation of the claimed superiority of IXA-LEN-DEX over BORT-DEX and a considerable overestimation of its cost effectiveness. In subsequent submissions, Takeda withdrew second-line treatment as an option for IXA-LEN-DEX. The manufacturer’s first submission comparing IXA-LEN-DEX with LEN-DEX for third-line therapy employed Tourmaline-MM1 data from third- and fourth-line patients as proxy for a third-line population. The appraisal committee did not consider this reasonable because randomization in Tourmaline-MM1 was stratified according to one previous treatment and two or more previous treatments. A further deficiency was considered to be the manufacturer’s use of interim survival data rather than the most mature data available. A second submission from the company focussed on IXA-LEN-DEX vs. LEN-DEX as third- or fourth-line treatment (the two or more previous lines population) and a new patient access scheme was introduced. Covariate modeling of survival outcomes was proposed using the most mature survival data. The Evidence Review Group’s main criticisms of the new evidence included: the utility associated with the pre-progression health state was overestimated, treatment costs of ixazomib were underestimated, survival models were still associated with great uncertainty, leading to clinically implausible anomalies and highly variable incremental cost-effectiveness ratio estimates, and the company had not explored a strong assumption that the survival benefit of IXA-LEN-DEX over LEN-DEX would be fully maintained for a further 22 years beyond the observed data, which encompassed only approximately 2.5 years of observation. The appraisal committee remained unconvinced that ixazomib represented a cost-effective use of National Health Service resources. Takeda’s third submission offered new base-case parametric models for survival outcomes, a new analysis of utilities, and proposed a commercial access agreement. In a brief critique of the third submission, the Evidence Review Group agreed that the selection of appropriate survival models was problematic and at the request of the National Institute for Health Care and Excellence investigated external sources of evidence regarding survival outcomes. The Evidence Review Group considered that some cost and utility estimates in the submission may have remained biased in favor of ixazomib. As a result of their third appraisal meeting, the committee judged that for the two to three prior therapies population, and at the price agreed in a commercial access agreement, ixazomib had the potential to be cost effective. It was referred to the Cancer Drugs Fund so that further data could accrue with the aim of diminishing the clinical uncertainties

A surveillance system to assess the need for updating systematic reviews.

BackgroundSystematic reviews (SRs) can become outdated as new evidence emerges over time. Organizations that produce SRs need a surveillance method to determine when reviews are likely to require updating. This report describes the development and initial results of a surveillance system to assess SRs produced by the Agency for Healthcare Research and Quality (AHRQ) Evidence-based Practice Center (EPC) Program.MethodsTwenty-four SRs were assessed using existing methods that incorporate limited literature searches, expert opinion, and quantitative methods for the presence of signals triggering the need for updating. The system was designed to begin surveillance six months after the release of the original review, and then ceforth every six months for any review not classified as being a high priority for updating. The outcome of each round of surveillance was a classification of the SR as being low, medium or high priority for updating.ResultsTwenty-four SRs underwent surveillance at least once, and ten underwent surveillance a second time during the 18 months of the program. Two SRs were classified as high, five as medium, and 17 as low priority for updating. The time lapse between the searches conducted for the original reports and the updated searches (search time lapse - STL) ranged from 11 months to 62 months: The STL for the high priority reports were 29 months and 54 months; those for medium priority reports ranged from 19 to 62 months; and those for low priority reports ranged from 11 to 33 months. Neither the STL nor the number of new relevant articles was perfectly associated with a signal for updating. Challenges of implementing the surveillance system included determining what constituted the actual conclusions of an SR that required assessing; and sometimes poor response rates of experts.ConclusionIn this system of regular surveillance of 24 systematic reviews on a variety of clinical interventions produced by a leading organization, about 70% of reviews were determined to have a low priority for updating. Evidence suggests that the time period for surveillance is yearly rather than the six months used in this project

Test accuracy of drug and antibody assays for predicting response to anti-Tumour Necrosis Factor treatment in Crohn’s disease : a systematic review and meta-analysis

Objective: To present meta-analytic test accuracy estimates of levels of anti-TNF and antibodies to anti-TNF to predict loss of response or lack of regaining response in anti-TNF managed Crohn’s disease patients. Methods: MEDLINE, Embase, the Cochrane Library and Science Citation Index were searched from inception to October / November 2014 to identify studies which reported 2x2 table data of the association between levels of anti-TNF or its antibodies and clinical status. Hierarchical / bivariate meta-analysis was undertaken with the user-written “metandi” package of Harbord and Whiting using Stata 11 software, for Infliximab, Adalimumab, anti-Infliximab and anti-Adalimumab levels as predictors of loss of response. Prevalence of Crohn’s disease in included studies was meta-analysed using a random effects model in MetaAnalyst software to calculate positive and negative predictive values. The search was updated in January 2017. Results: 31 studies were included in the review. Studies were heterogeneous with respect to type of test used, criteria for establishing response and loss of response, population examined, and results. Metaanalytic summary point estimates for sensitivity and specificity were 65.7% and 80.6% for Infliximab trough levels and 56% and 79% for antibodies to Infliximab, respectively. Pooled results for Adalimumab trough levels and antibodies to Adalimumab were similar. Pooled positive and negative predictive values ranged between 70% and 80% implying that between 20% and 30% of both positive and negative test results may be incorrect in predicting loss of response. Conclusion: The available evidence suggests that these tests have modest predictive accuracy for clinical status, direct test accuracy comparisons in the same population are needed. More clinical trial evidence from test-treat studies is required before the clinical utility of the tests can be reliably evaluated