Assessment of a method to detect signals for updating systematic reviews.

BackgroundSystematic reviews are a cornerstone of evidence-based medicine but are useful only if up-to-date. Methods for detecting signals of when a systematic review needs updating have face validity, but no proposed method has had an assessment of predictive validity performed.MethodsThe AHRQ Comparative Effectiveness Review program had produced 13 comparative effectiveness reviews (CERs), a subcategory of systematic reviews, by 2009, 11 of which were assessed in 2009 using a surveillance system to determine the degree to which individual conclusions were out of date and to assign a priority for updating each report. Four CERs were judged to be a high priority for updating, four CERs were judged to be medium priority for updating, and three CERs were judged to be low priority for updating. AHRQ then commissioned full update reviews for 9 of these 11 CERs. Where possible, we matched the original conclusions with their corresponding conclusions in the update reports, and compared the congruence between these pairs with our original predictions about which conclusions in each CER remained valid. We then classified the concordance of each pair as good, fair, or poor. We also made a summary determination of the priority for updating each CER based on the actual changes in conclusions in the updated report, and compared these determinations with the earlier assessments of priority.ResultsThe 9 CERs included 149 individual conclusions, 84% with matches in the update reports. Across reports, 83% of matched conclusions had good concordance, and 99% had good or fair concordance. The one instance of poor concordance was partially attributable to the publication of new evidence after the surveillance signal searches had been done. Both CERs originally judged as being low priority for updating had no substantive changes to their conclusions in the actual updated report. The agreement on overall priority for updating between prediction and actual changes to conclusions was Kappa = 0.74.ConclusionsThese results provide some support for the validity of a surveillance system for detecting signals indicating when a systematic review needs updating

How physicians evaluate patients with dementia who present with shortness of breath

BackgroundWhether the presence of dementia in patients makes it difficult for physicians to assess the risk such patients might have for serious conditions such as pulmonary embolism (PE) is unknown. Our objective was to examine the differential association of four clinical factors (deep venous thrombosis (DVT)/PE, malignancy, recent surgery, and tachycardia) with PE testing for patients with dementia compared to patients without dementia.MethodsWe performed a cross-sectional study of emergency department (ED) visits to 104 Veterans Affairs (VA) hospitals from 2011 to 2018 by patients aged 60 years and over presenting with shortness of breath (SOB). Our outcomes were PE testing (CT scan and/or D-dimer) and subsequently diagnosed acute PE.ResultsThe sample included 593,001 patient visits for SOB across 7124 ED physicians; 5.6% of the sample had dementia, and 10.6% received PE testing. Three of the four clinical factors examined had a lower association with PE testing for patients with dementia. For example, after taking into account that at baseline, physicians were 0.9 percentage points less likely to test patients with dementia than patients without dementia for PE, physicians were an additional 2.6 percentage points less likely to test patients with dementia who had tachycardia than patients without dementia who had tachycardia. We failed to find evidence that any clinical factor examined had a differentially lower association with a subsequently diagnosed acute PE for patients with dementia.ConclusionsClinical factors known to be predictive of PE risk had a lower association with PE testing for patients with dementia compared to patients without dementia. These results may be consistent with physicians missing these clinical factors more often when evaluating patients with dementia, but also with physicians recognizing such factors but not using them in the decision-making process. Further understanding how physicians evaluate patients with dementia presenting with common acute symptoms may help improve the care delivered to such patients

An evidence map of the effect of Tai Chi on health outcomes.

BackgroundThis evidence map describes the volume and focus of Tai Chi research reporting health outcomes. Originally developed as a martial art, Tai Chi is typically taught as a series of slow, low-impact movements that integrate the breath, mind, and physical activity to achieve greater awareness and a sense of well-being.MethodsThe evidence map is based on a systematic review of systematic reviews. We searched 11 electronic databases from inception to February 2014, screened reviews of reviews, and consulted with topic experts. We used a bubble plot to graphically display clinical topics, literature size, number of reviews, and a broad estimate of effectiveness.ResultsThe map is based on 107 systematic reviews. Two thirds of the reviews were published in the last five years. The topics with the largest number of published randomized controlled trials (RCTs) were general health benefits (51 RCTs), psychological well-being (37 RCTs), interventions for older adults (31 RCTs), balance (27 RCTs), hypertension (18 RCTs), fall prevention (15 RCTs), and cognitive performance (11 RCTs). The map identified a number of areas with evidence of a potentially positive treatment effect on patient outcomes, including Tai Chi for hypertension, fall prevention outside of institutions, cognitive performance, osteoarthritis, depression, chronic obstructive pulmonary disease, pain, balance confidence, and muscle strength. However, identified reviews cautioned that firm conclusions cannot be drawn due to methodological limitations in the original studies and/or an insufficient number of existing research studies.ConclusionsTai Chi has been applied in diverse clinical areas, and for a number of these, systematic reviews have indicated promising results. The evidence map provides a visual overview of Tai Chi research volume and content.Systematic review registrationPROSPERO CRD42014009907

Reflections and aspirations : the journal after 5 years

The journal recently celebrated its fifth anniversary. Like systematic reviews themselves, the journal is thriving and publishing a variety of protocols, reviews, and methods papers. We have also had success in publishing-themed series

A surveillance system to assess the need for updating systematic reviews.

BackgroundSystematic reviews (SRs) can become outdated as new evidence emerges over time. Organizations that produce SRs need a surveillance method to determine when reviews are likely to require updating. This report describes the development and initial results of a surveillance system to assess SRs produced by the Agency for Healthcare Research and Quality (AHRQ) Evidence-based Practice Center (EPC) Program.MethodsTwenty-four SRs were assessed using existing methods that incorporate limited literature searches, expert opinion, and quantitative methods for the presence of signals triggering the need for updating. The system was designed to begin surveillance six months after the release of the original review, and then ceforth every six months for any review not classified as being a high priority for updating. The outcome of each round of surveillance was a classification of the SR as being low, medium or high priority for updating.ResultsTwenty-four SRs underwent surveillance at least once, and ten underwent surveillance a second time during the 18 months of the program. Two SRs were classified as high, five as medium, and 17 as low priority for updating. The time lapse between the searches conducted for the original reports and the updated searches (search time lapse - STL) ranged from 11 months to 62 months: The STL for the high priority reports were 29 months and 54 months; those for medium priority reports ranged from 19 to 62 months; and those for low priority reports ranged from 11 to 33 months. Neither the STL nor the number of new relevant articles was perfectly associated with a signal for updating. Challenges of implementing the surveillance system included determining what constituted the actual conclusions of an SR that required assessing; and sometimes poor response rates of experts.ConclusionIn this system of regular surveillance of 24 systematic reviews on a variety of clinical interventions produced by a leading organization, about 70% of reviews were determined to have a low priority for updating. Evidence suggests that the time period for surveillance is yearly rather than the six months used in this project

Identifying continuous quality improvement publications: what makes an improvement intervention ‘CQI’?

Background: The term continuous quality improvement (CQI) is often used to refer to a method for improving care, but no consensus statement exists on the definition of CQI. Evidence reviews are critical for advancing science, and depend on reliable definitions for article selection. Methods: As a preliminary step towards improving CQI evidence reviews, this study aimed to use expert panel methods to identify key CQI definitional features and develop and test a screening instrument for reliably identifying articles with the key features. We used a previously published method to identify 106 articles meeting the general definition of a quality improvement intervention (QII) from 9427 electronically identified articles from PubMed. Two raters then applied a six-item CQI screen to the 106 articles. Results: Per cent agreement ranged from 55.7% to 75.5% for the six items, and reviewer-adjusted intra-class correlation ranged from 0.43 to 0.62. ‘Feedback of systematically collected data’ was the most common feature (64%), followed by being at least ‘somewhat’ adapted to local conditions (61%), feedback at meetings involving participant leaders (46%), using an iterative development process (40%), being at least ‘somewhat’ data driven (34%), and using a recognised change method (28%). All six features were present in 14.2% of QII articles. Conclusions: We conclude that CQI features can be extracted from QII articles with reasonable reliability, but only a small proportion of QII articles include all features. Further consensus development is needed to support meaningful use of the term CQI for scientific communication