A Comparison of methods for the Systematic Review of Qualitative Research : Two Examples Using Meta-Ethnography and Meta-Study

Systematic reviews and meta-analysis have been a central pillar of evidence-based practice and policy making in healthcare over recent years. Traditionally, this has focused on effectiveness evidence from trials. There is increasing understanding, however, that other study designs also provide essential information and this has led to interest in developing ways to review and synthesis such evidence. Qualitative research has unique potential to illuminate the patient experience

Barriers to, and facilitators of, the prevention of unintentional injury in children in the home: a systematic review and synthesis of qualitative research

Journal ArticleMeta-AnalysisResearch Support, Non-U.S. Gov'tReviewBACKGROUND: This review considers barriers to, and facilitators of, success for interventions to reduce unintentional injury to children in the home through supply and/or installation of home safety equipment, and looks at risk assessments. METHODS: A systematic review of qualitative research. Bibliographic databases were searched for studies on interventions to reduce unintentional child injury in the home, or on related attitudes and behaviours. Studies were quality appraised, findings extracted, and a conceptual framework was developed to assess factors affecting the success of interventions. RESULTS: Nine peer-reviewed journal articles were included. Barriers and facilitators were highlighted at organisational, environmental and personal levels. Effective provision of safety equipment involves ongoing support with installation and maintenance. Take up and success of interventions depends on adjusting interventions according to practical limitations and parents' cultural expectations. A particular barrier was parents' inability to modify rented or shared accommodation. CONCLUSIONS: The review highlights ways in which health inequalities affect the take up and success of home safety interventions, and how health workers can use this knowledge to facilitate future interventions.NICE Centre for Public Health Excellenc

Synthesising qualitative research findings: what are they, where are they and what should I do with them?

Ruth Garside makes a presentation on synthesizing qualitative research findings and the different methods for qualitative evidence synthesi

Preventing unintentional injuries to children under 15 years in the outdoors: A systematic review of the effectiveness of educational programs

Introduction Unintentional injuries to children in the outdoors have a significant impact on child mortality, development and healthcare costs. This paper presents the findings of a systematic review about the effectiveness of programs that provided information, advice or education about the prevention of unintentional injuries to children under 15 years during outdoor play and leisure. Methods A structured search strategy was conducted in a range of databases. All report titles and abstracts were screened using pre-defined criteria. Included reports were quality appraised using a modified Graphical Appraisal Tool for Epidemiological studies (GATE) tool. All quality appraisals and data extraction were checked by a second reviewer. If not provided in the original reports, ORs and mean differences were calculated, where sufficient data were available. Results Twenty-three studies met the inclusion criteria. There was a paucity of robust study designs. The majority of studies only reported a short-term follow-up of intermediate outcome measures. Only two studies measured injury rates; both reported a reduction, but both studies also had considerable methodological weaknesses. The five studies that measured the use of protective equipment reported mixed results, although there is some evidence that suggests that more extensive educational programs (such as health fairs and media campaigns) increase their use. The 20 studies that measured behaviour, attitude or knowledge outcomes reported highly mixed results. Discussion Methodological weaknesses of the included studies limit support for a particular course of action. To better inform policy and practice, future research should (1) use robust study designs and (2) not rely on short-term proxy outcome measures

Is it safe to go back into the water? A systematic review and meta-analysis of the risk of acquiring infections from recreational exposure to seawater

Background: Numerous illnesses are associated with bathing in natural waters, although it is assumed that the risk of illness among bathers exposed to relatively clean waters found in high-income countries is negligible. A systematic review was carried out to quantify the increased risk of experiencing a range of adverse health outcomes among bathers exposed to coastal water compared with non-bathers. Methods: In all 6919 potentially relevant titles and abstracts were screened, and from these 40 studies were eligible for inclusion in the review. Odds ratios (OR) were extracted from 19 of these reports and combined in random-effect meta-analyses for the following adverse health outcomes: incident cases of any illness, ear infections, gastrointestinal illness and infections caused by specific microorganisms. Results: There is an increased risk of experiencing symptoms of any illness [OR = 1.86, 95% confidence interval (CI): 1.31 to 2.64, P = 0.001] and ear ailments (OR = 2.05, 95% CI: 1.49 to 2.82, P < 0.001) in bathers compared with non-bathers. There is also an increased risk of experiencing gastrointestinal ailments (OR = 1.29, 95% CI: 1.12 to 1.49, P < 0.001). Conclusions: This is the first systematic review to evaluate evidence on the increased risk of acquiring illnesses from bathing in seawater compared with non-bathers. Our results support the notion that infections are acquired from bathing in coastal waters, and that bathers have a greater risk of experiencing a variety of illnesses compared with non-bathers

Meta-analyses of randomized controlled trials show suboptimal validity of surrogate outcomes for overall survival in advanced colorectal cancer

Objectives: To quantify and compare the treatment effects on three surrogate end points, progression-free survival (PFS), time to progression (TTP), and tumor response rate (TR) vs. overall survival (OS) based on a meta-analysis of randomized controlled trials (RCTs) of drug interventions in advanced colorectal cancer (aCRC). Study Design and Setting: We systematically searched for RCTs of pharmacologic therapies in aCRC between 2003 and 2013. Trial characteristics, risk of bias, and outcomes were recorded based on a predefined form. Univariate and multivariate random-effects meta-analyses were used to estimate pooled summary treatment effects. The ratio of hazard ratios (HRs)/odds ratios (ORs) and difference in medians were used to quantify the degree of difference in treatment effects on the surrogate end points and OS. Spearman ρ, surrogate threshold effect (STE), and R2 were also estimated across predefined trial-level covariates. Results: We included 101 RCTs. In univariate and multivariate meta-analyses, we found larger treatment effects for the surrogates than for OS. Compared with OS, treatment effects were on average 13% higher when HRs were measured and 3% to 45% higher when ORs were considered; differences in median PFS/TTP were higher than on OS by an average of 0.5 month. Spearman ρ ranged from 0.39 to 0.80, mean R2 from 0.06 to 0.65, and STE was 0.8 for HRPFS, 0.64 for HRTTP, or 0.28 for ORTR. The stratified analyses revealed high variability across all strata. Conclusion: None of the end points in this study were found to achieve the level of evidence (ie, mean R2trial &gt; 0.60) that has been set to select high or excellent correlation levels by common surrogate evaluation tools. Previous surrogacy relationships observed between PFS and TTP vs. OS in selected settings may not apply across other classes or lines of therapy

Can a meta-ethnography be updated by different reviewers? Reflections from a recent update

Over the last decade, there has been a proliferation of published meta-ethnographies. Yet, strategies and techniques for updating have not received the same attention, rendering answers to important methodological questions still elusive. One such question has to do with who can perform an update. Although it is not uncommon for quantitative systematic reviews and statistical meta-analyses to be updated by different reviewers, qualitative synthesists might find themselves caught between a rock and a hard place. On the one hand, as meta-ethnography constitutes an interpretation three times removed from the lived experience of the participants in the original studies, it could be argued that an update by different reviewers might add an extra layer of interpretation. By comparison, updating by the same reviewers could give rise to concerns about the robustness of updated findings, as an implicit drive for making new data fit the original work might be difficult to control for. We recently reported the findings of our attempt to update an earlier meta-ethnography of primary care antibiotic prescribing, conducted by a different team of reviewers. In this article, we wish to contribute to the emerging debate on the necessity of promoting a culture of updating in qualitative evidence synthesis, by discussing some of the practical and methodological issues we considered at each stage of the process and offering lessons learnt from our experience

Cochrane Qualitative and Implementation Methods Group guidance paper 6:Methods for question formulation, searching, and protocol development for qualitative evidence synthesis

This paper updates previous Cochrane guidance on question formulation, searching, and protocol development, reflecting recent developments in methods for conducting qualitative evidence syntheses to inform Cochrane intervention reviews. Examples are used to illustrate how decisions about boundaries for a review are formed via an iterative process of constructing lines of inquiry and mapping the available information to ascertain whether evidence exists to answer questions related to effectiveness, implementation, feasibility, appropriateness, economic evidence, and equity. The process of question formulation allows reviewers to situate the topic in relation to how it informs and explains effectiveness, using the criterion of meaningfulness, appropriateness, feasibility, and implementation. Questions related to complex questions and interventions can be structured by drawing on an increasingly wide range of question frameworks. Logic models and theoretical frameworks are useful tools for conceptually mapping the literature to illustrate the complexity of the phenomenon of interest. Furthermore, protocol development may require iterative question formulation and searching. Consequently, the final protocol may function as a guide rather than a prescriptive route map, particularly in qualitative reviews that ask more exploratory and open-ended questions

Gender and active travel: a qualitative data synthesis informed by machine learning

Abstract: Background: Innovative approaches are required to move beyond individual approaches to behaviour change and develop more appropriate insights for the complex challenge of increasing population levels of activity. Recent research has drawn on social practice theory to describe the recursive and relational character of active living but to date most evidence is limited to small-scale qualitative research studies. To ‘upscale’ insights from individual contexts, we pooled data from five qualitative studies and used machine learning software to explore gendered patterns in the context of active travel. Methods: We drew on 280 transcripts from five research projects conducted in the UK, including studies of a range of populations, travel modes and settings, to conduct unsupervised ‘topic modelling analysis’. Text analytics software, Leximancer, was used in the first phase of the analysis to produce inter-topic distance maps to illustrate inter-related ‘concepts’. The outputs from this first phase guided a second researcher-led interpretive analysis of text excerpts to infer meaning from the computer-generated outputs. Results: Guided by social practice theory, we identified ‘interrelated’ and ‘relating’ practices across the pooled datasets. For this study we particularly focused on respondents’ commutes, travelling to and from work, and on differentiated experiences by gender. Women largely described their commute as multifunctional journeys that included the school run or shopping, whereas men described relatively linear journeys from A to B but highlighted ‘relating’ practices resulting from or due to their choice of commute mode or journey such as showering or relaxing. Secondly, we identify a difference in discourses about practices across the included datasets. Women spoke more about ‘subjective’, internal feelings of safety (‘I feel unsafe’), whereas men spoke more about external conditions (‘it is a dangerous road’). Conclusion: This rare application of machine learning to qualitative social science research has helped to identify potentially important differences in co-occurrence of practices and discourses about practice between men’s and women’s accounts of travel across diverse contexts. These findings can inform future research and policy decisions for promoting travel-related social practices associated with increased physical activity that are appropriate across genders

Non-pharmacological interventions for ADHD in school settings: an overarching synthesis of systematic reviews

OBJECTIVE: This overarching synthesis brings together the findings of four systematic reviews including 138 studies focused on non-pharmacological interventions for ADHD used in school settings. These reviews considered the effectiveness of school-based interventions for ADHD, attitudes toward and experience of school-based interventions for ADHD, and the experience of ADHD in school settings. METHOD: We developed novel methods to compare the findings across these reviews inductively and deductively. RESULTS: Key contextual issues that may influence the effectiveness and implementation of interventions include the relationships that pupils with ADHD have with their teachers and peers, the attributions individuals make about the etiology of ADHD, and stigma related to ADHD or intervention attendance. CONCLUSION: Although we found some positive effects for some outcomes and intervention categories, heterogeneity in effect size estimates and research evidence suggests a range of diverse contextual factors potentially moderate the implementation and effectiveness of school-based interventions for ADHD.National Institute for Health Research Health Technology Assessment (NIHR HTA) programNIHR Collaboration for Leadership in Applied HealthResearch and Care South West Peninsula (PenCLAHRC