Synthesizing diverse evidence: the use of primary qualitative data analysis methods and logic models in public health reviews

Objectives: The nature of public health evidence presents challenges for conventional systematic review processes, with increasing recognition of the need to include a broader range of work including observational studies and qualitative research, yet with methods to combine diverse sources remaining underdeveloped. The objective of this paper is to report the application of a new approach for review of evidence in the public health sphere. The method enables a diverse range of evidence types to be synthesized in order to examine potential relationships between a public health environment and outcomes. Study design: The study drew on previous work by the National Institute for Health and Clinical Excellence on conceptual frameworks. It applied and further extended this work to the synthesis of evidence relating to one particular public health area: the enhancement of employee mental well-being in the workplace. Methods: The approach utilized thematic analysis techniques from primary research, together with conceptual modelling, to explore potential relationships between factors and outcomes. Results: The method enabled a logic framework to be built from a diverse document set that illustrates how elements and associations between elements may impact on the well-being of employees. Conclusions: Whilst recognizing potential criticisms of the approach, it is suggested that logic models can be a useful way of examining the complexity of relationships between factors and outcomes in public health, and of highlighting potential areas for interventions and further research. The use of techniques from primary qualitative research may also be helpful in synthesizing diverse document types. (C) 2010 The Royal Society for Public Health. Published by Elsevier Ltd. All rights reserved

Using framework-based synthesis for conducting reviews of qualitative studies

Framework analysis is a technique used for data analysis in primary qualitative research. Recent years have seen its being adapted to conduct syntheses of qualitative studies. Framework-based synthesis shows considerable promise in addressing applied policy questions. An innovation in the approach, known as 'best fit' framework synthesis, has been published in BMC Medical Research Methodology this month. It involves reviewers in choosing a conceptual model likely to be suitable for the question of the review, and using it as the basis of their initial coding framework. This framework is then modified in response to the evidence reported in the studies in the reviews, so that the final product is a revised framework that may include both modified factors and new factors that were not anticipated in the original model. 'Best fit' framework-based synthesis may be especially suitable in addressing urgent policy questions where the need for a more fully developed synthesis is balanced by the need for a quick answer

‘Trying to pin down jelly’ - exploring intuitive processes in quality assessment for meta-ethnography

Background: Studies that systematically search for and synthesise qualitative research are becoming more evident in health care, and they can make an important contribution to patient care. However, there is still no agreement as to whether, or how we should appraise studies for inclusion. We aimed to explore the intuitive processes that determined the ‘quality’ of qualitative research for inclusion in qualitative research syntheses. We were particularly interested to explore the way that knowledge was constructed. Methods: We used qualitative methods to explore the process of quality appraisal within a team of seven qualitative researchers funded to undertake a meta-ethnography of chronic non-malignant musculoskeletal pain. Team discussions took place monthly between October 2010 and June 2012 and were recorded and transcribed. Data was coded and organised using constant comparative method. The development of our conceptual analysis was both iterative and collaborative. The strength of this team approach to quality came from open and honest discussion, where team members felt free to agree, disagree, or change their position within the safety of the group. Results: We suggest two core facets of quality for inclusion in meta-ethnography - (1) Conceptual clarity; how clearly has the author articulated a concept that facilitates theoretical insight. (2) Interpretive rigour; fundamentally, can the interpretation ‘be trusted?’ Our findings showed that three important categories help the reader to judge interpretive rigour: (ii) What is the context of the interpretation? (ii) How inductive is the interpretation? (iii) Has the researcher challenged their interpretation? Conclusions: We highlight that methods alone do not determine the quality of research for inclusion into a meta-ethnography. The strength of a concept and its capacity to facilitate theoretical insight is integral to meta-ethnography, and arguably to the quality of research. However, we suggest that to be judged ‘good enough’ there also needs to be some assurance that qualitative findings are more than simply anecdotal. Although our conceptual model was developed specifically for meta-ethnography, it may be transferable to other research methodologies

Scoping studies: towards a methodological framework

This paper focuses on scoping studies, an approach to reviewing the literature which to date has received little attention in the research methods literature. We distinguish between different types of scoping studies and indicate where these stand in relation to full systematic reviews. We outline a framework for conducting a scoping study based on our recent experiences of reviewing the literature on services for carers for people with mental health problems. Where appropriate, our approach to scoping the field is contrasted with the procedures followed in systematic reviews. We emphasize how including a consultation exercise in this sort of study may enhance the results, making them more useful to policy makers, practitioners and service users. Finally, we consider the advantages and limitations of the approach and suggest that a wider debate is called for about the role of the scoping study in relation to other types of literature reviews

Do women consult more than men? A review of gender and consultation for back pain and headache

Objectives: Because women consult their general practitioners more frequently on average than men, it is commonly assumed that they consult more for all symptoms and conditions. This assumption is reinforced by qualitative studies reporting a widespread reluctance to consult by men. However, few studies directly compare consultation in men and women experiencing similar symptoms or conditions. Methods: A systematic review of the evidence on gender and consultation for two common symptoms, back pain and headache. Extensive electronic searches identified 15 papers reporting the relationship between gender and help-seeking for back pain and 11 papers for headache. Two independent reviewers assessed articles for inclusion and extracted data from eligible studies. Results: Few studies compared consultation patterns for these symptoms among men and women known to have experienced the symptom. The quality of the studies was variable. Overall, evidence for greater consultation by women with back pain was weak and inconsistent. Among those with back pain, the odds ratios for women seeking help, compared with men, ranged from 0.6 (95% confidence intervals 0.3, 1.2, adjusted only for age) to 2.17 (95% confidence intervals 1.35, 3.57, unadjusted), although none of the reported odds ratio, below 1.00 was statistically significant. The evidence for women being more likely to consult for headache was a little stronger. Five studies showed a statistically elevated odds ratio, and none suggested that men with headache symptoms were more likely to consult than women with headache symptoms. Limitations to the studies are discussed. Conclusion: Given the strength of assumptions that women consult more readily for common symptoms, the evidence for greater consultation amongst women for two common symptoms, headache and back pain, was surprisingly weak and inconsistent, especially with respect to back pain.</p

A worked example of "best fit" framework synthesis: A systematic review of views concerning the taking of some potential chemopreventive agents

<p>Abstract</p> <p>Background</p> <p>A variety of different approaches to the synthesis of qualitative data are advocated in the literature. The aim of this paper is to describe the application of a pragmatic method of qualitative evidence synthesis and the lessons learned from adopting this "best fit" framework synthesis approach.</p> <p>Methods</p> <p>An evaluation of framework synthesis as an approach to the qualitative systematic review of evidence exploring the views of adults to the taking of potential agents within the context of the primary prevention of colorectal cancer.</p> <p>Results</p> <p>Twenty papers from North America, Australia, the UK and Europe met the criteria for inclusion. Fourteen themes were identified <it>a priori </it>from a related, existing conceptual model identified in the literature, which were then used to code the extracted data. Further analysis resulted in the generation of a more sophisticated model with additional themes. The synthesis required a combination of secondary framework and thematic analysis approaches and was conducted within a health technology assessment timeframe.</p> <p>Conclusion</p> <p>The novel and pragmatic "best fit" approach to framework synthesis developed and described here was found to be fit for purpose. Future research should seek to test further this approach to qualitative data synthesis.</p

Where is the evidence for emergency planning: a scoping review

Background Recent terrorist attacks and natural disasters have led to an increased awareness of the importance of emergency planning. However, the extent to which emergency planners can access or use evidence remains unclear. The aim of this study was to identify, analyse and assess the location, source and quality of emergency planning publications in the academic and UK grey literature. Methods We conducted a scoping review, using as data sources for academic literature Embase, Medline, Medline in Process, Psychinfo, Biosis, Science Citation Index, Cinahl, Cochrane library and Clinicaltrials.gov. For grey literature identification we used databases at the Health Protection Agency, NHS Evidence, British Association of Immediate Care Schemes, Emergency Planning College and the Health and Safety Executive, and the websites of UK Department of Health Emergency Planning Division and UK Resilience. Aggregative synthesis was used to analyse papers and documents against a framework based on a modified FEMA Emergency Planning cycle. Results Of 2736 titles identified from the academic literature, 1603 were relevant. 45% were from North America, 27% were commentaries or editorials and 22% were event reports. Of 192 documents from the grey literature, 97 were relevant. 76% of these were event reports. The majority of documents addressed emergency planning and response. Very few documents related to hazard analysis, mitigation or capability assessment. Conclusions Although a large body of literature exists, its validity and generalisability is unclear There is little evidence that this potential evidence base has been exploited through synthesis to inform policy and practice. The type and structure of evidence that would be of most value of emergency planners and policymakers has yet to be identified

Methods for the thematic synthesis of qualitative research in systematic reviews

<p>Abstract</p> <p>Background</p> <p>There is a growing recognition of the value of synthesising qualitative research in the evidence base in order to facilitate effective and appropriate health care. In response to this, methods for undertaking these syntheses are currently being developed. Thematic analysis is a method that is often used to analyse data in primary qualitative research. This paper reports on the use of this type of analysis in systematic reviews to bring together and integrate the findings of multiple qualitative studies.</p> <p>Methods</p> <p>We describe thematic synthesis, outline several steps for its conduct and illustrate the process and outcome of this approach using a completed review of health promotion research. Thematic synthesis has three stages: the coding of text 'line-by-line'; the development of 'descriptive themes'; and the generation of 'analytical themes'. While the development of descriptive themes remains 'close' to the primary studies, the analytical themes represent a stage of interpretation whereby the reviewers 'go beyond' the primary studies and generate new interpretive constructs, explanations or hypotheses. The use of computer software can facilitate this method of synthesis; detailed guidance is given on how this can be achieved.</p> <p>Results</p> <p>We used thematic synthesis to combine the studies of children's views and identified key themes to explore in the intervention studies. Most interventions were based in school and often combined learning about health benefits with 'hands-on' experience. The studies of children's views suggested that fruit and vegetables should be treated in different ways, and that messages should not focus on health warnings. Interventions that were in line with these suggestions tended to be more effective. Thematic synthesis enabled us to stay 'close' to the results of the primary studies, synthesising them in a transparent way, and facilitating the explicit production of new concepts and hypotheses.</p> <p>Conclusion</p> <p>We compare thematic synthesis to other methods for the synthesis of qualitative research, discussing issues of context and rigour. Thematic synthesis is presented as a tried and tested method that preserves an explicit and transparent link between conclusions and the text of primary studies; as such it preserves principles that have traditionally been important to systematic reviewing.</p