Time to rethink the systematic review catechism? Moving from 'what works' to 'what happens'.

Systematic review methods are developing rapidly, and most researchers would recognise their key methodological aspects, such as a closely focussed question, a comprehensive search, and a focus on synthesising 'stronger' rather than 'weaker' evidence. However, it may be helpful to question some of these underlying principles, because while they work well for simpler review questions, they may result in overly narrow approaches to more complex questions and interventions. This commentary discusses some core principles of systematic reviews, and how they may require further rethinking, particularly as reviewers turn their attention to increasingly complex issues, where a Bayesian perspective on evidence synthesis, which would aim to assemble evidence - of different types, if necessary - in order to inform decisions', may be more productive than the 'traditional' systematic review model. Among areas identified for future research are the examination of publication bias in qualitative research; research on the efficiency and potential biases of comprehensive searches in different disciplines; and the use of Bayesian methods in evidence synthesis. The incorporation of a systems perspective into systematic reviews is also an area which needs rapid development

Parents' champions vs. vested interests: who do parents believe about MMR? A qualitative study.

BACKGROUND: Despite the Government acting quickly to reassure parents about MMR safety following the publication of the 1998 paper by Wakefield and colleagues, MMR uptake declined. One of the reasons suggested for this decline is a loss of public trust in politicians and health professionals. The purpose of this analysis was to examine parents' views on the role the media, politicians and health professionals have played in providing credible evidence about MMR safety. METHODS: A qualitative focus group study conducted with parents living in Central Scotland. Eighteen focus groups were conducted with 72 parents (64 mothers and 8 fathers) between November 2002 and March 2003. Purposive sampling was used to ensure maximum variation among parents. RESULTS: In the period after the MMR controversy, parents found it difficult to know who to trust to offer balanced and accurate information. The general consensus was that politicians were untrustworthy in matters of health. The motives of primary health care providers were suspected by some parents, who saw them as having a range of vested interests (including financial incentives). Among the sources of evidence seen by some parents as more credible were other parents, and Andrew Wakefield who was viewed as an important whistle-blower and champion of ordinary parents. CONCLUSION: The provision of accurate information is only one aspect of helping parents make immunisation decisions. Establishing and maintaining trust in the information provided is also important. The MMR controversy may provide useful lessons for health professionals about trust and credibility that may be generalisable to future health controversies

How do systematic reviews incorporate risk of bias assessments into the synthesis of evidence? A methodological study

Background: Systematic reviews (SRs) are expected to critically appraise included studies and privilege those at lowest risk of bias (RoB) in the synthesis. This study examines if and how critical appraisals inform the synthesis and interpretation of evidence in SRs.<p></p> Methods: All SRs published in March–May 2012 in 14 high-ranked medical journals and a sample from the Cochrane library were systematically assessed by two reviewers to determine if and how: critical appraisal was conducted; RoB was summarised at study, domain and review levels; and RoB appraisals informed the synthesis process.<p></p> Results: Of the 59 SRs studied, all except six (90%) conducted a critical appraisal of the included studies, with most using or adapting existing tools. Almost half of the SRs reported critical appraisal in a manner that did not allow readers to determine which studies included in a review were most robust. RoB assessments were not incorporated into synthesis in one-third (20) of the SRs, with their consideration more likely when reviews focused on randomised controlled trials. Common methods for incorporating critical appraisals into the synthesis process were sensitivity analysis, narrative discussion and exclusion of studies at high RoB. Nearly half of the reviews which investigated multiple outcomes and carried out study-level RoB summaries did not consider the potential for RoB to vary across outcomes.<p></p> Conclusions: The conclusions of the SRs, published in major journals, are frequently uninformed by the critical appraisal process, even when conducted. This may be particularly problematic for SRs of public health topics that often draw on diverse study designs

Psychosocial risk factors in home and community settings and their associations with population health and health inequalities: a systematic meta-review.

BACKGROUND: The effects of psychosocial risk factors on population health and health inequalities has featured prominently in epidemiological research literature as well as public health policy strategies. We have conducted a meta-review (a review of reviews) exploring how psychosocial factors may relate to population health in home and community settings. METHODS: Systematic review (QUORUM) of literature reviews (published in any language or country) on the health associations of psychosocial risk factors in community settings. The literature search included electronic and manual searches. Two reviewers appraised included reviews using criteria for assessing systematic reviews. Data from the more robust reviews were extracted, tabulated and synthesised. RESULTS: Thirty-one reviews met our inclusion criteria. These explored a variety of psychosocial factors including social support and networks, social capital, social cohesion, collective efficacy, participation in local organisations - and less favourable psychosocial risk factors such as demands, exposure to community violence or anti-social behaviour, exposure to discrimination, and stress related to acculturation to western society. Most of the reviews focused on associations between social networks/support and physical or mental health. We identified some evidence of favourable psychosocial environments associated with better health. Reviews also found evidence of unfavourable psychosocial risk factors linked to poorer health, particularly among socially disadvantaged groups. However, the more robust reviews each identified studies with inconclusive findings, as well as studies finding evidence of associations. We also identified some evidence of apparently favourable psychosocial risk factors associated with poorer health. CONCLUSION: From the review literature we have synthesised, where associations have been identified, they generally support the view that favourable psychosocial environments go hand in hand with better health. Poor psychosocial environments may be health damaging and contribute to health inequalities. The evidence that underpins our understanding of these associations is of variable quality and consistency. Future research should seek to improve this evidence base, with more longitudinal analysis (and intervention evaluations) of the effects of apparently under-researched psychosocial factors such as control and participation within communities. Future policy interventions relevant to this field should be developed in partnership with researchers to enable a better understanding of psychosocial mechanisms and the effects of psychosocial interventions

Population-level interventions to reduce alcohol-related harm: an overview of systematic reviews.

OBJECTIVE: To analyse available review-level evidence on the effectiveness of population-level interventions in non-clinical settings to reduce alcohol consumption or related health or social harm. METHOD: Health, social policy and specialist review databases between 2002 and 2012 were searched for systematic reviews of the effectiveness of population-level alcohol interventions on consumption or alcohol-related health or social outcomes. Data were extracted on review research aim, inclusion criteria, outcome indicators, results, conclusions and limitations. Reviews were quality-assessed using AMSTAR criteria. A narrative synthesis was conducted overall and by policy area. RESULTS: Fifty-two reviews were included from ten policy areas. There is good evidence for policies and interventions to limit alcohol sale availability, to reduce drink-driving, to increase alcohol price or taxation. There is mixed evidence for family- and community-level interventions, school-based interventions, and interventions in the alcohol server setting and the mass media. There is weak evidence for workplace interventions and for interventions targeting illicit alcohol sales. There is evidence of the ineffectiveness of interventions in higher education settings. CONCLUSION: There is a pattern of support from the evidence base for regulatory or statutory enforcement interventions over local non-regulatory approaches targeting specific population groups

Lost in translation? Theory, policy and practice in systems-based environmental approaches to obesity prevention in the Healthy Towns programme in England.

This paper explores how system-wide approaches to obesity prevention were 'theorised' and translated into practice in the 'Healthy Towns' programme implemented in nine areas in England. Semi-structured interviews with 20 informants, purposively selected to represent national and local programme development, management and delivery were undertaken. Results suggest that informants articulated a theoretical understanding of a system-wide approach to obesity prevention, but simplifying this complex task in the context of uncertainty over programme aims and objectives, and absence of a clear direction from the central government, resulted in local programmes relying on traditional multi-component approaches to programme delivery. The development of clear, practical guidance on implementation should form a central part of future system-wide approaches to obesity prevention

The harvest plot: a method for synthesising evidence about the differential effects of interventions.

BACKGROUND: One attraction of meta-analysis is the forest plot, a compact overview of the essential data included in a systematic review and the overall 'result'. However, meta-analysis is not always suitable for synthesising evidence about the effects of interventions which may influence the wider determinants of health. As part of a systematic review of the effects of population-level tobacco control interventions on social inequalities in smoking, we designed a novel approach to synthesis intended to bring aspects of the graphical directness of a forest plot to bear on the problem of synthesising evidence from a complex and diverse group of studies. METHODS: We coded the included studies (n = 85) on two methodological dimensions (suitability of study design and quality of execution) and extracted data on effects stratified by up to six different dimensions of inequality (income, occupation, education, gender, race or ethnicity, and age), distinguishing between 'hard' (behavioural) and 'intermediate' (process or attitudinal) outcomes. Adopting a hypothesis-testing approach, we then assessed which of three competing hypotheses (positive social gradient, negative social gradient, or no gradient) was best supported by each study for each dimension of inequality. RESULTS: We plotted the results on a matrix ('harvest plot') for each category of intervention, weighting studies by the methodological criteria and distributing them between the competing hypotheses. These matrices formed part of the analytical process and helped to encapsulate the output, for example by drawing attention to the finding that increasing the price of tobacco products may be more effective in discouraging smoking among people with lower incomes and in lower occupational groups. CONCLUSION: The harvest plot is a novel and useful method for synthesising evidence about the differential effects of population-level interventions. It contributes to the challenge of making best use of all available evidence by incorporating all relevant data. The visual display assists both the process of synthesis and the assimilation of the findings. The method is suitable for adaptation to a variety of questions in evidence synthesis and may be particularly useful for systematic reviews addressing the broader type of research question which may be most relevant to policymakers

How does money influence health?

This study looks at hundreds of theories to consider how income influences health. There is a graded association between money and health – increased income equates to better health. But the reasons are debated.<p></p> Researchers have reviewed theories from 272 wide-ranging papers, most of which examined the complex interactions between people’s income and their health throughout their lives.<p></p> Key points<p></p> This research identifies four main ways money affects people’s wellbeing:<p></p> Material: Money buys goods and services that improve health. The more money families have, the better the goods they can buy.<p></p> Psychosocial: Managing on a low income is stressful. Comparing oneself to others and feeling at the bottom of the social ladder can be distressing, which can lead to biochemical changes in the body, eventually causing ill health.<p></p> Behavioural: For various reasons, people on low incomes are more likely to adopt unhealthy behaviours – smoking and drinking, for example – while those on higher incomes are more able to afford healthier lifestyles.<p></p> Reverse causation (poor health leads to low income): Health may affect income by preventing people from taking paid employment. Childhood health may also affect educational outcomes, limiting job opportunities and potential earnings