Stakeholder views on publication bias in health services research

Objectives: While the presence of publication bias in clinical research is well documented, little is known about its role in the reporting of health services research. This paper explores stakeholder perceptions and experiences with regard to the role of publication and related biases in quantitative research relating to the quality, accessibility and organization of health services. Methods: We present findings from semi-structured interviews with those responsible for the funding, publishing and/or conduct of quantitative health services research, primarily in the UK. Additional data collection includes interviews with health care decision makers as ‘end users’ of health services research, and a focus group with patient and service user representatives. The final sample comprised 24 interviews and eight focus group participants. Results: Many study participants felt unable to say with any degree of certainty whether publication bias represents a significant problem in quantitative health services research. Participants drew broad contrasts between externally funded and peer reviewed research on the one hand, and end user funded quality improvement projects on the other, with the latter perceived as more vulnerable to selective publication and author over-claiming. Multiple study objectives, and a general acceptance of ‘mess and noise’ in the data and its interpretation was seen to reduce the importance attached to replicable estimates of effect sizes in health services research. The relative absence of external scrutiny, either from manufacturers of interventions or health system decision makers, added to this general sense of ‘low stakes’ of health services research. As a result, while many participants advocated study pre-registration and using protocols to pre-identify outcomes, others saw this as an unwarranted imposition. Conclusions: This study finds that incentives towards publication and related bias are likely to be present, but not to the same degree as in clinical research. In health services research, these were seen as being offset by other forms of ‘novelty’ bias in the reporting and publishing of research findings

Assessment of publication bias and outcome reporting bias in systematic reviews of health services and delivery research:A meta-epidemiological study

Strategies to identify and mitigate publication bias and outcome reporting bias are frequently adopted in systematic reviews of clinical interventions but it is not clear how often these are applied in systematic reviews relating to quantitative health services and delivery research (HSDR). We examined whether these biases are mentioned and/or otherwise assessed in HSDR systematic reviews, and evaluated associating factors to inform future practice. We randomly selected 200 quantitative HSDR systematic reviews published in the English language from 2007-2017 from the Health Systems Evidence database (www.healthsystemsevidence.org). We extracted data on factors that may influence whether or not authors mention and/or assess publication bias or outcome reporting bias. We found that 43% (n = 85) of the reviews mentioned publication bias and 10% (n = 19) formally assessed it. Outcome reporting bias was mentioned and assessed in 17% (n = 34) of all the systematic reviews. Insufficient number of studies, heterogeneity and lack of pre-registered protocols were the most commonly reported impediments to assessing the biases. In multivariable logistic regression models, both mentioning and formal assessment of publication bias were associated with: inclusion of a meta-analysis; being a review of intervention rather than association studies; higher journal impact factor, and; reporting the use of systematic review guidelines. Assessment of outcome reporting bias was associated with: being an intervention review; authors reporting the use of Grading of Recommendations, Assessment, Development and Evaluations (GRADE), and; inclusion of only controlled trials. Publication bias and outcome reporting bias are infrequently assessed in HSDR systematic reviews. This may reflect the inherent heterogeneity of HSDR evidence and different methodological approaches to synthesising the evidence, lack of awareness of such biases, limits of current tools and lack of pre-registered study protocols for assessing such biases. Strategies to help raise awareness of the biases, and methods to minimise their occurrence and mitigate their impacts on HSDR systematic reviews, are needed

Involving and engaging pregnant women in maternity-related research: Reflections on an innovative approach

Background: Meaningful public involvement in maternity research remains challenging, partly due to the transient nature of pregnancy. This paper reflects on the development, implementation and simple evaluation of an innovative and inclusive approach to engaging and involving pregnant and early postnatal women in research. Methods: Between January and February 2018, a Research Fellow in Maternity Care, a Professor of Evidence Based Maternity Care, and a Patient and Public Involvement Lead convened for a number of meetings to discuss how public involvement and engagement might be improved for pregnancy-related research. A stakeholder group was created, including a local community matron, a community engagement officer at a local children’s centre, public contributors, and senior members of the Maternal and Child Health theme of the West Midlands Collaboration for Leadership in Applied Health Research and Care (CLAHRC WM). The team worked together to develop a format for Yoga for Bump sessions: a free 90-min session, offered weekly, which included research involvement/engagement, pregnancy yoga, and a ‘question and answer’ session with a midwife. Results: A total of 67 women from two local communities in Birmingham attended Yoga for Bump sessions, which ran between May and December of 2018. Evaluation of the sessions suggested benefits to both women and researchers: it created mutually beneficial relationships between contributors and researchers, provided opportunities for women to engage and get involved in research that was directly relevant to them, and provided a convenient and efficient way for researchers to involve and engage pregnant women from diverse backgrounds in their research. Unintended benefits included self-reported improvements in women’s health and wellbeing. Conclusions: Yoga for Bump demonstrates an innovative approach to engaging and involving pregnant and early postnatal women; combining a free exercise class with healthcare advice and opportunities to engage with and be involved in research, and demonstrating mutual benefits for those involved. This model has the potential to be replicated elsewhere to support inclusive public involvement in pregnancy-related research. Further work is needed to design and evaluate similar approaches to involvement/engagement and explore potential funding avenues to enhance sustainability

Publication and related bias in quantitative health services & delivery research : systematic reviews, case studies, inception cohorts and informant interviews

Background Bias in the publication and reporting of research findings (referred to as publication and related bias here) poses a major threat in evidence synthesis and evidence-based decision-making. Although this bias has been well documented in clinical research, little is known about its occurrence and magnitude in health services and delivery research. Objectives To obtain empirical evidence on publication and related bias in quantitative health services and delivery research; to examine current practice in detecting/mitigating this bias in health services and delivery research systematic reviews; and to explore stakeholders’ perception and experiences concerning such bias. Methods The project included five distinct but interrelated work packages. Work package 1 was a systematic review of empirical and methodological studies. Work package 2 involved a survey (meta-epidemiological study) of randomly selected systematic reviews of health services and delivery research topics (n = 200) to evaluate current practice in the assessment of publication and outcome reporting bias during evidence synthesis. Work package 3 included four case studies to explore the applicability of statistical methods for detecting such bias in health services and delivery research. In work package 4 we followed up four cohorts of health services and delivery research studies (total n = 300) to ascertain their publication status, and examined whether publication status was associated with statistical significance or perceived ‘positivity’ of study findings. Work package 5 involved key informant interviews with diverse health services and delivery research stakeholders (n = 24), and a focus group discussion with patient and service user representatives (n = 8). Results We identified only four studies that set out to investigate publication and related bias in health services and delivery research in work package 1. Three of these studies focused on health informatics research and one concerned health economics. All four studies reported evidence of the existence of this bias, but had methodological weaknesses. We also identified three health services and delivery research systematic reviews in which findings were compared between published and grey/unpublished literature. These reviews found that the quality and volume of evidence and effect estimates sometimes differed significantly between published and unpublished literature. Work package 2 showed low prevalence of considering/assessing publication (43%) and outcome reporting (17%) bias in health services and delivery research systematic reviews. The prevalence was lower among reviews of associations than among reviews of interventions. The case studies in work package 3 highlighted limitations in current methods for detecting these biases due to heterogeneity and potential confounders. Follow-up of health services and delivery research cohorts in work package 4 showed positive association between publication status and having statistically significant or positive findings. Diverse views concerning publication and related bias and insights into how features of health services and delivery research might influence its occurrence were uncovered through the interviews with health services and delivery research stakeholders and focus group discussion conducted in work package 5. Conclusions This study provided prima facie evidence on publication and related bias in quantitative health services and delivery research. This bias does appear to exist, but its prevalence and impact may vary depending on study characteristics, such as study design, and motivation for conducting the evaluation. Emphasis on methodological novelty and focus beyond summative assessments may mitigate/lessen the risk of such bias in health services and delivery research. Methodological and epistemological diversity in health services and delivery research and changing landscape in research publication need to be considered when interpreting the evidence. Collection of further empirical evidence and exploration of optimal health services and delivery research practice are required

Views of the public about Snacktivity™: a small changes approach to promoting physical activity and reducing sedentary behaviour

Background: Many people do not meet the recommended health guidance of participation in a minimum of 150–300 min of moderate intensity physical activity per week, often promoted as at least 30 min of physical activity on 5 days of the week. This is concerning and highlights the importance of finding innovative ways to help people to be physically active each day. Snacktivity™ is a novel approach that aims to encourage people to do small, 2–5 min bouts of physical activity ‘snacks’ throughout the whole day, such that they achieve at least 150 min of moderate intensity activity per week. However, before it can be recommended, there is a need to explore whether the concept is acceptable to the public. Methods: A survey to assess the views of the public about Snacktivity™ was distributed to adult patients registered at six general practices in the West Midlands, UK and to health care employees in the same region. Results: A total of 5989 surveys were sent to patients, of which 558 were returned (9.3%). A further 166 surveys were completed by health care employees. A total of 85% of respondents liked the Snacktivity™ concept. The flexibility of the approach was highly rated. A high proportion of participants (61%) reported that the ability to self-monitor their behaviour would help them to do Snacktivity™ throughout their day. Physically inactive participants perceived that Snacktivity™ would help to increase their physical activity, more than those who were physically active (OR = 0.41, 95% CI: 0.25–0.67). Approximately 90% of respondents perceived that Snacktivity™ was easy to do on a non-working day compared to 60% on a working day. Aerobic activity ‘snacks’ were preferred to those which were strength based. Conclusions: The Snacktivity™ approach to promoting physical activity was viewed positively by the public and interventions to test the merits of such an approach now need to be developed and tested in a variety of everyday contexts

Snacktivity™ to promote physical activity and reduce future risk of disease in the population: protocol for a feasibility randomised controlled trial and nested qualitative study

Background: Many people do not regularly participate in physical activity, which may negatively impact their health. Current physical activity guidelines are focused on promoting weekly accumulation of at least 150 min of moderate to vigorous intensity physical activity (MVPA). Whilst revised guidance now recognises the importance of making small changes to physical activity behaviour, guidance still focuses on adults needing to achieve at least 150 min of MVPA per week. An alternative ‘whole day’ approach that could motivate the public to be more physically active, is a concept called Snacktivity™. Instead of focusing on achieving 150 min per week of physical activity, for example 30 min of MVPA over 5 days, Snacktivity™ encourages the public to achieve this through small, but frequent, 2–5 min ‘snacks’ of MVPA throughout the whole day. Methods: The primary aim is to undertake a feasibility trial with nested qualitative interviews to assess the feasibility and acceptability of the Snacktivity™ intervention to inform the design of a subsequent phase III randomised trial. A two-arm randomised controlled feasibility trial aiming to recruit 80 inactive adults will be conducted. Recruitment will be from health and community settings and social media. Participants will be individually randomised (1:1 ratio) to receive either the Snacktivity™ intervention or usual care. The intervention will last 12 weeks with assessment of outcomes completed before and after the intervention in all participants. We are interested in whether the Snacktivity™ trial is appealing to participants (assessed by the recruitment rate) and if the Snacktivity™ intervention and trial methods are acceptable to participants (assessed by Snacktivity™/physical activity adherence and retention rates). The intervention will be delivered by health care providers within health care consultations or by researchers. Participants’ experiences of the trial and intervention, and health care providers’ views of delivering the intervention within health consultations will be explored. Discussion: The development of physical activity interventions that can be delivered at scale are needed. The findings from this study will inform the viability and design of a phase III trial to assess the effectiveness and cost-effectiveness of Snacktivity™ to increase physical activity. Trial registration: ISRCTN: 64851242