Exploiting the potential of mixed methods studies in health services research.

Mixed methods studies, where qualitative and quantitative methods are used together in a single study, are undertaken in health services research (HSR). The question addressed here is whether researchers in HSR are fully exploiting the potential of mixed methods studies, and if not, then how they might maximise the potential of this approach. Methods used to examine this question included a review of the literature on mixed methods research; a quantitative documentary analysis of the research proposals, reports and publications of 75 mixed methods studies funded by ten Department of Health programmes in the period 1994 - 2004; and a qualitative study involving semi-structured face-to-face interviews with 20 researchers. It was evident from the documentary analysis that researchers are mixing methods in a range of different ways, with quantitative methods dominating, thus reflecting the conventional hierarchy of evidence in HSR. However, researchers could further exploit this approach by being clear about the purpose and practice of mixing methods when planning their studies, exploiting the contribution of qualitative components of studies, engaging with a wider range of ways of integrating data and findings from different components of a study, and being explicit in peerreviewed journal articles about any unique contribution made by this approach. Findings from the interviews with researchers suggest that researchers can contribute to fully exploiting the potential of mixed methods research by learning more about the different ways of integrating data and findings, respecting and understanding the strengths of the different methodological approaches, communicating with team members, and valuing integration. In HSR a multidisciplinary approach to team working is the norm whereby study components are undertaken separately. An interdisciplinary approach to team working is less common but may be associated with exploiting more of the potential of mixed methods studies. The external research environment appears to be conducive to interdisciplinary endeavour but not to interdisciplinary outputs. Structural change, as well as change in researcher behaviour, will be necessary if health services researchers are to fully exploit the potential of using mixed methods research

What do users value about the emergency ambulance service?

Introduction: Response times have been used as a key quality indicator for emergency ambulance services in the United Kingdom, but criticised for their narrow focus. Consequently, there is a need to consider wider measures of quality. The patient perspective is becoming an increasingly important dimension in pre-hospital outcomes research. To that end, we aimed to investigate patients’ experiences of the 999 ambulance service to understand the processes and outcomes important to them. Methods: We employed a qualitative design, using semi-structured interviews with a purposive sample of people who had recently used a 999 ambulance in the East Midlands. We recruited patients of different age, sex, geographical location, and ambulance service response including ‘hear and treat’, ‘see and treat’ and ‘see and convey’. Results: We interviewed 20 service users. Eleven men and nine women participated and 12 were aged 65 years and over. Users valued a quick response when they perceived the call to be an emergency. This was of less value to those who did not perceive their situation as an emergency and irrelevant to ‘hear and treat’ users. All users valued the professional approach and information and advice given by call handlers, crew and first responders, which provided them with reassurance in a worrying situation. ‘See and convey’ users valued a seamless handover to secondary care. Limitations: We found it challenging to engage participants to consider quality indicators beyond response times because these were considered to be abstract in comparison with their concrete experiences. Conclusions and recommendations: Aspects other than response times were important to patients, particularly in situations perceived by patients to be non-emergency. The results will be combined with issues identified from systematic reviews and used in a Delphi study to identify candidates for new outcome measures for emergency ambulance services

Developing interventions to improve health: a systematic mapping review of international practice between 2015 and 2016

Background Researchers publish the processes they use to develop interventions to improve health. Reflecting on this endeavour may help future developers to improve their practice. Methods Our aim was to collate, describe, and analyse the actions developers take when developing complex interventions to improve health. We carried out a systematic mapping review of empirical research studies that report the development of complex interventions to improve health. A search was undertaken of five databases over 2015–2016 using the term ‘intervention dev*’. Eighty-seven journal articles reporting the process of intervention development were identified. A purposive subset of 30 articles, using a range of published approaches to developing interventions, was selected for in-depth analysis using principles of realist synthesis to identify the actions of intervention development and rationales underpinning those actions. Results The 87 articles were from the USA (39/87), the UK (32/87), continental Europe (6/87), and the rest of the world (10/87). These mainly took a pragmatic self-selected approach (n = 43); a theory- and evidence-based approach, e.g. Intervention Mapping, Behaviour Change Wheel (n = 22); or a partnership approach, e.g. community-based participatory research, co-design (n = 10). Ten actions of intervention development were identified from the subset of 30 articles, including identifying a need for an intervention, selecting the intervention development approach to follow, considering the needs of the target population, reviewing published evidence, involving stakeholders, drawing or generating theory, and designing and refining the intervention. Rationales for these actions were that they would produce more engaging, acceptable, feasible, and effective interventions. Conclusions Developers take a variety of approaches to the international endeavour of complex intervention development. We have identified and described a set of actions taken within this endeavour regardless of whether developers follow a published approach or not. Future developers can use these actions and the rationales that underpin them to help them make decisions about the process of intervention development

Conducting qualitative research within Clinical Trials Units: Avoiding potential pitfalls

The value of using qualitative research within or alongside randomised controlled trials (RCTs) is becoming more widely accepted. Qualitative research may be conducted concurrently with pilot or full RCTs to understand the feasibility and acceptability of the interventions being tested, or to improve trial conduct. Clinical Trials Units (CTUs) in the United Kingdom (UK) manage large numbers of RCTs and, increasingly, manage the qualitative research or collaborate with qualitative researchers external to the CTU. CTUs are beginning to explicitly manage the process, for example, through the use of standard operating procedures for designing and implementing qualitative research with trials. We reviewed the experiences of two UK Clinical Research Collaboration (UKCRC) registered CTUs of conducting qualitative research concurrently with RCTs. Drawing on experiences gained from 15 studies, we identify the potential for the qualitative research to undermine the successful completion or scientific integrity of RCTs. We show that potential problems can arise from feedback of interim or final qualitative findings to members of the trial team or beyond, in particular reporting qualitative findings whilst the trial is on-going. The problems include: 1. Unplanned modifications of the trial intervention during the full RCT 2. Selection bias and threats to external validity 3. Unblinding of group allocation 4. Breach of participant anonymity and confidentiality in small RCTs 5. Unplanned modifications of the trial processes and procedures 6. Threats to completion of recruitment, retention and outcome measurement. We make recommendations for improving the management of qualitative research within CTU

How to incorporate patient and public perspectives into the design and conduct of research

International government guidance recommends patient and public involvement (PPI) to improve the relevance and quality of research. PPI is defined as research being carried out ‘with’ or ‘by’ patients and members of the public rather than ‘to’, ‘about’ or ‘for’ them (http://www.invo.org.uk/). Patient involvement is different from collecting data from patients as participants. Ethical considerations also differ. PPI is about patients actively contributing through discussion to decisions about research design, acceptability, relevance, conduct and governance from study conception to dissemination. Occasionally patients lead or do research. The research methods of PPI range from informal discussions to partnership research approaches such as action research, co-production and co-learning. This article discusses how researchers can involve patients when they are applying for research funding and considers some opportunities and pitfalls. It reviews research funder requirements, draws on the literature and our collective experiences as clinicians, patients, academics and members of UK funding panels

Prioritising outcomes measures for ambulance service care: a three stage consensus study

Background Historically ambulance care quality and performance has been measured by response times rather than clinical need or effectiveness. This limits the evidence about the effectiveness of the care the ambulance service provides. The Pre-hospital Outcomes for Evidence Based Evaluation (PhOEBE) project is a 5 year research programme which aims to develop new ways of measuring the performance, quality and impact of ambulance service care that better reflect the care provided. Methods We held a 1 day consensus event with a multi-stakeholder group to identify and prioritise which features of ambulance service care are important and worth further development as measures of performance and quality. The day comprised a series of small group discussions to share opinions about measures identified from literature reviews and the opportunity for participants to add their ideas, together with a live vote to help identify and rank measures for further development. Time measures were considered in a separate online questionnaire. The results from the online questionnaire and Consensus Event were fed into a Delphi study. Results 43 participants attended the consensus event, representing a range of participant groups including patient and public, ambulance services, commissioners and policy groups. The top 5 outcome measures prioritised by the participants were: Accuracy of dispatch decisions; Completeness and accuracy of patient records; Accuracy of call taker identification of different conditions or needs (e.g. heart attack, stroke, suitable for nurse advice); Pain measurement and symptom relief; Patient experience. From the online survey of time measures the top 5 measures were Time of call to time of arrival at scene; Time symptoms start (e.g. chest pain) to time of treatment with balloon for heart attack; Time taken to answer the emergency call; Total time spent on scene; Incident or start of symptoms to calling the ambulance service Conclusions Using consensus methods we have identified a set of outcome measures that can potentially be used to measure the performance and quality of ambulance service care. Some of these measures fit with existing government targets, such as response times and time taken to answer the emergency call. However most measures are new ways of measuring ambulance service care and the importance of these measures and methods of measurement will be assessed as part of an on-going Delphi study

Structural issues affecting mixed methods studies in health research: a qualitative study

<p>Abstract</p> <p>Background</p> <p>Health researchers undertake studies which combine qualitative and quantitative methods. Little attention has been paid to the structural issues affecting this mixed methods approach. We explored the facilitators and barriers to undertaking mixed methods studies in health research.</p> <p>Methods</p> <p>Face-to-face semi-structured interviews with 20 researchers experienced in mixed methods research in health in the United Kingdom.</p> <p>Results</p> <p>Structural facilitators for undertaking mixed methods studies included a perception that funding bodies promoted this approach, and the multidisciplinary constituency of some university departments. Structural barriers to exploiting the potential of these studies included a lack of education and training in mixed methods research, and a lack of templates for reporting mixed methods articles in peer-reviewed journals. The 'hierarchy of evidence' relating to effectiveness studies in health care research, with the randomised controlled trial as the gold standard, appeared to pervade the health research infrastructure. Thus integration of data and findings from qualitative and quantitative components of mixed methods studies, and dissemination of integrated outputs, tended to occur through serendipity and effort, further highlighting the presence of structural constraints. Researchers are agents who may also support current structures - journal reviewers and editors, and directors of postgraduate training courses - and thus have the ability to improve the structural support for exploiting the potential of mixed methods research.</p> <p>Conclusion</p> <p>The environment for health research in the UK appears to be conducive to mixed methods research but not to exploiting the potential of this approach. Structural change, as well as change in researcher behaviour, will be necessary if researchers are to fully exploit the potential of using mixed methods research.</p

Process evaluation of complex interventions: Medical Research Council guidance.

Process evaluation is an essential part of designing and testing complex interventions. New MRC guidance provides a framework for conducting and reporting process evaluation studiesThe work was funded by the MRC Population Health Science Research Network (PHSRN45).This is the final published version. It first appeared at http://www.bmj.com/content/350/bmj.h1258

Recording harms in randomized controlled trials of behavior change interventions: a scoping review and map of the evidence

Objectives: Randomized controlled trials evaluate diverse interventions. This can include medical interventions such as drugs or surgical procedures, or behavior change interventions (BCIs) that aim to change a habit, belief, or attitude to improve health, for example, healthy eating, psychological wellbeing. Harms are often recorded poorly or inconsistently within randomized controlled trials of BCIs. This scoping review aimed to collate and describe literature on categories, definitions, and mechanisms of harms from BCIs; methods of identifying plausible harms; and recommendations for recording harms. Study Design and Setting: A scoping review was conducted. Three databases (MEDLINE, PsycINFO, and CINAHL) were searched. Reference list checking and citation searching were performed. Articles were included if they discussed (1) interventions that aimed to modify behavior, (2) categories or mechanisms of harms, and (3) methods or recommendations for recording harms. All research designs were included. One reviewer reviewed titles, abstracts, and full texts; queries were checked with another reviewer. Data were extracted and synthesized descriptively by one reviewer and checked by another reviewer. A thematic map was constructed to summarize the review findings. Harms described from specific BCIs were identified, and examples were selected and summarized. Results: The review included 37 articles. Nineteen of 37 articles contributed to a thematic review. Three articles described categories of harms; categories of harm included physical, psychological, group and social interactions, cultural, equity, opportunity cost, environmental, and economic. Seven articles included mechanisms or underlying factors for harms including feelings of failure leading to shame or stigma, and group interventions enabling knowledge exchange on unhealthy behaviors. Twelve articles provided recommendations for recording harms, including taking a proportionate approach by focusing on the most plausible and important harms, collecting different perspectives on whether harms had occurred (eg, caregivers and family members), and using qualitative research methods to identify harms. One article described a three-step method to identify plausible harms from an intervention, and six articles supported aspects of the method. Eighteen of 37 articles contributed to a review which collated harms arising from specific interventions, for example, a peer support intervention in inflammatory bowel disease caused distressing conversations which might lead to anxiety and confrontation with a possible negative future. Conclusion: BCIs can cause harm. This review identified categories and proposed mechanisms of harms, as well as methods and recommendations for identifying and recording harms in BCIs for inclusion in forthcoming recommendations

Who does not participate in telehealth trials and why? A cross-sectional survey

BackgroundTelehealth interventions use information and communication technology to provide clinical support. Some randomised controlled trials of telehealth report high patient decline rates. A large study was undertaken to determine which patients decline to participate in telehealth trials and their reasons for doing so.MethodsTwo linked randomised controlled trials were undertaken, one for patients with depression and one for patients with raised cardiovascular disease risk (the Healthlines Study). The trials compared usual care with additional support delivered by the telephone and internet. Patients were recruited via their general practice and could return a form about why they were not participating.ResultsOf the patients invited, 82.9 % (20,021/24,152) did not accept the study invite, either by returning a decline form (n = 7134) or by not responding (n = 12,887). In both trials patients registered at deprived general practices were less likely to accept the study invite. Decline forms were received from 29.5 % (7134/24,152) of patients invited. There were four frequently reported types of reasons for declining. The most common was telehealth-related: 54.7 % (3889/,7115) of decliners said they did not have access or the skills to use the internet and/or computers. This was more prevalent amongst older patients and patients registered at deprived general practices. The second was health need-related: 40.1 % (n = 2852) of decliners reported that they did not need additional support for their health condition. The third was related to life circumstances: 27.2 % (n = 1932) of decliners reported being too busy. The fourth was research-related: 15.3 % (n = 1092) of decliners were not interested in the research.ConclusionsA large proportion of patients declining participation in these telehealth trials did so because they were unable to engage with telehealth or did not perceive a need for it. This has implications for engagement with telehealth in routine practice, as well as for trials, with a need to offer technological support to increase patients’ engagement with telehealth. More generally, triallists should assess why people decline to participate in their studies