The Mixed Methods Appraisal Tool (MMAT) version 2018 for information professionals and researchers

INTRODUCTION: Appraising the quality of studies included in systematic reviews combining qualitative and quantitative evidence is challenging. To address this challenge, a critical appraisal tool was developed: the Mixed Methods Appraisal Tool (MMAT). The aim of this paper is to present the enhancements made to the MMAT. DEVELOPMENT: The MMAT was initially developed in 2006 based on a literature review on systematic reviews combining qualitative and quantitative evidence. It was subject to pilot and interrater reliability testing. A revised version of the MMAT was developed in 2018 based on the results from usefulness testing, a literature review on critical appraisal tools and a modified e-Delphi study with methodological experts to identify core criteria. TOOL DESCRIPTION: The MMAT assesses the quality of qualitative, quantitative, and mixed methods studies. It focuses on methodological criteria and includes five core quality criteria for each of the following five categories of study designs: (a) qualitative, (b) randomized controlled, (c) nonrandomized, (d) quantitative descriptive, and (e) mixed methods. CONCLUSION: The MMAT is a unique tool that can be used to appraise the quality of different study designs. Also, by limiting to core criteria, the MMAT can provide a more efficient appraisal

Explaining variation in emergency admissions: a mixed-methods study of emergency and urgent care systems

OBJECTIVE: To identify factors affecting variation in avoidable emergency admissions that are not usually identified in statistical regression. METHODS: As part of an ethnographic residual analysis, we compared six emergency and urgent care systems in England, interviewing 82 commissioners and providers of key emergency and urgent care services. RESULTS: There was variation between the six cases in how interviewees described three parts of their emergency and urgent care systems. First, interviewees' descriptions revealed variation in the availability of services before patients decided to attend emergency departments. Poor availability of general practice out of hours services in some of the cases reportedly made attendance at emergency departments the easier option for patients. Second, there was variation in how interviewees described patients being dealt with during their emergency department visit in terms of availability of senior review by specialists and in coding practices when patients were at risk of breaching the NHS's 4-hour waiting time target. Third, there was variability in services described as facilitating discharge home from emergency departments. In some cases, emergency department staff described dealing with multiple agencies in multiple localities outside the hospital, making admission the easier option. In other cases, proactive multidisciplinary rapid assessment teams were described as available to avoid admissions. Perceptions of resources available out of hours and the extent of integration between different health services, and between health and social services, also differed by case. CONCLUSIONS: This comparative case study approach identified further factors that may affect avoidable emergency admissions. Initiatives to improve GP out of hours services, make coding more accurately reflect patient experience, increase senior review in emergency departments, offer proactive multidisciplinary admission avoidance teams, improve the availability of out of hours care in the wider emergency and urgent care system, and increase service integration may reduce avoidable admissions. Evaluation of such initiatives would be necessary before wide-scale adoption

Attending to design when developing complex health interventions: A qualitative interview study with intervention developers and associated stakeholders

Background Guidance and frameworks exist to assist those developing health interventions but may offer limited discussion of ‘design’, the part of development concerned with generating ideas for and making decisions about an intervention’s content, format and delivery. The aim of this paper is to describe and understand the views and experiences of developers and associated stakeholders in relation to how design occurs in health intervention development. Methods Semi-structured interviews were conducted with 21 people who had developed complex interventions to improve health and/or who were relevant stakeholders (e.g. funders and publishers of intervention development work), regarding their views, experiences and approaches to intervention design. Sampling was purposive in terms of maximising diversity. A thematic inductive analysis was conducted. Results Approaches to design varied substantially between intervention developers. This contrasted with consistency in other activities undertaken during development, such as literature review. Design also posed more challenges than other parts of development. We identified six ‘modes’ of design: informed; negotiated; structured; delegated; ‘my baby’; and creative partnership. In understanding the differences between these different modes, and the challenges posed by intervention design, we identified three key themes: enabling creativity during the design process; working with different types of knowledge; and ‘stabilising’ (developing clear shared understandings of) the intervention development to enable design. Conclusions Design has received less attention than other activities undertaken when developing interventions to improve health. Developers take a variety of approaches to design and often find it challenging. Guidance for intervention development in health has tended to see design as proceeding in a predictable and controlled manner from acquired knowledge. Our study suggests that design rarely reflects this rational ideal. Future guidance on intervention development in healthcare should support developers to work effectively with different types of knowledge, to help design progress more smoothly and to maximise creativity

When is forgetting not forgetting? A discursive analysis of differences in forgetting talk between adults with cystic fibrosis with different levels of adherence to nebulizer treatments

Forgetting is often cited as a reason why people struggle to adhere to treatments for chronic conditions. Interventions have tried to improve forgetting behavior using reminders. We used a discursive psychological approach to explore differences in how high and low adherers constructed forgetting their nebulizer treatments for cystic fibrosis. Interviews were conducted with 18 adults from a cystic fibrosis center in the United Kingdom. High adherers constructed forgetting treatments as occasional lapses in automaticity and temporary lapses in memory that they found easy to repair. Low adherers utilized forgetting to normalize more consistent nonadherence to treatments. However, it is important to contextualize forgetting as a discursive resource that helped these participants to negotiate moral discourses around adherence to treatment that reminder interventions cannot address; we therefore recommend a more behavioral, patient-focused, theory-driven approach to intervention development

Not Managing Expectations: A Grounded Theory of Intimate Partner Violence From the Perspective of Pakistani People

Intimate partner violence (IPV) is a major social and public health problem affecting people from different cultures and societies. Much research has been undertaken to understand the phenomenon, its determinants, and its consequences in numerous countries. However, there is a paucity of research on IPV in many areas of the world including Pakistan. The present study aimed to develop a theory of the meaning and process of IPV from the perspective of Pakistani men and women living in and outside Pakistan

Challenges, solutions and future directions in the evaluation of service innovations in health care and public health

HeadlineEvaluating service innovations in health care and public health requires flexibility, collaboration and pragmatism; this collection identifies robust, innovative and mixed methods to inform such evaluations.This is the final version of the article. It first appeared from the National Institute for Helath Research via http://dx.doi.org/10.3310/hsdr0416

Implementing street triage: a qualitative study of collaboration between police and mental health services.

BACKGROUND: Street Triage is a collaborative service between mental health workers and police which aims to improve the emergency response to individuals experiencing crisis, but peer reviewed evidence of the effectiveness of these services is limited. We examined the design and potential impact of two services, along with factors that hindered and facilitated the implementation of the services. METHODS: We conducted 14 semi-structured interviews with mental health and police stakeholders with experience of a Street Triage service in two locations of the UK. Framework analysis identified themes related to key aspects of the Street Triage service, perceived benefits of Street Triage, and ways in which the service could be developed in the future. RESULTS: Stakeholders endorsed the Street Triage services which utilised different operating models. These models had several components including a joint response vehicle or a mental health worker in a police control room. Operating models were developed with consideration of the local geographical and population density. The ability to make referrals to the existing mental health service was perceived as key to the success of the service yet there was evidence to suggest Street Triage had the potential to increase pressure on already stretched mental health and police services. Identifying staff with skills and experience for Street Triage work was important, and their joint response resulted in shared decision making which was less risk averse for the police and regarded as in the interest of patient care by mental health professionals. Collaboration during Street Triage improved the understanding of roles and responsibilities in the 'other' agency and led to the development of local information sharing agreements. Views about the future direction of the service focused on expansion of Street Triage to address other shared priorities such as frequent users of police and mental health services, and a reduction in the police involvement in crisis response. CONCLUSION: The Street Triage service received strong support from stakeholders involved in it. Referral to existing health services is a key function of Street Triage, and its impact on referral behaviour requires rigorous evaluation. Street Triage may result in improvement to collaborative working but competing demands for resources within mental health and police services presented challenges for implementation

Factors perceived to influence risky sexual behaviours among university students in the United Kingdom: a qualitative telephone interview study

Background In the United Kingdom people under the age of 25 years are at increased risk of contracting sexually transmitted infections. Most university students belong to this age group but little is known about their sexual behaviours. The aim of the study was to explore university students’ perspectives of factors and mechanisms that influence risky sexual behaviours among university students in the United Kingdom. Methods All students at a university in a northern city of England were invited via email to participate in qualitative telephone interviews. Interviews were audio recorded and transcribed verbatim. Framework analytical approach was used. Results Twenty interviews were conducted with a diverse sample of students. The social context of university lifestyle was perceived to affect risky sex through high levels of alcohol consumption, increased sexual opportunities, liberation from moral surveillance and expectations of the stereotypical highly sexually active student. Individual and cultural differences were also perceived to account for some patterns of risky sex with older students, overseas students and religious students perceived to be less likely to engage in risky sex due to academic priorities and a tendency to be more likely to adhere to moral values. Risk denial was also a key factor that led students to engage in risky sex. Poor access to sexual health services including inconvenient opening times, lack of confidentiality and stigma were perceived to contribute to the limited use of sexually transmitted infections testing and contraceptive services. Conclusions Lifestyle, individual and structural factors seem to play an important role in influencing the risky sexual behaviours of university students. Therefore preventive interventions that focus on these factors could be very useful in this sub-population of young people. This study provides useful baseline information that helps us understand how and why some United Kingdom university students engage in risky sexual behaviours that puts them at risk of contracting sexually transmitted infections

Socioeconomic inequalities in mortality, morbidity and diabetes management for adults with type 1 diabetes: A systematic review.

AIMS: To systematically review the evidence of socioeconomic inequalities for adults with type 1 diabetes in relation to mortality, morbidity and diabetes management. METHODS: We carried out a systematic search across six relevant databases and included all studies reporting associations between socioeconomic indicators and mortality, morbidity, or diabetes management for adults with type 1 diabetes. Data extraction and quality assessment was undertaken for all included studies. A narrative synthesis was conducted. RESULTS: A total of 33 studies were identified. Twelve cohort, 19 cross sectional and 2 case control studies met the inclusion criteria. Regardless of healthcare system, low socioeconomic status was associated with poorer outcomes. Following adjustments for other risk factors, socioeconomic status was a statistically significant independent predictor of mortality in 9/10 studies and morbidity in 8/10 studies for adults with type 1 diabetes. There appeared to be an association between low socioeconomic status and some aspects of diabetes management. Although only 3 of 16 studies made adjustments for confounders and other risk factors, poor diabetes management was associated with lower socioeconomic status in 3/3 of these studies. CONCLUSIONS: Low socioeconomic status is associated with higher levels of mortality and morbidity for adults with type 1 diabetes even amongst those with access to a universal healthcare system. The association between low socioeconomic status and diabetes management requires further research given the paucity of evidence and the potential for diabetes management to mitigate the adverse effects of low socioeconomic status

An intervention to support adherence to inhaled medication in adults with cystic fibrosis: the ACtiF research programme including RCT

Background People with cystic fibrosis frequently have low levels of adherence to inhaled medications. Objectives The objectives were to develop and evaluate an intervention for adults with cystic fibrosis to improve adherence to their inhaled medication. Design We used agile software methods to develop an online platform. We used mixed methods to develop a behaviour change intervention for delivery by an interventionist. These were integrated to become the CFHealthHub intervention. We undertook a feasibility study consisting of a pilot randomised controlled trial and process evaluation in two cystic fibrosis centres. We evaluated the intervention using an open-label, parallel-group randomised controlled trial with usual care as the control. Participants were randomised in a 1 : 1 ratio to intervention or usual care. Usual care consisted of clinic visits every 3 months. We undertook a process evaluation alongside the randomised controlled trial, including a fidelity study, a qualitative interview study and a mediation analysis. We undertook a health economic analysis using both a within-trial and model-based analysis. Setting The randomised controlled trial took place in 19 UK cystic fibrosis centres. Participants Participants were people aged ≥ 16 years with cystic fibrosis, on the cystic fibrosis registry, not post lung transplant or on the active transplant list, who were able to consent and not using dry-powder inhalers. Intervention People with cystic fibrosis used a nebuliser with electronic monitoring capabilities. This transferred data automatically to a digital platform. People with cystic fibrosis and clinicians could monitor adherence using these data, including through a mobile application (app). CFHealthHub displayed graphs of adherence data as well as educational and problem-solving information. A trained interventionist helped people with cystic fibrosis to address their adherence. Main outcome measures Randomised controlled trial – adjusted incidence rate ratio of pulmonary exacerbations meeting the modified Fuchs criteria over a 12-month follow-up period (primary outcome); change in percentage adherence; and per cent predicted forced expiratory volume in 1 second (key secondary outcomes). Process evaluation – percentage fidelity to intervention delivery, and participant and interventionist perceptions of the intervention. Economic modelling – incremental cost per quality-adjusted life-year gained. Results Randomised controlled trial – 608 participants were randomised to the intervention (n = 305) or usual care (n = 303). To our knowledge, this was the largest randomised controlled trial in cystic fibrosis undertaken in the UK. The adjusted rate of exacerbations per year (primary outcome) was 1.63 in the intervention and 1.77 in the usual-care arm (incidence rate ratio 0.96, 95% confidence interval 0.83 to 1.12; p = 0.638) after adjustment for covariates. The adjusted difference in mean weekly normative adherence was 9.5% (95% confidence interval 8.6% to 10.4%) across 1 year, favouring the intervention. Adjusted mean difference in forced expiratory volume in 1 second (per cent) predicted at 12 months was 1.4% (95% confidence interval –0.2% to 3.0%). No adverse events were related to the intervention. Process evaluation – fidelity of intervention delivery was high, the intervention was acceptable to people with cystic fibrosis, participants engaged with the intervention [287/305 (94%) attended the first intervention visit], expected mechanisms of action were identified and contextual factors varied between randomised controlled trial sites. Qualitative interviews with 22 people with cystic fibrosis and 26 interventionists identified that people with cystic fibrosis welcomed the objective adherence data as proof of actions to self and others, and valued the relationship that they built with the interventionists. Economic modelling – the within-trial analysis suggests that the intervention generated 0.01 additional quality-adjusted life-years at an additional cost of £865.91 per patient, leading to an incremental cost-effectiveness ratio of £71,136 per quality-adjusted life-year gained. This should be interpreted with caution owing to the short time horizon. The health economic model suggests that the intervention is expected to generate 0.17 additional quality-adjusted life-years and cost savings of £1790 over a lifetime (70-year) horizon; hence, the intervention is expected to dominate usual care. Assuming a willingness-to-pay threshold of £20,000 per quality-adjusted life-year gained, the probability that the intervention generates more net benefit than usual care is 0.89. The model results are dependent on assumptions regarding the duration over which costs and effects of the intervention apply, the impact of the intervention on forced expiratory volume in 1 second (per cent) predicted and the relationship between increased adherence and drug-prescribing levels. Limitations Number of exacerbations is a sensitive and valid measure of clinical change used in many trials. However, data collection of this outcome in this context was challenging and could have been subject to bias. It was not possible to measure baseline adherence accurately. It was not possible to quantify the impact of the intervention on the number of packs of medicines prescribed. Conclusions We developed a feasible and acceptable intervention that was delivered to fidelity in the randomised controlled trial. We observed no statistically significant difference in the primary outcome of exacerbation rates over 12 months. We observed an increase in normative adherence levels in a disease where adherence levels are low. The magnitude of the increase in adherence may not have been large enough to affect exacerbations