A mixed-methods study of challenges experienced by clinical teams in measuring improvement

Objective: Measurement is an indispensable element of most quality improvement (QI) projects, but it is undertaken to variable standards. We aimed to characterise challenges faced by clinical teams in undertaking measurement in the context of a safety QI programme that encouraged local selection of measures. Methods: Drawing on an independent evaluation of a multi-site improvement programme (Safer Clinical Systems), we combined a qualitative study of participating teams’ experiences and perceptions of measurement with expert review of measurement plans and analysis of data collected for the programme. Multidisciplinary teams of frontline clinicians at nine UK NHS sites took part across the two phases of the programme between 2011 and 2016. Results: Developing and implementing a measurement plan against which to assess their improvement goals was an arduous task for participating sites. The operational definitions of the measures that they selected were often imprecise or missed important details. Some measures used by the teams were not logically linked to the improvement actions they implemented. Regardless of the specific type of data used (routinely collected or selected ex-novo), the burdensome nature of data collection was underestimated. Problems also emerged in identifying and using suitable analytical approaches. Conclusion: Measurement is a highly technical task requiring a degree of expertise. Simply leveraging individual clinicians’ motivation is unlikely to defeat the persistent difficulties experienced by clinical teams when attempting to measure their improvement efforts. We suggest that more structural initiatives and broader capability-building programmes should be pursued by the professional community. Improving access to, and ability to use repositories of validated measures, and increasing transparency in reporting measurement attempts, is likely to be helpful

‘Trying to pin down jelly’ - exploring intuitive processes in quality assessment for meta-ethnography

Background: Studies that systematically search for and synthesise qualitative research are becoming more evident in health care, and they can make an important contribution to patient care. However, there is still no agreement as to whether, or how we should appraise studies for inclusion. We aimed to explore the intuitive processes that determined the ‘quality’ of qualitative research for inclusion in qualitative research syntheses. We were particularly interested to explore the way that knowledge was constructed. Methods: We used qualitative methods to explore the process of quality appraisal within a team of seven qualitative researchers funded to undertake a meta-ethnography of chronic non-malignant musculoskeletal pain. Team discussions took place monthly between October 2010 and June 2012 and were recorded and transcribed. Data was coded and organised using constant comparative method. The development of our conceptual analysis was both iterative and collaborative. The strength of this team approach to quality came from open and honest discussion, where team members felt free to agree, disagree, or change their position within the safety of the group. Results: We suggest two core facets of quality for inclusion in meta-ethnography - (1) Conceptual clarity; how clearly has the author articulated a concept that facilitates theoretical insight. (2) Interpretive rigour; fundamentally, can the interpretation ‘be trusted?’ Our findings showed that three important categories help the reader to judge interpretive rigour: (ii) What is the context of the interpretation? (ii) How inductive is the interpretation? (iii) Has the researcher challenged their interpretation? Conclusions: We highlight that methods alone do not determine the quality of research for inclusion into a meta-ethnography. The strength of a concept and its capacity to facilitate theoretical insight is integral to meta-ethnography, and arguably to the quality of research. However, we suggest that to be judged ‘good enough’ there also needs to be some assurance that qualitative findings are more than simply anecdotal. Although our conceptual model was developed specifically for meta-ethnography, it may be transferable to other research methodologies

A qualitative study of speaking out about patient safety concerns in intensive care units

Much policy focus has been afforded to the role of “whistleblowers” in raising concerns about quality and safety of patient care in healthcare settings. However, most opportunities for personnel to identify and act on these concerns are likely to occur much further upstream, in the day-to-day mundane interactions of everyday work. Using qualitative data from over 900 h of ethnographic observation and 98 interviews across 19 English intensive care units (ICUs), we studied how personnel gave voice to concerns about patient safety or poor practice. We observed much low-level social control occurring as part of day-to-day functioning on the wards, with challenges and sanctions routinely used in an effort to prevent or address mistakes and norm violations. Pre-emptions were used to intervene when patients were at immediate risk, and included strategies such as gentle reminders, use of humour, and sharp words. Corrective interventions included education and evidence-based arguments, while sanctions that were applied when it appeared that a breach of safety had occurred included “quiet words”, bantering, public exposure or humiliation, scoldings and brutal reprimands. These forms of social control generally functioned effectively to maintain safe practice. However, they were not consistently effective, and sometimes risked reinforcing norms and idiosyncratic behaviours that were not necessarily aligned with goals of patient safety and high-quality healthcare. Further, making challenges across professional boundaries or hierarchies was sometimes problematic. Our findings suggest that an emphasis on formal reporting or communication training as the solution to giving voice to safety concerns is simplistic; a more sophisticated understanding of social control is needed.This study was funded by the Health Foundation, charity number 286967, and by Mary Dixon-Woods' Wellcome Trust Senior Investigator Award (WT097899)

Using framework-based synthesis for conducting reviews of qualitative studies

Framework analysis is a technique used for data analysis in primary qualitative research. Recent years have seen its being adapted to conduct syntheses of qualitative studies. Framework-based synthesis shows considerable promise in addressing applied policy questions. An innovation in the approach, known as 'best fit' framework synthesis, has been published in BMC Medical Research Methodology this month. It involves reviewers in choosing a conceptual model likely to be suitable for the question of the review, and using it as the basis of their initial coding framework. This framework is then modified in response to the evidence reported in the studies in the reviews, so that the final product is a revised framework that may include both modified factors and new factors that were not anticipated in the original model. 'Best fit' framework-based synthesis may be especially suitable in addressing urgent policy questions where the need for a more fully developed synthesis is balanced by the need for a quick answer

Scoping studies: towards a methodological framework

This paper focuses on scoping studies, an approach to reviewing the literature which to date has received little attention in the research methods literature. We distinguish between different types of scoping studies and indicate where these stand in relation to full systematic reviews. We outline a framework for conducting a scoping study based on our recent experiences of reviewing the literature on services for carers for people with mental health problems. Where appropriate, our approach to scoping the field is contrasted with the procedures followed in systematic reviews. We emphasize how including a consultation exercise in this sort of study may enhance the results, making them more useful to policy makers, practitioners and service users. Finally, we consider the advantages and limitations of the approach and suggest that a wider debate is called for about the role of the scoping study in relation to other types of literature reviews

Methods for the thematic synthesis of qualitative research in systematic reviews

<p>Abstract</p> <p>Background</p> <p>There is a growing recognition of the value of synthesising qualitative research in the evidence base in order to facilitate effective and appropriate health care. In response to this, methods for undertaking these syntheses are currently being developed. Thematic analysis is a method that is often used to analyse data in primary qualitative research. This paper reports on the use of this type of analysis in systematic reviews to bring together and integrate the findings of multiple qualitative studies.</p> <p>Methods</p> <p>We describe thematic synthesis, outline several steps for its conduct and illustrate the process and outcome of this approach using a completed review of health promotion research. Thematic synthesis has three stages: the coding of text 'line-by-line'; the development of 'descriptive themes'; and the generation of 'analytical themes'. While the development of descriptive themes remains 'close' to the primary studies, the analytical themes represent a stage of interpretation whereby the reviewers 'go beyond' the primary studies and generate new interpretive constructs, explanations or hypotheses. The use of computer software can facilitate this method of synthesis; detailed guidance is given on how this can be achieved.</p> <p>Results</p> <p>We used thematic synthesis to combine the studies of children's views and identified key themes to explore in the intervention studies. Most interventions were based in school and often combined learning about health benefits with 'hands-on' experience. The studies of children's views suggested that fruit and vegetables should be treated in different ways, and that messages should not focus on health warnings. Interventions that were in line with these suggestions tended to be more effective. Thematic synthesis enabled us to stay 'close' to the results of the primary studies, synthesising them in a transparent way, and facilitating the explicit production of new concepts and hypotheses.</p> <p>Conclusion</p> <p>We compare thematic synthesis to other methods for the synthesis of qualitative research, discussing issues of context and rigour. Thematic synthesis is presented as a tried and tested method that preserves an explicit and transparent link between conclusions and the text of primary studies; as such it preserves principles that have traditionally been important to systematic reviewing.</p

Changing the narratives for patient safety

Mary Dixon-Woods is supported by a Wellcome Trust Senior Investigator award (WT097899)