Logic models help make sense of complexity in systematic reviews and health technology assessments

OBJECTIVE: To describe the development and application of logic model templates for systematic reviews and health technology assessments (HTA) of complex interventions STUDY DESIGN AND SETTING: This study demonstrates the development of a method to conceptualise complexity and make underlying assumptions transparent. Examples from systematic reviews with specific relevance to sub-Saharan Africa (SSA) and other low- and middle-income countries (LMICs) illustrate its usefulness. RESULTS: Two distinct templates are presented: the system-based logic model, describing the system in which the interaction between participants, intervention and context takes place; and the process-orientated logic model, which displays the processes and causal pathways that lead from the intervention to multiple outcomes. CONCLUSION: Logic models can help authors of systematic reviews and HTAs to explicitly address and make sense of complexity, adding value by achieving a better understanding of the interactions between the intervention, its implementation and its multiple outcomes among a given population and context. They thus have the potential to help build systematic review capacity -in SSA and other LMICs - at an individual level, by equipping authors with a tool that facilitates the review process; and at a system-level, by improving communication between producers and potential users of research evidence

Making sense of complexity in context and implementation: the Context and Implementation of Complex Interventions (CICI) framework.

BACKGROUND: The effectiveness of complex interventions, as well as their success in reaching relevant populations, is critically influenced by their implementation in a given context. Current conceptual frameworks often fail to address context and implementation in an integrated way and, where addressed, they tend to focus on organisational context and are mostly concerned with specific health fields. Our objective was to develop a framework to facilitate the structured and comprehensive conceptualisation and assessment of context and implementation of complex interventions. METHODS: The Context and Implementation of Complex Interventions (CICI) framework was developed in an iterative manner and underwent extensive application. An initial framework based on a scoping review was tested in rapid assessments, revealing inconsistencies with respect to the underlying concepts. Thus, pragmatic utility concept analysis was undertaken to advance the concepts of context and implementation. Based on these findings, the framework was revised and applied in several systematic reviews, one health technology assessment (HTA) and one applicability assessment of very different complex interventions. Lessons learnt from these applications and from peer review were incorporated, resulting in the CICI framework. RESULTS: The CICI framework comprises three dimensions-context, implementation and setting-which interact with one another and with the intervention dimension. Context comprises seven domains (i.e., geographical, epidemiological, socio-cultural, socio-economic, ethical, legal, political); implementation consists of five domains (i.e., implementation theory, process, strategies, agents and outcomes); setting refers to the specific physical location, in which the intervention is put into practise. The intervention and the way it is implemented in a given setting and context can occur on a micro, meso and macro level. Tools to operationalise the framework comprise a checklist, data extraction tools for qualitative and quantitative reviews and a consultation guide for applicability assessments. CONCLUSIONS: The CICI framework addresses and graphically presents context, implementation and setting in an integrated way. It aims at simplifying and structuring complexity in order to advance our understanding of whether and how interventions work. The framework can be applied in systematic reviews and HTA as well as primary research and facilitate communication among teams of researchers and with various stakeholders

Adapting evidence-informed population health interventions for new contexts: a scoping review of current practice

Background Implementing evidence-informed population health interventions in new contexts often requires adaptations. While the need to adapt interventions to better fit new contexts is recognised, uncertainties remain regarding why and when to adapt (or not), and how to assess the benefits (or not) of adaptation. The ADAPT Study aims to develop comprehensive guidance on adaptation. This scoping review informs guidance development by mapping and exploring how adaptation has been undertaken in practice, in public health and health services research. Methods We searched seven databases from January 2000 and October 2018 to identify eligible studies for this scoping review and a related systematic review of adaptation guidance. We mapped the studies of adaptation by coding data from all eligible studies describing the methods, contexts, and interventions considered for adaptation. From this map, we selected a sample of studies for in-depth examination. Two reviewers extracted data independently into seven categories: description, key concepts, types, rationale, processes, evaluation methods, evaluation justification, and accounts of failures and successes. Results We retrieved 6694 unique records. From 429 records screened at full text, we identified 298 eligible studies for mapping and selected 28 studies for in-depth examination. The majority of studies in our map focused on micro- (i.e., individual-) level interventions (84%), related to transferring an intervention to a new population group within the same country (62%) and did not report using guidance (73%). Studies covered a range of topic areas, including health behaviour (24%), mental health (19%), sexual health (16%), and parenting and family-centred interventions (15%). Our in-depth analysis showed that adaptation is seen to save costs and time relative to developing a new intervention, and to enhance contextual relevance and cultural compatibility. It commonly follows a structured process and involves stakeholders to help with decisions on what to adapt, when, and how. Conclusions Adaptation has been undertaken on a range of health topics and largely in line with existing guidance. Significant gaps relate to adaptation of macro- (e.g., national-) level interventions, consideration of programme theories, mechanisms and contexts (i.e., a functional view of interventions), nuances around stakeholder involvement, and evaluation of the adapted interventions. Registration Open Science Framework, 2019, osf.io/udzma

When and how do 'effective' interventions need to be adapted and/or re-evaluated in new contexts? The need for guidance

No abstract available

The Moral of the Tale: Stories, Trust, and Public Engagement with Clinical Ethics via Radio and Theatre.

Trust is frequently discussed with reference to the professional-patient relationship. However, trust is less explored in relation to the ways in which understanding of, and responses to, questions of ethics are discussed by both the "public" and "experts." Public engagement activity in healthcare ethics may invoke "trust" in analysing a moral question or problem but less frequently conceives of trust as integral to "public engagement" itself. This paper explores the relationship between trust and the ways in which questions of healthcare ethics are identified and negotiated by both "experts" and the public. Drawing on two examples from the author's "public engagement" work-a radio programme for the British Broadcasting Corporation and work with a playwright and theatre-the paper interrogates the ways in which "public engagement" is often characterized. The author argues that the common approach to public engagement in questions of ethics is unhelpfully constrained by a systemic disposition which continues to privilege the professional or expert voice at the expense of meaningful exchange and dialogue. By creating space for novel interactions between the "expert" and the "public," authentic engagement is achieved that enables not only the participants to flourish but also contributes to trust itself

Adapting interventions to new contexts—the ADAPT guidance

Implementing interventions with a previous evidence base in new contexts might be more efficient than developing new interventions for each context. Although some interventions transfer well, effectiveness and implementation often depend on the context. Achieving a good fit between intervention and context then requires careful and systematic adaptation. This paper presents new evidence and consensus informed guidance for adapting and transferring interventions to new contexts

The influence of contextual factors on healthcare quality improvement initiatives:what works, for whom and in what setting? Protocol for a realist review

Background  Context shapes the effectiveness of knowledge implementation and influences health improvement. Successful healthcare quality improvement (QI) initiatives frequently fail to transfer to different settings, with local contextual factors often cited as the cause. Understanding and overcoming contextual barriers is therefore crucial to implementing effective improvement; yet context is still poorly understood. There is a paucity of information on the mechanisms underlyinghowandwhyQI projects succeed or fail in given settings. A realist review of empirical studies of healthcare QI initiatives will be undertaken to examine the influence and impact of contextual factors on quality improvement in healthcare settings and explore whether QI initiatives can work in all contexts.  Methods  The review will explore which contextual factors are important, and how, why, when and for whom they are important, within varied settings. The dynamic nature of context and change over time will be explored by examining which aspects of context impact at key points in the improvement trajectory. The review will also consider the influence of context on improvement outcomes (provider- and patient-level), spread and sustainability. The review process will follow five iterative steps: (1) clarify scope, (2) search for evidence, (3) appraise primary studies and extract data, (4) synthesise evidence and draw conclusions and (5) disseminate findings. The reviewers will consult with experts and stakeholders in the early stages to focus the review and develop a programme theory consisting of explanatory ‘context–mechanism–outcome’ configurations. Searches for primary evidence will be conducted iteratively. Data will be extracted and tested against the programme theory. A review advisory group will oversee the review process. Review findings will follow RAMESES guidelines and will be disseminated via a report, presentations and peer-reviewed publications.  Discussion  The review will update and consolidate evidence on the contextual conditions for effective improvement and distil new knowledge to inform the design and development of context-sensitive QI initiatives. This review ties in with the study of improvement programmes as vehicles of change and the development of an evidence base around healthcare improvement by addressing whether QI initiatives can work in all contexts.  Systematic review registration  PROSPERO CRD4201706213

Adapting evidence-informed complex population health interventions for new contexts : a systematic review of guidance

Background Adapting interventions that have worked elsewhere can save resources associated with developing new interventions for each specific context. While a developing body of evidence shows benefits of adapted interventions compared with interventions transported without adaptation, there are also examples of interventions which have been extensively adapted, yet have not worked in the new context. Decisions on when, to what extent, and how to adapt interventions therefore are not straightforward, particularly when conceptualising intervention effects as contingent upon contextual interactions in complex systems. No guidance currently addresses these questions comprehensively. To inform development of an overarching guidance on adaptation of complex population health interventions, this systematic review synthesises the content of the existing guidance papers. Methods We searched for papers published between January 2000 and October 2018 in 7 bibliographic databases. We used citation tracking and contacted authors and experts to locate further papers. We double screened all the identified records. We extracted data into the following categories: descriptive information, key concepts and definitions, rationale for adaptation, aspects of adaptation, process of adaptation, evaluating and reporting adapted interventions. Data extraction was conducted independently by two reviewers, and retrieved data were synthesised thematically within pre-specified and emergent categories. Results We retrieved 6694 unique records. Thirty-eight papers were included in the review representing 35 sources of guidance. Most papers were developed in the USA in the context of implementing evidence-informed interventions among different population groups within the country, such as minority populations. We found much agreement on how the papers defined key concepts, aims, and procedures of adaptation, including involvement of key stakeholders, but also identified gaps in scope, conceptualisation, and operationalisation in several categories. Conclusions Our review found limitations that should be addressed in future guidance on adaptation. Specifically, future guidance needs to be reflective of adaptations in the context of transferring interventions across countries, including macro- (e.g. national-) level interventions, better theorise the role of intervention mechanisms and contextual interactions in the replicability of effects and accordingly conceptualise key concepts, such as fidelity to intervention functions, and finally, suggest evidence-informed strategies for adaptation re-evaluation and reporting

Embracing complexity and uncertainty to create impact: Exploring the processes and transformative potential of co-produced research through development of a social impact model

© 2018 The Author(s). The potential use, influence and impact of health research is seldom fully realised. This stubborn problem has caused burgeoning global interest in research aiming to address the implementation 'gap' and factors inhibiting the uptake of scientific evidence. Scholars and practitioners have questioned the nature of evidence used and required for healthcare, highlighting the complex ways in which knowledge is formed, shared and modified in practice and policy. This has led to rapid expansion, expertise and innovation in the field of knowledge mobilisation and funding for experimentation into the effectiveness of different knowledge mobilisation models. One approach gaining prominence involves stakeholders (e.g. researchers, practitioners, service users, policy-makers, managers and carers) in the co-production, and application, of knowledge for practice, policy and research (frequently termed integrated knowledge translation in Canada). Its popularity stems largely from its potential to address dilemmas inherent in the implementation of knowledge generated using more reductionist methods. However, despite increasing recognition, demands for co-produced research to illustrate its worth are becoming pressing while the means to do so remain challenging. This is due not only to the diversity of approaches to co-production and their application, but also to the ways through which different stakeholders conceptualise, measure, reward and use research. While research co-production can lead to demonstrable benefits such as policy or practice change, it may also have more diffuse and subtle impact on relationships, knowledge sharing, and in engendering culture shifts and research capacity-building. These relatively intangible outcomes are harder to measure and require new emphases and tools. This opinion paper uses six Canadian and United Kingdom case studies to explore the principles and practice of co-production and illustrate how it can influence interactions between research, policy and practice, and benefit diverse stakeholders. In doing so, we identify a continuum of co-production processes. We propose and illustrate the use of a new 'social model of impact' and framework to capture multi-layered and potentially transformative impacts of co-produced research. We make recommendations for future directions in research co-production and impact measurement