The Adapt Study: adaptation of evidence-informed complex population health interventions for implementation and/or re-evaluation in new contexts: protocol for a Delphi consensus exercise to develop guidance.

Introduction: Complex population health interventions that are effective in one context may not be effective elsewhere, and may even be harmful. As such, an intervention may require adaptation to ensure it fits with a new context. To date, there is no overarching guidance to help researchers to adapt and evaluate interventions in new contexts, and no criteria to support research funders or journals assess proposed or reported adaptations or evaluation. There is limited assistance for policy-makers and practitioners to decide if evidence-informed interventions are appropriate to their context, or if adaptation and further evaluation is needed. This Delphi exercise will contribute to the development of guidance for these communities to support the adaptation, implementation and/or re-evaluation of complex population health interventions in new contexts. Methods: We will conduct a Delphi consensus exercise to gather expert opinion from researchers, research funders, journal editors and policy-makers. Expert opinion will be sought on: appropriate definitions and concepts, identifying key methodological considerations and establishing adaptations and processes to be undertaken during adaptation of complex population health interventions in new contexts. Ethics and dissemination: Ethics approval for the Delphi exercise has been obtained from the University of Glasgow and and the RAND institutional research board. Dissemination of the results of this study will be through peer-reviewed publications, workshops at national and international conferences, and a summary of the guidance developed for key organisations and stakeholders

ADAPT study: adaptation of evidence-informed complex population health interventions for implementation and/or re-evaluation in new contexts: protocol for a Delphi consensus exercise to develop guidance

Introduction Complex population health interventions that are effective in one context may not be effective elsewhere, and may even be harmful. As such, an intervention may require adaptation to ensure it fits with a new context. To date, there is no overarching guidance to help researchers to adapt and evaluate interventions in new contexts, and no criteria to support research funders or journals assess proposed or reported adaptations or evaluation. There is limited assistance for policy-makers and practitioners to decide if evidence-informed interventions are appropriate to their context, or if adaptation and further evaluation is needed. This Delphi exercise will contribute to the development of guidance for these communities to support the adaptation, implementation and/or re-evaluation of complex population health interventions in new contexts. Methods We will conduct a Delphi consensus exercise to gather expert opinion from researchers, research funders, journal editors and policy-makers. Expert opinion will be sought on: appropriate definitions and concepts, identifying key methodological considerations and establishing adaptations and processes to be undertaken during adaptation of complex population health interventions in new contexts. Ethics and dissemination Ethics approval for the Delphi exercise has been obtained from the University of Glasgow and and the RAND institutional research board. Dissemination of the results of this study will be through peer-reviewed publications, workshops at national and international conferences, and a summary of the guidance developed for key organisations and stakeholders

Prioritizing Needs and Outcomes for Adolescent Substance Use Treatment Planning: An Online Modified-Delphi Process

Key stakeholders can have differing views about which information is essential to inform placement decisions for all patients. This study examined consensus across stakeholder groups on the most important individual needs and treatment outcomes for informing decisions specifically about the level of care for an adolescent in substance use treatment. Methods: We conducted an online modified-Delphi process with treatment providers, policymakers, researchers, and parents of adolescents who have received substance use treatment. Participants rated 48 individual needs from the Global Appraisal of Individual Needs–Initial that were mapped onto the 6 dimensions of the American Society of Addiction Medicine Criteria. In addition, participants rated 10 treatment outcomes from the Substance Abuse and Mental Health Services Administration's National Outcome Measures. We assessed consensus within stakeholder groups using the RAND/UCLA Appropriateness Method. We considered the items reaching consensus with the highest ratings across stakeholder groups as the most important individual needs and treatment outcomes. Results: We recruited 194 participants (81 providers, 54 policymakers, 32 researchers, 27 parents). Participants identified suicidality and severity of substance use disorder symptoms as the most important individual needs, and reduction in substance use as the most important treatment outcome. Conclusions: Standardized procedures for matching adolescents to levels of care for substance use treatment should at a minimum be based on assessments of suicidality and severity of substance use disorder symptoms, and consider reduction in substance use as a primary treatment outcome. These findings can progress the development of “level-of-care” decision rules specifically for adolescents

Participant Experiences with a New Online Modified-Delphi Approach for Engaging Patients and Caregivers in Developing Clinical Guidelines

Background Patient engagement in clinical practice guideline (CPG) development has increased significantly in recent years. However, only a few patients and caregivers join CPG development groups. Objective To describe participant experiences with a novel online, scalable approach for patient and caregiver engagement in CPG development. Design We developed and tested the RAND/PPMD Patient-Centeredness Method (RPM), a novel online modified-Delphi approach to patient engagement in CPG development that consists of an optional idea generation round and two rating rounds interspersed with an online discussion round. Setting and Participants Using the online ExpertLens™ system, we ran 2 concurrent panels of patients and caregivers of individuals with Duchenne Muscular Dystrophy (DMD). Main Outcome Measures We surveyed all 95 panel participants about their participation and satisfaction with the process. We also conducted telephone interviews with 25 participants. Results Participants expressed satisfaction with various ExpertLens™ features, noting that the system fostered lively interaction among them. Panelists also appreciated participating in an educational, interactive and convenient discussion forum that allowed them to share their opinions on the importance and acceptability of different recommendations. The RPM was viewed as empowering by patients and their caregivers who felt it would be useful for CPG developers. Discussion and Conclusion The results of our study show the overall participant satisfaction with a novel, scalable, online approach to engaging patients and caregivers in CPG development, which allows them to share their perspectives and lived experiences using a rigorous, systematic and iterative way that is similar to how clinicians provide their input

The RAND/PPMD Patient-Centeredness Method: a novel online approach to engaging patients and their representatives in guideline development

Although clinical practice guidelines (CPGs) provide recommendations for how best to treat a typical patient with a given condition, patients and their representatives are not always engaged in CPG development. Despite the agreement that patient participation may improve the quality and utility of CPGs, there is no systematic, scalable method for engaging patients and their representatives, as well as no consensus on what exactly patients and their representatives should be asked to do during CPG development. To address these gaps, an interdisciplinary team of researchers, patient representatives, and clinicians developed the RAND/PPMD Patient-Centeredness Method (RPM) - a novel online approach to engaging patients and their representatives in CPG development. The RPM is an iterative approach that allows patients and their representatives to provide input by (1) generating ideas; (2) rating draft recommendations on two criteria (importance and acceptability); (3) explaining and discussing their ratings with other participants using online, asynchronous, anonymous, moderated discussion boards, and (4) revising their responses if needed. The RPM was designed to be consistent with the RAND/UCLA Appropriateness Method used by clinicians and researchers to develop CPG, while helping patients and their representative rate outcome importance and recommendation acceptability - two key components of the GRADE Evidence to Decision (EtD) framework. With slight modifications, the RPM has the potential to explore consensus among key stakeholders on other dimensions of the EtD, including feasibility, equity, and resource use

Does a quality improvement campaign accelerate take-up of new evidence? A ten-state cluster-randomized controlled trial of the Institute for Health Improvement’s Project JOINTS

Background: A decade ago, the Institute for Healthcare Improvement pioneered a quality improvement (QI) campaign, leveraging organizational and personal social networks to disseminate new practices. There have been few rigorous studies of the QI campaign approach. Methods: Project JOINTS (Joining Organizations IN Tackling SSIs) engaged a network of state-based organizations and professionals in a 6-month QI campaign promoting adherence to three new evidence-based practices known to reduce the risk of infection after joint replacement. We conducted a cluster-randomized trial including ten states (five campaign states and five non-campaign states) with 188 hospitals providing joint replacement to Medicare. We measured adherence to the evidence-based practices before and after the campaign using a survey of surgical staff and a difference-in-difference design with multivariable adjustment to compare adherence to each of the relevant practices and an all-or-none composite measure of the three new practices. Results: In the campaign states, there were statistically significant increases in adherence to the three new evidence-based practices promoted by the campaign. Compared to the non-campaign states, the relative increase in adherence to the three new practices in the campaign states ranged between 1.9 and 15.9 percentage points, but only one of these changes (pre-operative nasal screening for Staphylococcus aureus carriage and decolonization prior to surgery) was statistically significant (p < 0.05). On the all-or-none composite measure, adherence to all three evidence-based practices increased from 19.6 to 37.9% in the campaign states, but declined slightly in the comparison states, yielding a relative increase of 23 percentage points (p = 0.004). In the non-campaign states, changes in adherence were not statistically significant. Conclusions: Within 6 months, in a cluster-randomized trial, a multi-state campaign targeting hospitals and professionals involved in surgical care and infection control was associated with an increase in adherence to evidence-based practices that can reduce surgical site infection. Electronic supplementary material The online version of this article (doi:10.1186/s13012-017-0579-7) contains supplementary material, which is available to authorized users

Erratum to: Does a quality improvement campaign accelerate take-up of new evidence? A ten-state cluster-randomized controlled trial of the IHI’s Project JOINTS

Abstract Background A decade ago, the Institute for Healthcare Improvement pioneered a quality improvement (QI) campaign, leveraging organizational and personal social networks to disseminate new practices. There have been few rigorous studies of the QI campaign approach. Methods Project JOINTS (Joining Organizations IN Tackling SSIs) engaged a network of state-based organizations and professionals in a 6-month QI campaign promoting adherence to three new evidence-based practices known to reduce the risk of infection after joint replacement. We conducted a cluster-randomized trial including ten states (five campaign states and five non-campaign states) with 188 hospitals providing joint replacement to Medicare. We measured adherence to the evidence-based practices before and after the campaign using a survey of surgical staff and a difference-in-difference design with multivariable adjustment to compare adherence to each of the relevant practices and an all-or-none composite measure of the three new practices. Results In the campaign states, there were statistically significant increases in adherence to the three new evidence-based practices promoted by the campaign. Compared to the non-campaign states, the relative increase in adherence to the three new practices in the campaign states ranged between 1.9 and 15.9 percentage points, but only one of these changes (pre-operative nasal screening for Staphylococcus aureus carriage and decolonization prior to surgery) was statistically significant (p < 0.05). On the all-or-none composite measure, adherence to all three evidence-based practices increased from 19.6 to 37.9% in the campaign states, but declined slightly in the comparison states, yielding a relative increase of 23 percentage points (p = 0.004). In the non-campaign states, changes in adherence were not statistically significant. Conclusions Within 6 months, in a cluster-randomized trial, a multi-state campaign targeting hospitals and professionals involved in surgical care and infection control was associated with an increase in adherence to evidence-based practices that can reduce surgical site infection

Conducting Online Expert panels: a feasibility and experimental replicability study

<p>Abstract</p> <p>Background</p> <p>This paper has two goals. First, we explore the feasibility of conducting online expert panels to facilitate consensus finding among a large number of geographically distributed stakeholders. Second, we test the replicability of panel findings across four panels of different size.</p> <p>Method</p> <p>We engaged 119 panelists in an iterative process to identify definitional features of Continuous Quality Improvement (CQI). We conducted four parallel online panels of different size through three one-week phases by using the RAND's ExpertLens process. In Phase I, participants rated potentially definitional CQI features. In Phase II, they discussed rating results online, using asynchronous, anonymous discussion boards. In Phase III, panelists re-rated Phase I features and reported on their experiences as participants.</p> <p>Results</p> <p>66% of invited experts participated in all three phases. 62% of Phase I participants contributed to Phase II discussions and 87% of them completed Phase III. Panel disagreement, measured by the mean absolute deviation from the median (MAD-M), decreased after group feedback and discussion in 36 out of 43 judgments about CQI features. Agreement between the four panels after Phase III was fair (four-way kappa = 0.36); they agreed on the status of five out of eleven CQI features. Results of the post-completion survey suggest that participants were generally satisfied with the online process. Compared to participants in smaller panels, those in larger panels were more likely to agree that they had debated each others' view points.</p> <p>Conclusion</p> <p>It is feasible to conduct online expert panels intended to facilitate consensus finding among geographically distributed participants. The online approach may be practical for engaging large and diverse groups of stakeholders around a range of health services research topics and can help conduct multiple parallel panels to test for the reproducibility of panel conclusions.</p

Trust and control in counterpoint: a case study of conductorless orchestras

Can a large organization be successful without a single leader? Common wisdom suggests that organizations need a leader to control and be ultimately responsible for decision-making and guarding against inefficiency and vulnerability to the behavior of employees. Nonetheless, my case study of two large conductorless orchestras -- Persimfans and Orpheus -- suggests otherwise. Such apparently leaderless organizations reject the idea of hierarchical control because it violates their fundamental goals of artistic freedom and creativity. Yet the absence of a single leader does not mean that conductorless orchestras are, in fact, leaderless. More so than conventional orchestras, they benefit from the talent, commitment, and professionalism of all their members. In contrast to rigid hierarchical control, both Persimfans and Orpheus rely extensively on trust-based governance mechanisms that are essential for collaborative decision-making. Nonetheless, they have also developed a number of less formal control strategies that facilitate the search for consensus and help create and maintain trustworthy relationships among musicians. Reliance on trust and less formal control mechanisms, however, blurs the line between these two governance strategies. To use a musical analogy, trust and control in conductorless orchestras create counterpoint, or the combination of different melodies into a more beautiful polyphonic whole. Besides offering insights into intra-organizational trust and control, my research also contributes to the literature on post-bureaucratic organizations, leadership, power, and collaboration.Ph.D.Includes bibliographical references (p. 207-224)by Dmitry Khodyako