Revisiting concepts of evidence in implementation science

BACKGROUND: Evidence, in multiple forms, is a foundation of implementation science. For public health and clinical practice, evidence includes the following: type 1 evidence on etiology and burden; type 2 evidence on effectiveness of interventions; and type 3: evidence on dissemination and implementation (D&I) within context. To support a vision for development and use of evidence in D&I science that is more comprehensive and equitable (particularly for type 3 evidence), this article aims to clarify concepts of evidence, summarize ongoing debates about evidence, and provide a set of recommendations and tools/resources for addressing the how-to in filling evidence gaps most critical to advancing implementation science. MAIN TEXT: Because current conceptualizations of evidence have been relatively narrow and insufficiently characterized in our opinion, we identify and discuss challenges and debates about the uses, usefulness, and gaps in evidence for implementation science. A set of questions is proposed to assist in determining when evidence is sufficient for dissemination and implementation. Intersecting gaps include the need to (1) reconsider how the evidence base is determined, (2) improve understanding of contextual effects on implementation, (3) sharpen the focus on health equity in how we approach and build the evidence-base, (4) conduct more policy implementation research and evaluation, and (5) learn from audience and stakeholder perspectives. We offer 15 recommendations to assist in filling these gaps and describe a set of tools for enhancing the evidence most needed in implementation science. CONCLUSIONS: To address our recommendations, we see capacity as a necessary ingredient to shift the field\u27s approach to evidence. Capacity includes the push for implementation science where researchers are trained to develop and evaluate evidence which should be useful and feasible for implementers and reflect community or stakeholder priorities. Equally important, there has been inadequate training and too little emphasis on the pull for implementation science (e.g., training implementers, practice-based research). We suggest that funders and reviewers of research should adopt and support a more robust definition of evidence. By critically examining the evolving nature of evidence, implementation science can better fulfill its vision of facilitating widespread and equitable adoption, delivery, and sustainment of scientific advances

Dissemination and stakeholder engagement practices among dissemination & implementation scientists: Results from an online survey

IntroductionThere has been an increasing focus on disseminating research findings, but less about practices specific to disseminating and engaging non-researchers. The present project sought to describe dissemination practices and engagement of stakeholders among dissemination & implementation (D&I) scientists.MethodsMethods to disseminate to and engage non-research stakeholders were assessed using an online survey sent to a broad, diverse sample of D&I scientists.ResultsSurveys were received from 210 participants. The majority of respondents were from university or research settings in the United States. (69%) or Canada (13%), representing a mix of clinical (28%) and community settings (34%). 26% had received formal training in D&I. Respondents indicated routinely engaging in a variety of dissemination-related activities, with academic journal publications (88%), conference presentations (86%), and reports to funders (74%) being the most frequent. Journal publication was identified as the most impactful on respondents' careers (94%), but face-to-face meetings with stakeholders were rated as most impactful on practice or policy (40%). Stakeholder involvement in research was common, with clinical and community-based researchers engaging stakeholder groups in broadly similar ways, but with critical differences noted between researchers with greater seniority, those with more D&I training, those based in the United States vs. Canada, and those in community vs. clinical research settings.ConclusionsThere have been increases in stakeholder engagement, but few other practices since the 2012 survey, and some differences across subgroups. Methods to engage different stakeholders deserve more in-depth investigation. D&I researchers report substantial misalignment of incentives and behaviors related to dissemination to non-research audiences

Long-term effects of the Mediterranean lifestyle program: a randomized clinical trial for postmenopausal women with type 2 diabetes

BACKGROUND: Multiple-risk-factor interventions offer a promising means for addressing the complex interactions between lifestyle behaviors, psychosocial factors, and the social environment. This report examines the long-term effects of a multiple-risk-factor intervention. METHODS: Postmenopausal women (N = 279) with type 2 diabetes participated in the Mediterranean Lifestyle Program (MLP), a randomized, comprehensive lifestyle intervention study. The intervention targeted healthful eating, physical activity, stress management, smoking cessation, and social support. Outcomes included lifestyle behaviors (i.e., dietary intake, physical activity, stress management, smoking cessation), psychosocial variables (e.g., social support, problem solving, self-efficacy, depression, quality of life), and cost analyses at baseline, and 6, 12, and 24 months. RESULTS: MLP participants showed significant 12- and 24-month improvements in all targeted lifestyle behaviors with one exception (there were too few smokers to analyze tobacco use effects), and in psychosocial measures of use of supportive resources, problem solving, self-efficacy, and quality of life. CONCLUSION: The MLP was more effective than usual care over 24 months in producing improvements on behavioral and psychosocial outcomes. Directions for future research include replication with other populations

Recruitment and retention of participants in a pragmatic randomized intervention trial at three community health clinics: Results and lessons learned

Background: Obesity and hypertension and their associated health complications disproportionately affect communities of color and people of lower socioeconomic status. Recruitment and retention of these populations in research trials, and retention in weight loss trials has been an ongoing challenge. Methods: Be Fit, Be Well was a pragmatic randomized weight loss and hypertension management trial of patients attending one of three community health centers in Boston, Massachusetts. Participants were asked to complete follow-up assessments every 6-months for two years. We describe challenges encountered and strategies implemented to recruit and retain trial participants over the 24-month intervention. We also identify baseline participant characteristics associated with retention status. Retention strategies included financial incentives, contact between assessment visits, building relationships with health center primary care providers (PCPs) and staff, and putting participant convenience first. Results: Active refusal rates were low with 130 of 2,631 patients refusing participation (4.9%). Of 474 eligible persons completing telephone screening, 365 (77.0%) completed their baseline visit and were randomized into the study. The study population was predominantly non-Hispanic Black (71.2%), female (68.5%) and reported annual household income of less than $35,000 (70.1%). Recruitment strategies included use of passive approval of potential participants by PCPs, use of part-time staff, and outsourcing calls to a call center. A total of 314 (86.0%) people completed the 24-month visit. Retention levels varied across study visits and intervention condition. Most participants completed three or more visits (69.6%), with 205 (56.2%) completing all four. At 24-months, lower retention was observed for males and the intervention condition. Retention strategies included building strong relationships with clinic staff, flexibility in overcoming participant barriers through use of taxi vouchers, night and weekend appointments, and keeping participants engaged via newsletters and social gatherings. Conclusion: We were able to retain 86.0% of participants at 24-months. Recruitment and retention of high percentages of racial/ethnic minorities and lower income samples is possible with planning, coordination with a trusted community setting and staff (e.g. community health centers and RAs), adaptability and building strong relationships. Trial registration Clinicaltrials.gov Identifier: NCT0066181

Perspectives of scientists on disseminating research findings to non-research audiences

BACKGROUND: Little is known about practices used to disseminate findings to non-research, practitioner audiences. This study describes the perspectives, experience and activities of dissemination & implementation (D&I) scientists around disseminating their research findings. METHODS: The study explored D&I scientists\u27 experiences and recommendations for assessment of dissemination activities to non-research audiences. Existing list serves were used to recruit scientists. Respondents were asked three open-ended questions on an Internet survey about dissemination activities, recommendations for changing evaluation systems and suggestions to improve their own dissemination of their work. RESULTS: Surveys were completed by 159 scientists reporting some training, funding and/or publication history in D&I. Three themes emerged across each of the three open-ended questions. Question 1 on evaluation generated the themes of: 1a) promotional review; 1b) funding requirements and 1c) lack of acknowledgement of dissemination activities. Question 2 on recommended changes generated the themes of: 2a) dissemination as a requirement of the academic promotion process; 2b) requirement of dissemination plan and 2c) dissemination metrics. Question 3 on personal changes to improve dissemination generated the themes of: 3a) allocation of resources for dissemination activities; 3b) emerging dissemination channels and 3c) identify and address issues of priority for stakeholders. CONCLUSIONS: Our findings revealed different types of issues D&I scientists encounter when disseminating findings to clinical, public health or policy audiences and their suggestions to improve the process. Future research should consider key requirements which determine academic promotion and grant funding as an opportunity to expand dissemination efforts

Assessing Organizational Readiness for Depression Care Quality Improvement: Relative Commitment and Implementation Capability

Background: Depression is a major cause of morbidity and cost in primary care patient populations. Successful depression improvement models, however, are complex. Based on organizational readiness theory, a practice’s commitment to change and its capability to carry out the change are both important predictors of initiating improvement. We empirically explored the links between relative commitment (i.e., the intention to move forward within the following year) and implementation capability. Methods: The DIAMOND initiative administered organizational surveys to medical and quality improvement leaders from each of 83 primary care practices in Minnesota. Surveys preceded initiation of activities directed at implementation of a collaborative care model for improving depression care. To assess implementation capability, we developed composites of survey items for five types of organizational factors postulated to be collaborative care barriers and facilitators. To assess relative commitment for each practice, we averaged leader ratings on an identical survey question assessing practice priorities. We used multivariable regression analyses to assess the extent to which implementation capability predicted relative commitment. We explored whether relative commitment or implementation capability measures were associated with earlier initiation of DIAMOND improvements. Results: All five implementation capability measures independently predicted practice leaders’ relative commitment to improving depression care in the following year. These included the following: quality improvement culture and attitudes (p = 0.003), depression culture and attitudes (p \u3c0.001), prior depression quality improvement activities (p \u3c0.001), advanced access and tracking capabilities (p = 0.03), and depression collaborative care features in place (p = 0.03). Higher relative commitment (p = 0.002) and prior depression quality improvement activities appeared to be associated with earlier participation in the DIAMOND initiative. Conclusions: The study supports the concept of organizational readiness to improve quality of care and the use of practice leader surveys to assess it. Practice leaders’ relative commitment to depression care improvement may be a useful measure of the likelihood that a practice is ready to initiate evidence-based depression care changes. A comprehensive organizational assessment of implementation capability for depression care improvement may identify specific barriers or facilitators to readiness that require targeted attention from implementers

Measurement resources for dissemination and implementation research in health

BACKGROUND: A 2-day consensus working meeting, hosted by the United States National Institutes of Health and the Veterans Administration, focused on issues related to dissemination and implementation (D&I) research in measurement and reporting. Meeting participants included 23 researchers, practitioners, and decision makers from the USA and Canada who concluded that the field would greatly benefit from measurement resources to enhance the ease, harmonization, and rigor of D&I evaluation efforts. This paper describes the findings from an environmental scan and literature review of resources for D&I measures. FINDINGS: We identified a total of 17 resources, including four web-based repositories and 12 static reviews or tools that attempted to synthesize and evaluate existing measures for D&I research. Thirteen resources came from the health discipline, and 11 were populated from database reviews. Ten focused on quantitative measures, and all were generated as a resource for researchers. Fourteen were organized according to an established D&I theory or framework, with the number of constructs and measures ranging from 1 to more than 450. Measure metadata was quite variable with only six providing information on the psychometric properties of measures. CONCLUSIONS: Additional guidance on the development and use of measures are needed. A number of approaches, resources, and critical areas for future work are discussed. Researchers and stakeholders are encouraged to take advantage of a number of funding mechanisms supporting this type of work. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (doi:10.1186/s13012-016-0401-y) contains supplementary material, which is available to authorized users

Enhancing Success of Medicare\u27s Shared Decision Making Mandates Using Implementation Science: Examples Applying the Pragmatic Robust Implementation and Sustainability Model (PRISM)

The Centers for Medicare and Medicaid Services (CMS) has mandated shared decision making (SDM) using patient decision aids for three conditions (lung cancer screening, atrial fibrillation, and implantable defibrillators). These forward-thinking approaches are in response to a wealth of efficacy data demonstrating that decision aids can improve patient decision making. However, there has been little focus on how to implement these approaches in real-world practice. This article demonstrates how using an implementation science framework may help programs understand multilevel challenges and opportunities to improve adherence to the CMS mandates. Using the PRISM (Pragmatic Robust Implementation and Sustainability Model) framework, we discuss general challenges to implementation of SDM, issues specific to each mandate, and how to plan for, enhance, and assess SDM implementation outcomes. Notably, a theme of this discussion is that successful implementation is context-specific and to truly have successful and sustainable changes in practice, context variability, and adaptation to context must be considered and addressed

The Effects of Patient-Centered Depression Care on Patient Satisfaction and Depression Remission

Background: While health systems are striving for patient-centered care, they have little evidence to guide them on how to engage patients in their care, or how this may affect patient experiences and outcomes. Objective: To explore which specific patient-centered aspects of care were best associated with depression improvement and care satisfaction. Methods: Design - observational. Setting - 83 primary care clinics across Minnesota. Subjects - Primary care patients with new prescriptions for antidepressants for depression were recruited from 2007 to 2009. Outcome measures - Patients completed phone surveys regarding demographics and self-rated health status and depression severity at baseline and 6 months. Patient centeredness was assessed via a modified version of the Patient Assessment of Chronic Illness Care. Differences in rates of remission and satisfaction between positive and negative responses for each care process were evaluated using chi-square tests. Results: At 6 months, 37% of 792 patients ages 18–88 achieved depression remission, and 79% rated their care as good-to-excellent. Soliciting patient preferences for care and questions or concerns, providing treatment plans, utilizing depression scales and asking about suicide risk were patient centered measures that were positively associated with depression remission in the unadjusted model; these associations were mildly weakened after adjustment for depression severity and health status. Nearly all measures of patient centeredness were positively associated with care ratings. Conclusion: The patient centeredness of care influences how patients experience and rate their care. This study identified specific actions providers can take to improve patient satisfaction and depression outcomes