Dissemination and implementation science training needs: Insights from practitioners and researchers

INTRODUCTION: Dissemination and implementation research training has great potential to improve the impact and reach of health-related research; however, research training needs from the end user perspective are unknown. This paper identifies and prioritizes dissemination and implementation research training needs. METHODS: A diverse sample of researchers, practitioners, and policymakers was invited to participate in Concept Mapping in 2014–2015. Phase 1 (Brainstorming) gathered participants' responses to the prompt: To improve the impact of research evidence in practice and policy settings, a skill in which researchers need more training is… The resulting statement list was edited and included subsequent phases. Phase 2 (Sorting) asked participants to sort each statement into conceptual piles. In Phase 3 (Rating), participants rated the difficulty and importance of incorporating each statement into a training curriculum. A multidisciplinary team synthesized and interpreted the results in 2015–2016. RESULTS: During Brainstorming, 60 researchers and 60 practitioners/policymakers contributed 274 unique statements. Twenty-nine researchers and 16 practitioners completed sorting and rating. Nine concept clusters were identified: Communicating Research Findings, Improve Practice Partnerships, Make Research More Relevant, Strengthen Communication Skills, Develop Research Methods and Measures, Consider and Enhance Fit, Build Capacity for Research, and Understand Multilevel Context. Though researchers and practitioners had high agreement about importance (r =0.93) and difficulty (r =0.80), ratings differed for several clusters (e.g., Build Capacity for Research). CONCLUSIONS: Including researcher and practitioner perspectives in competency development for dissemination and implementation research identifies skills and capacities needed to conduct and communicate contextualized, meaningful, and relevant research

Training scholars in dissemination and implementation research for cancer prevention and control: A mentored approach

Abstract Background As the field of D&I (dissemination and implementation) science grows to meet the need for more effective and timely applications of research findings in routine practice, the demand for formalized training programs has increased concurrently. The Mentored Training for Dissemination and Implementation Research in Cancer (MT-DIRC) Program aims to build capacity in the cancer control D&I research workforce, especially among early career researchers. This paper outlines the various components of the program and reports results of systematic evaluations to ascertain its effectiveness. Methods Essential features of the program include selection of early career fellows or more experienced investigators with a focus relevant to cancer control transitioning to a D&I research focus, a 5-day intensive training institute, ongoing peer and senior mentoring, mentored planning and work on a D&I research proposal or project, limited pilot funding, and training and ongoing improvement activities for mentors. The core faculty and staff members of the MT-DIRC program gathered baseline and ongoing evaluation data regarding D&I skill acquisition and mentoring competency through participant surveys and analyzed it by iterative collective reflection. Results A majority (79%) of fellows are female, assistant professors (55%); 59% are in allied health disciplines, and 48% focus on cancer prevention research. Forty-three D&I research competencies were assessed; all improved from baseline to 6 and 18 months. These effects were apparent across beginner, intermediate, and advanced initial D&I competency levels and across the competency domains. Mentoring competency was rated very highly by the fellows––higher than rated by the mentors themselves. The importance of different mentoring activities, as rated by the fellows, was generally congruent with their satisfaction with the activities, with the exception of relatively greater satisfaction with the degree of emotional support and relatively lower satisfaction for skill building and opportunity initially. Conclusions These first years of MT-DIRC demonstrated the program’s ability to attract, engage, and improve fellows’ competencies and skills and implement a multicomponent mentoring program that was well received. This account of the program can serve as a basis for potential replication and evolution of this model in training future D&I science researchers

Mentored training and its association with dissemination and implementation research output: A quasi-experimental evaluation

BACKGROUND: There is a continued need to evaluate training programs in dissemination and implementation (D&I) research. Scientific products yielded from trainees are an important and objective measure to understand the capacity growth within the D&I field. This study evaluates our mentored training program in terms of scientific productivity among applicants. METHODS: Post-doctoral and early-career cancer researchers were recruited and applied to the R25 Mentored Training for Dissemination and Implementation Research in Cancer (MT-DIRC) between 2014 and 2017. Using application details and publicly available bibliometric and funding data, we compared selected fellows with unsuccessful applicants (nonfellows). We extracted Scopus citations and US federal grant funding records for all applicants (N = 102). Funding and publication abstracts were de-identified and coded for D&I focus and aggregated to the applicant level for analysis. Logistic regression models were explored separately for the odds of (1) a D&I publication and (2) US federal grant funding post year of application among fellows (N = 55) and nonfellows (N = 47). Additional models were constructed to include independent variables that attenuated the program\u27s association by 5% or more. Only US-based applicants (N = 87) were included in the grant funding analysis. RESULTS: Fellows and nonfellows were similar across several demographic characteristics. Fellows were more than 3 times more likely than nonfellows to have grant funding after MT-DIRC application year (OR 3.2; 95% CI 1.1-11.0) while controlling for time since application year; the association estimate was 3.1 (95% CI 0.98-11.0) after adjusting for both cancer research area and previous grant funding. For publications, fellows were almost 4 times more likely to publish D&I-focused work adjusting for time (OR 3.8; 95% CI 1.7-9.0). This association lessened after adjusting for previous D&I publication and years since undergraduate degree (OR 2.9; 95% CI 1.2-7.5). CONCLUSIONS: We document the association of a mentored training approach with built-in networks of peers to yield productive D&I researchers. Future evaluation efforts could be expanded to include other forms of longer-term productivity such as policy or practice change as additional objective measures. D&I research trainings in the USA and internationally should consider common evaluation measures

Toward optimal implementation of cancer prevention and control programs in public health: A study protocol on mis-implementation

Abstract Background Much of the cancer burden in the USA is preventable, through application of existing knowledge. State-level funders and public health practitioners are in ideal positions to affect programs and policies related to cancer control. Mis-implementation refers to ending effective programs and policies prematurely or continuing ineffective ones. Greater attention to mis-implementation should lead to use of effective interventions and more efficient expenditure of resources, which in the long term, will lead to more positive cancer outcomes. Methods This is a three-phase study that takes a comprehensive approach, leading to the elucidation of tactics for addressing mis-implementation. Phase 1: We assess the extent to which mis-implementation is occurring among state cancer control programs in public health. This initial phase will involve a survey of 800 practitioners representing all states. The programs represented will span the full continuum of cancer control, from primary prevention to survivorship. Phase 2: Using data from phase 1 to identify organizations in which mis-implementation is particularly high or low, the team will conduct eight comparative case studies to get a richer understanding of mis-implementation and to understand contextual differences. These case studies will highlight lessons learned about mis-implementation and identify hypothesized drivers. Phase 3: Agent-based modeling will be used to identify dynamic interactions between individual capacity, organizational capacity, use of evidence, funding, and external factors driving mis-implementation. The team will then translate and disseminate findings from phases 1 to 3 to practitioners and practice-related stakeholders to support the reduction of mis-implementation. Discussion This study is innovative and significant because it will (1) be the first to refine and further develop reliable and valid measures of mis-implementation of public health programs; (2) bring together a strong, transdisciplinary team with significant expertise in practice-based research; (3) use agent-based modeling to address cancer control implementation; and (4) use a participatory, evidence-based, stakeholder-driven approach that will identify key leverage points for addressing mis-implementation among state public health programs. This research is expected to provide replicable computational simulation models that can identify leverage points and public health system dynamics to reduce mis-implementation in cancer control and may be of interest to other health areas

Proceedings of the 8th Annual Conference on the Science of Dissemination and Implementation

A1 Introduction to the 8(th) Annual Conference on the Science of Dissemination and Implementation: Optimizing Personal and Population Health David Chambers, Lisa Simpson D1 Discussion forum: Population health D&I research Felicia Hill-Briggs D2 Discussion forum: Global health D&I research Gila Neta, Cynthia Vinson D3 Discussion forum: Precision medicine and D&I research David Chambers S1 Predictors of community therapists’ use of therapy techniques in a large public mental health system Rinad Beidas, Steven Marcus, Gregory Aarons, Kimberly Hoagwood, Sonja Schoenwald, Arthur Evans, Matthew Hurford, Ronnie Rubin, Trevor Hadley, Frances Barg, Lucia Walsh, Danielle Adams, David Mandell S2 Implementing brief cognitive behavioral therapy (CBT) in primary care: Clinicians' experiences from the field Lindsey Martin, Joseph Mignogna, Juliette Mott, Natalie Hundt, Michael Kauth, Mark Kunik, Aanand Naik, Jeffrey Cully S3 Clinician competence: Natural variation, factors affecting, and effect on patient outcomes Alan McGuire, Dominique White, Tom Bartholomew, John McGrew, Lauren Luther, Angie Rollins, Michelle Salyers S4 Exploring the multifaceted nature of sustainability in community-based prevention: A mixed-method approach Brittany Cooper, Angie Funaiole S5 Theory informed behavioral health integration in primary care: Mixed methods evaluation of the implementation of routine depression and alcohol screening and assessment Julie Richards, Amy Lee, Gwen Lapham, Ryan Caldeiro, Paula Lozano, Tory Gildred, Carol Achtmeyer, Evette Ludman, Megan Addis, Larry Marx, Katharine Bradley S6 Enhancing the evidence for specialty mental health probation through a hybrid efficacy and implementation study Tonya VanDeinse, Amy Blank Wilson, Burgin Stacey, Byron Powell, Alicia Bunger, Gary Cuddeback S7 Personalizing evidence-based child mental health care within a fiscally mandated policy reform Miya Barnett, Nicole Stadnick, Lauren Brookman-Frazee, Anna Lau S8 Leveraging an existing resource for technical assistance: Community-based supervisors in public mental health Shannon Dorsey, Michael Pullmann S9 SBIRT implementation for adolescents in urban federally qualified health centers: Implementation outcomes Shannon Mitchell, Robert Schwartz, Arethusa Kirk, Kristi Dusek, Marla Oros, Colleen Hosler, Jan Gryczynski, Carolina Barbosa, Laura Dunlap, David Lounsbury, Kevin O'Grady, Barry Brown S10 PANEL: Tailoring Implementation Strategies to Context - Expert recommendations for tailoring strategies to context Laura Damschroder, Thomas Waltz, Byron Powell S11 PANEL: Tailoring Implementation Strategies to Context - Extreme facilitation: Helping challenged healthcare settings implement complex programs Mona Ritchie S12 PANEL: Tailoring Implementation Strategies to Context - Using menu-based choice tasks to obtain expert recommendations for implementing three high-priority practices in the VA Thomas Waltz S13 PANEL: The Use of Technology to Improve Efficient Monitoring of Implementation of Evidence-based Programs - Siri, rate my therapist: Using technology to automate fidelity ratings of motivational interviewing David Atkins, Zac E. Imel, Bo Xiao, Doğan Can, Panayiotis Georgiou, Shrikanth Narayanan S14 PANEL: The Use of Technology to Improve Efficient Monitoring of Implementation of Evidence-based Programs - Identifying indicators of implementation quality for computer-based ratings Cady Berkel, Carlos Gallo, Irwin Sandler, C. Hendricks Brown, Sharlene Wolchik, Anne Marie Mauricio S15 PANEL: The Use of Technology to Improve Efficient Monitoring of Implementation of Evidence-based Programs - Improving implementation of behavioral interventions by monitoring emotion in spoken speech Carlos Gallo, C. Hendricks Brown, Sanjay Mehrotra S16 Scorecards and dashboards to assure data quality of health management information system (HMIS) using R Dharmendra Chandurkar, Siddhartha Bora, Arup Das, Anand Tripathi, Niranjan Saggurti, Anita Raj S17 A big data approach for discovering and implementing patient safety insights Eric Hughes, Brian Jacobs, Eric Kirkendall S18 Improving the efficacy of a depression registry for use in a collaborative care model Danielle Loeb, Katy Trinkley, Michael Yang, Andrew Sprowell, Donald Nease S19 Measurement feedback systems as a strategy to support implementation of measurement-based care in behavioral health Aaron Lyon, Cara Lewis, Meredith Boyd, Abigail Melvin, Semret Nicodimos, Freda Liu, Nathanial Jungbluth S20 PANEL: Implementation Science and Learning Health Systems: Intersections and Commonalities - Common loop assay: Methods of supporting learning collaboratives Allen Flynn S21 PANEL: Implementation Science and Learning Health Systems: Intersections and Commonalities - Innovating audit and feedback using message tailoring models for learning health systems Zach Landis-Lewis S22 PANEL: Implementation Science and Learning Health Systems: Intersections and Commonalities - Implementation science and learning health systems: Connecting the dots Anne Sales S23 Facilitation activities of Critical Access Hospitals during TeamSTEPPS implementation Jure Baloh, Marcia Ward, Xi Zhu S24 Organizational and social context of federally qualified health centers and variation in maternal depression outcomes Ian Bennett, Jurgen Unutzer, Johnny Mao, Enola Proctor, Mindy Vredevoogd, Ya-Fen Chan, Nathaniel Williams, Phillip Green S25 Decision support to enhance treatment of hospitalized smokers: A randomized trial Steven Bernstein, June-Marie Rosner, Michelle DeWitt, Jeanette Tetrault, James Dziura, Allen Hsiao, Scott Sussman, Patrick O’Connor, Benjamin Toll S26 PANEL: Developing Sustainable Strategies for the Implementation of Patient-Centered Care across Diverse US Healthcare Systems - A patient-centered approach to successful community transition after catastrophic injury Michael Jones, Julie Gassaway S27 PANEL: Developing Sustainable Strategies for the Implementation of Patient-Centered Care across Diverse US Healthcare Systems - Conducting PCOR to integrate mental health and cancer screening services in primary care Jonathan Tobin S28 PANEL: Developing Sustainable Strategies for the Implementation of Patient-Centered Care across Diverse US Healthcare Systems - A comparative effectiveness trial of optimal patient-centered care for US trauma care systems Douglas Zatzick S29 Preferences for in-person communication among patients in a multi-center randomized study of in-person versus telephone communication of genetic test results for cancer susceptibility Angela R Bradbury, Linda Patrick-Miller, Brian Egleston, Olufunmilayo I Olopade, Michael J Hall, Mary B Daly, Linda Fleisher, Generosa Grana, Pamela Ganschow, Dominique Fetzer, Amanda Brandt, Dana Farengo-Clark, Andrea Forman, Rikki S Gaber, Cassandra Gulden, Janice Horte, Jessica Long, Rachelle Lorenz Chambers, Terra Lucas, Shreshtha Madaan, Kristin Mattie, Danielle McKenna, Susan Montgomery, Sarah Nielsen, Jacquelyn Powers, Kim Rainey, Christina Rybak, Michelle Savage, Christina Seelaus, Jessica Stoll, Jill Stopfer, Shirley Yao and Susan Domchek S30 Working towards de-implementation: A mixed methods study in breast cancer surveillance care Erin Hahn, Corrine Munoz-Plaza, Jianjin Wang, Jazmine Garcia Delgadillo, Brian Mittman Michael Gould S31Integrating evidence-based practices for increasing cancer screenings in safety-net primary care systems: A multiple case study using the consolidated framework for implementation research Shuting (Lily) Liang, Michelle C. Kegler, Megan Cotter, Emily Phillips, April Hermstad, Rentonia Morton, Derrick Beasley, Jeremy Martinez, Kara Riehman S32 Observations from implementing an mHealth intervention in an FQHC David Gustafson, Lisa Marsch, Louise Mares, Andrew Quanbeck, Fiona McTavish, Helene McDowell, Randall Brown, Chantelle Thomas, Joseph Glass, Joseph Isham, Dhavan Shah S33 A multicomponent intervention to improve primary care provider adherence to chronic opioid therapy guidelines and reduce opioid misuse: A cluster randomized controlled trial protocol Jane Liebschutz, Karen Lasser S34 Implementing collaborative care for substance use disorders in primary care: Preliminary findings from the summit study Katherine Watkins, Allison Ober, Sarah Hunter, Karen Lamp, Brett Ewing S35 Sustaining a task-shifting strategy for blood pressure control in Ghana: A stakeholder analysis Juliet Iwelunmor, Joyce Gyamfi, Sarah Blackstone, Nana Kofi Quakyi, Jacob Plange-Rhule, Gbenga Ogedegbe S36 Contextual adaptation of the consolidated framework for implementation research (CFIR) in a tobacco cessation study in Vietnam Pritika Kumar, Nancy Van Devanter, Nam Nguyen, Linh Nguyen, Trang Nguyen, Nguyet Phuong, Donna Shelley S37 Evidence check: A knowledge brokering approach to systematic reviews for policy Sian Rudge S38 Using Evidence Synthesis to Strengthen Complex Health Systems in Low- and Middle-Income Countries Etienne Langlois S39 Does it matter: timeliness or accuracy of results? The choice of rapid reviews or systematic reviews to inform decision-making Andrea Tricco S40 Evaluation of the veterans choice program using lean six sigma at a VA medical center to identify benefits and overcome obstacles Sherry Ball, Anne Lambert-Kerzner, Christine Sulc, Carol Simmons, Jeneen Shell-Boyd, Taryn Oestreich, Ashley O'Connor, Emily Neely, Marina McCreight, Amy Labebue, Doreen DiFiore, Diana Brostow, P. Michael Ho, David Aron S41 The influence of local context on multi-stakeholder alliance quality improvement activities: A multiple case study Jillian Harvey, Megan McHugh, Dennis Scanlon S42 Increasing physical activity in early care and education: Sustainability via active garden education (SAGE) Rebecca Lee, Erica Soltero, Nathan Parker, Lorna McNeill, Tracey Ledoux S43 Marking a decade of policy implementation: The successes and continuing challenges of a provincial school food and nutrition policy in Canada Jessie-Lee McIsaac, Kate MacLeod, Nicole Ata, Sherry Jarvis, Sara Kirk S44 Use of research evidence among state legislators who prioritize mental health and substance abuse issues Jonathan Purtle, Elizabeth Dodson, Ross Brownson S45 PANEL: Effectiveness-Implementation Hybrid Designs: Clarifications, Refinements, and Additional Guidance Based on a Systematic Review and Reports from the Field - Hybrid type 1 designs Brian Mittman, Geoffrey Curran S46 PANEL: Effectiveness-Implementation Hybrid Designs: Clarifications, Refinements, and Additional Guidance Based on a Systematic Review and Reports from the Field - Hybrid type 2 designs Geoffrey Curran S47 PANEL: Effectiveness-Implementation Hybrid Designs: Clarifications, Refinements, and Additional Guidance Based on a Systematic Review and Reports from the Field - Hybrid type 3 designs Jeffrey Pyne S48 Linking team level implementation leadership and implementation climate to individual level attitudes, behaviors, and implementation outcomes Gregory Aarons, Mark Ehrhart, Elisa Torres S49 Pinpointing the specific elements of local context that matter most to implementation outcomes: Findings from qualitative comparative analysis in the RE-inspire study of VA acute stroke care Edward Miech S50 The GO score: A new context-sensitive instrument to measure group organization level for providing and improving care Edward Miech S51 A research network approach for boosting implementation and improvement Kathleen Stevens, I.S.R.N. Steering Council S52 PANEL: Qualitative methods in D&I Research: Value, rigor and challenge - The value of qualitative methods in implementation research Alison Hamilton S53 PANEL: Qualitative methods in D&I Research: Value, rigor and challenge - Learning evaluation: The role of qualitative methods in dissemination and implementation research Deborah Cohen S54 PANEL: Qualitative methods in D&I Research: Value, rigor and challenge - Qualitative methods in D&I research Deborah Padgett S55 PANEL: Maps & models: The promise of network science for clinical D&I - Hospital network of sharing patients with acute and chronic diseases in California Alexandra Morshed S56 PANEL: Maps & models: The promise of network science for clinical D&I - The use of social network analysis to identify dissemination targets and enhance D&I research study recruitment for pre-exposure prophylaxis for HIV (PrEP) among men who have sex with men Rupa Patel S57 PANEL: Maps & models: The promise of network science for clinical D&I - Network and organizational factors related to the adoption of patient navigation services among rural breast cancer care providers Beth Prusaczyk S58 A theory of de-implementation based on the theory of healthcare professionals’ behavior and intention (THPBI) and the becker model of unlearning David C. Aron, Divya Gupta, Sherry Ball S59 Observation of registered dietitian nutritionist-patient encounters by dietetic interns highlights low awareness and implementation of evidence-based nutrition practice guidelines Rosa Hand, Jenica Abram, Taylor Wolfram S60 Program sustainability action planning: Building capacity for program sustainability using the program sustainability assessment tool Molly Hastings, Sarah Moreland-Russell S61 A review of D&I study designs in published study protocols Rachel Tabak, Alex Ramsey, Ana Baumann, Emily Kryzer, Katherine Montgomery, Ericka Lewis, Margaret Padek, Byron Powell, Ross Brownson S62 PANEL: Geographic variation in the implementation of public health services: Economic, organizational, and network determinants - Model simulation techniques to estimate the cost of implementing foundational public health services Cezar Brian Mamaril, Glen Mays, Keith Branham, Lava Timsina S63 PANEL: Geographic variation in the implementation of public health services: Economic, organizational, and network determinants - Inter-organizational network effects on the implementation of public health services Glen Mays, Rachel Hogg S64 PANEL: Building capacity for implementation and dissemination of the communities that care prevention system at scale to promote evidence-based practices in behavioral health - Implementation fidelity, coalition functioning, and community prevention system transformation using communities that care Abigail Fagan, Valerie Shapiro, Eric Brown S65 PANEL: Building capacity for implementation and dissemination of the communities that care prevention system at scale to promote evidence-based practices in behavioral health - Expanding capacity for implementation of communities that care at scale using a web-based, video-assisted training system Kevin Haggerty, David Hawkins S66 PANEL: Building capacity for implementation and dissemination of the communities that care prevention system at scale to promote evidence-based practices in behavioral health - Effects of communities that care on reducing youth behavioral health problems Sabrina Oesterle, David Hawkins, Richard Catalano S68 When interventions end: the dynamics of intervention de-adoption and replacement Virginia McKay, M. Margaret Dolcini, Lee Hoffer S69 Results from next-d: can a disease specific health plan reduce incident diabetes development among a national sample of working-age adults with pre-diabetes? Tannaz Moin, Jinnan Li, O. Kenrik Duru, Susan Ettner, Norman Turk, Charles Chan, Abigail Keckhafer, Robert Luchs, Sam Ho, Carol Mangione S70 Implementing smoking cessation interventions in primary care settings (STOP): using the interactive systems framework Peter Selby, Laurie Zawertailo, Nadia Minian, Dolly Balliunas, Rosa Dragonetti, Sarwar Hussain, Julia Lecce S71 Testing the Getting To Outcomes implementation support intervention in prevention-oriented, community-based settings Matthew Chinman, Joie Acosta, Patricia Ebener, Patrick S Malone, Mary Slaughter S72 Examining the reach of a multi-component farmers’ market implementation approach among low-income consumers in an urban context Darcy Freedman, Susan Flocke, Eunlye Lee, Kristen Matlack, Erika Trapl, Punam Ohri-Vachaspati, Morgan Taggart, Elaine Borawski S73 Increasing implementation of evidence-based health promotion practices at large workplaces: The CEOs Challenge Amanda Parrish, Jeffrey Harris, Marlana Kohn, Kristen Hammerback, Becca McMillan, Peggy Hannon S74 A qualitative assessment of barriers to nutrition promotion and obesity prevention in childcare Taren Swindle, Geoffrey Curran, Leanne Whiteside-Mansell, Wendy Ward S75 Documenting institutionalization of a health communication intervention in African American churches Cheryl Holt, Sheri Lou Santos, Erin Tagai, Mary Ann Scheirer, Roxanne Carter, Janice Bowie, Muhiuddin Haider, Jimmie Slade, Min Qi Wang S76 Reduction in hospital utilization by underserved patients through use of a community-medical home Andrew Masica, Gerald Ogola, Candice Berryman, Kathleen Richter S77 Sustainability of evidence-based lay health advisor programs in African American communities: A mixed methods investigation of the National Witness Project Rachel Shelton, Lina Jandorf, Deborah Erwin S78 Predicting the long-term uninsured population and analyzing their gaps in physical access to healthcare in South Carolina Khoa Truong S79 Using an evidence-based parenting intervention in churches to prevent behavioral problems among Filipino youth: A randomized pilot study Joyce R. Javier, Dean Coffey, Sheree M. Schrager, Lawrence Palinkas, Jeanne Miranda S80 Sustainability of elementary school-based health centers in three health-disparate southern communities Veda Johnson, Valerie Hutcherson, Ruth Ellis S81 Childhood obesity prevention partnership in Louisville: creative opportunities to engage families in a multifaceted approach to obesity prevention Anna Kharmats, Sandra Marshall-King, Monica LaPradd, Fannie Fonseca-Becker S82 Improvements in cervical cancer prevention found after implementation of evidence-based Latina prevention care management program Deanna Kepka, Julia Bodson, Echo Warner, Brynn Fowler S83 The OneFlorida data trust: Achieving health equity through research & training capacity building Elizabeth Shenkman, William Hogan, Folakami Odedina, Jessica De Leon, Monica Hooper, Olveen Carrasquillo, Renee Reams, Myra Hurt, Steven Smith, Jose Szapocznik, David Nelson, Prabir Mandal S84 Disseminating and sustaining medical-legal partnerships: Shared value and social return on investment James Teufe

Variation in Research Designs Used to Test the Effectiveness of Dissemination and Implementation Strategies: A Review

BackgroundThe need for optimal study designs in dissemination and implementation (D&I) research is increasingly recognized. Despite the wide range of study designs available for D&I research, we lack understanding of the types of designs and methodologies that are routinely used in the field. This review assesses the designs and methodologies in recently proposed D&I studies and provides resources to guide design decisions.MethodsWe reviewed 404 study protocols published in the journal Implementation Science from 2/2006 to 9/2017. Eligible studies tested the efficacy or effectiveness of D&I strategies (i.e., not effectiveness of the underlying clinical or public health intervention); had a comparison by group and/or time; and used ≥1 quantitative measure. Several design elements were extracted: design category (e.g., randomized); design type [e.g., cluster randomized controlled trial (RCT)]; data type (e.g., quantitative); D&I theoretical framework; levels of treatment assignment, intervention, and measurement; and country in which the research was conducted. Each protocol was double-coded, and discrepancies were resolved through discussion.ResultsOf the 404 protocols reviewed, 212 (52%) studies tested one or more implementation strategy across 208 manuscripts, therefore meeting inclusion criteria. Of the included studies, 77% utilized randomized designs, primarily cluster RCTs. The use of alternative designs (e.g., stepped wedge) increased over time. Fewer studies were quasi-experimental (17%) or observational (6%). Many study design categories (e.g., controlled pre–post, matched pair cluster design) were represented by only one or two studies. Most articles proposed quantitative and qualitative methods (61%), with the remaining 39% proposing only quantitative. Half of protocols (52%) reported using a theoretical framework to guide the study. The four most frequently reported frameworks were Consolidated Framework for Implementing Research and RE-AIM (n = 16 each), followed by Promoting Action on Research Implementation in Health Services and Theoretical Domains Framework (n = 12 each).ConclusionWhile several novel designs for D&I research have been proposed (e.g., stepped wedge, adaptive designs), the majority of the studies in our sample employed RCT designs. Alternative study designs are increasing in use but may be underutilized for a variety of reasons, including preference of funders or lack of awareness of these designs. Promisingly, the prevalent use of quantitative and qualitative methods together reflects methodological innovation in newer D&I research

Data_Sheet_2.PDF

Background<p>The need for optimal study designs in dissemination and implementation (D&I) research is increasingly recognized. Despite the wide range of study designs available for D&I research, we lack understanding of the types of designs and methodologies that are routinely used in the field. This review assesses the designs and methodologies in recently proposed D&I studies and provides resources to guide design decisions.</p>Methods<p>We reviewed 404 study protocols published in the journal Implementation Science from 2/2006 to 9/2017. Eligible studies tested the efficacy or effectiveness of D&I strategies (i.e., not effectiveness of the underlying clinical or public health intervention); had a comparison by group and/or time; and used ≥1 quantitative measure. Several design elements were extracted: design category (e.g., randomized); design type [e.g., cluster randomized controlled trial (RCT)]; data type (e.g., quantitative); D&I theoretical framework; levels of treatment assignment, intervention, and measurement; and country in which the research was conducted. Each protocol was double-coded, and discrepancies were resolved through discussion.</p>Results<p>Of the 404 protocols reviewed, 212 (52%) studies tested one or more implementation strategy across 208 manuscripts, therefore meeting inclusion criteria. Of the included studies, 77% utilized randomized designs, primarily cluster RCTs. The use of alternative designs (e.g., stepped wedge) increased over time. Fewer studies were quasi-experimental (17%) or observational (6%). Many study design categories (e.g., controlled pre–post, matched pair cluster design) were represented by only one or two studies. Most articles proposed quantitative and qualitative methods (61%), with the remaining 39% proposing only quantitative. Half of protocols (52%) reported using a theoretical framework to guide the study. The four most frequently reported frameworks were Consolidated Framework for Implementing Research and RE-AIM (n = 16 each), followed by Promoting Action on Research Implementation in Health Services and Theoretical Domains Framework (n = 12 each).</p>Conclusion<p>While several novel designs for D&I research have been proposed (e.g., stepped wedge, adaptive designs), the majority of the studies in our sample employed RCT designs. Alternative study designs are increasing in use but may be underutilized for a variety of reasons, including preference of funders or lack of awareness of these designs. Promisingly, the prevalent use of quantitative and qualitative methods together reflects methodological innovation in newer D&I research.</p

Data_Sheet_3.PDF

Background<p>The need for optimal study designs in dissemination and implementation (D&I) research is increasingly recognized. Despite the wide range of study designs available for D&I research, we lack understanding of the types of designs and methodologies that are routinely used in the field. This review assesses the designs and methodologies in recently proposed D&I studies and provides resources to guide design decisions.</p>Methods<p>We reviewed 404 study protocols published in the journal Implementation Science from 2/2006 to 9/2017. Eligible studies tested the efficacy or effectiveness of D&I strategies (i.e., not effectiveness of the underlying clinical or public health intervention); had a comparison by group and/or time; and used ≥1 quantitative measure. Several design elements were extracted: design category (e.g., randomized); design type [e.g., cluster randomized controlled trial (RCT)]; data type (e.g., quantitative); D&I theoretical framework; levels of treatment assignment, intervention, and measurement; and country in which the research was conducted. Each protocol was double-coded, and discrepancies were resolved through discussion.</p>Results<p>Of the 404 protocols reviewed, 212 (52%) studies tested one or more implementation strategy across 208 manuscripts, therefore meeting inclusion criteria. Of the included studies, 77% utilized randomized designs, primarily cluster RCTs. The use of alternative designs (e.g., stepped wedge) increased over time. Fewer studies were quasi-experimental (17%) or observational (6%). Many study design categories (e.g., controlled pre–post, matched pair cluster design) were represented by only one or two studies. Most articles proposed quantitative and qualitative methods (61%), with the remaining 39% proposing only quantitative. Half of protocols (52%) reported using a theoretical framework to guide the study. The four most frequently reported frameworks were Consolidated Framework for Implementing Research and RE-AIM (n = 16 each), followed by Promoting Action on Research Implementation in Health Services and Theoretical Domains Framework (n = 12 each).</p>Conclusion<p>While several novel designs for D&I research have been proposed (e.g., stepped wedge, adaptive designs), the majority of the studies in our sample employed RCT designs. Alternative study designs are increasing in use but may be underutilized for a variety of reasons, including preference of funders or lack of awareness of these designs. Promisingly, the prevalent use of quantitative and qualitative methods together reflects methodological innovation in newer D&I research.</p

Data_Sheet_1.docx

Background<p>The need for optimal study designs in dissemination and implementation (D&I) research is increasingly recognized. Despite the wide range of study designs available for D&I research, we lack understanding of the types of designs and methodologies that are routinely used in the field. This review assesses the designs and methodologies in recently proposed D&I studies and provides resources to guide design decisions.</p>Methods<p>We reviewed 404 study protocols published in the journal Implementation Science from 2/2006 to 9/2017. Eligible studies tested the efficacy or effectiveness of D&I strategies (i.e., not effectiveness of the underlying clinical or public health intervention); had a comparison by group and/or time; and used ≥1 quantitative measure. Several design elements were extracted: design category (e.g., randomized); design type [e.g., cluster randomized controlled trial (RCT)]; data type (e.g., quantitative); D&I theoretical framework; levels of treatment assignment, intervention, and measurement; and country in which the research was conducted. Each protocol was double-coded, and discrepancies were resolved through discussion.</p>Results<p>Of the 404 protocols reviewed, 212 (52%) studies tested one or more implementation strategy across 208 manuscripts, therefore meeting inclusion criteria. Of the included studies, 77% utilized randomized designs, primarily cluster RCTs. The use of alternative designs (e.g., stepped wedge) increased over time. Fewer studies were quasi-experimental (17%) or observational (6%). Many study design categories (e.g., controlled pre–post, matched pair cluster design) were represented by only one or two studies. Most articles proposed quantitative and qualitative methods (61%), with the remaining 39% proposing only quantitative. Half of protocols (52%) reported using a theoretical framework to guide the study. The four most frequently reported frameworks were Consolidated Framework for Implementing Research and RE-AIM (n = 16 each), followed by Promoting Action on Research Implementation in Health Services and Theoretical Domains Framework (n = 12 each).</p>Conclusion<p>While several novel designs for D&I research have been proposed (e.g., stepped wedge, adaptive designs), the majority of the studies in our sample employed RCT designs. Alternative study designs are increasing in use but may be underutilized for a variety of reasons, including preference of funders or lack of awareness of these designs. Promisingly, the prevalent use of quantitative and qualitative methods together reflects methodological innovation in newer D&I research.</p