A Mixed Methods Multiple Case Study of Implementation as Usual in Children\u27s Social Service Organizations

Abstract

Increasing the adoption and sustainment of evidence-based treatments (EBTs) in children\u27s mental health and social service systems will require the development of evidence-based implementation strategies. In order to ensure that these strategies are feasible, acceptable, sustainable, and scalable, efforts to identify and develop implementation strategies will need to be grounded by a thorough understanding of real world service systems as well as organizational stakeholders\u27 preferences for particular strategies. In other words, there is a need for a better understanding of usual care settings, and in particular, what constitutes implementation as usual. This study employed a mixed methods, multiple-case study of six organizations that provide social and mental health services to children and youth in a Midwestern city to describe the state of implementation as usual in children\u27s social services, evaluate the extent to which implementation as usual reflects emerging best practices specified in the implementation literature, and inform the future development of implementation strategies that will be practical and effective. The specific aims of this study were: (1) to identify and characterize the implementation strategies used; (2) to explore how organizational leaders make decisions about which treatments and programs to implement and how to implement them; (3) to assess stakeholders\u27 (organizational leaders and clinicians) perceptions of the effectiveness, comparative effectiveness, feasibility, and appropriateness of implementation strategies; and (4) to examine the relationship between organizational social context (culture and climate) and implementation strategy selection, implementation decision making, and perceptions of implementation strategies. These aims were accomplished through semi-structured interviews, focus groups, document review, an online survey of stakeholders\u27 perceptions of implementation strategies, and a standardized measure of organizational social context. Organizations considered a range of factors when making treatment and implementation decisions. While some considered empirical evidence to make decisions about which treatments to implement, they rarely considered empirical evidence when considering how to implement interventions. Across organizations, provider-focused strategies (e.g., training, supervision) were dominant; however, many of these strategies were not offered at the frequency and intensity that is generally required to implement EBTs effectively. Multiple areas of implementation were not well addressed, including process, client, organizational, financial, and policy levels. Several problematic trends related to strategy use were identified, such as the inconsistent provision of training and supervision, monitoring fidelity in ways not thought to be helpful, and failing to measure or appropriately utilize clinical outcome data. Stakeholders generally perceived active implementation strategies to be more effective than passive strategies, and did not respond well to strategies that were punitive in nature. Findings demonstrate how organizational social context can impact implementation processes and stakeholders\u27 perceptions of the effectiveness of implementation strategies. Important implications for practice, policy and research were derived

    Similar works