Going above and beyond for implementation: the development and validity testing of the Implementation Citizenship Behavior Scale (ICBS).

BackgroundIn line with recent research on the role of the inner context of organizations in implementation effectiveness, this study extends research on organizational citizenship behavior (OCB) to the domain of evidence-based practice (EBP) implementation. OCB encompasses those behaviors that go beyond what is required for a given job that contribute to greater organizational effectiveness. The goal of this study was to develop and test a measure of implementation citizenship behavior (ICB) or those behaviors that employees perform that go above and beyond what is required in order to support EBP implementation.MethodsThe primary participants were 68 supervisors from ten mental health agencies throughout California. Items measuring ICB were developed based on past research on OCB and in consultation with experts on EBP implementation in mental health settings. Supervisors rated 357 of their subordinates on ICB and implementation success. In addition, 292 of the subordinates provided data on self-rated performance, attitudes towards EBPs, work experience, and full-time status. The supervisor sample was randomly split, with half used for exploratory factor analyses and the other half for confirmatory factor analyses. The entire sample of supervisors and subordinates was utilized for analyses assessing the reliability and construct validity of the measure.ResultsExploratory factor analyses supported the proposed two-factor structure of the Implementation Citizenship Behavior Scale (ICBS): (1) Helping Others and (2) Keeping Informed. Confirmatory factor analyses with the other half of the sample supported the factor structure. Additional analyses supported the reliability and construct validity for the ICBS.ConclusionsThe ICBS is a pragmatic brief measure (six items) that captures critical behaviors employees perform to go above and beyond the call of duty to support EBP implementation, including helping their fellow employees on implementation-related activities and keeping informed about issues related to EBP and implementation efforts. The ICBS can be used by researchers to better understand the outcomes of improved organizational support for implementation (i.e., implementation climate) and the proximal predictors of implementation effectiveness. The ICBS can also provide insight for organizations, practitioners, and managers by focusing on key employee behaviors that should increase the probability of implementation success

Assessing the organizational context for EBP implementation: the development and validity testing of the Implementation Climate Scale (ICS).

BackgroundAlthough the importance of the organizational environment for implementing evidence-based practices (EBP) has been widely recognized, there are limited options for measuring implementation climate in public sector health settings. The goal of this research was to develop and test a measure of EBP implementation climate that would both capture a broad range of issues important for effective EBP implementation and be of practical use to researchers and managers seeking to understand and improve the implementation of EBPs.MethodsParticipants were 630 clinicians working in 128 work groups in 32 US-based mental health agencies. Items to measure climate for EBP implementation were developed based on past literature on implementation climate and other strategic climates and in consultation with experts on the implementation of EBPs in mental health settings. The sample was randomly split at the work group level of analysis; half of the sample was used for exploratory factor analysis (EFA), and the other half was used for confirmatory factor analysis (CFA). The entire sample was utilized for additional analyses assessing the reliability, support for level of aggregation, and construct-based evidence of validity.ResultsThe EFA resulted in a final factor structure of six dimensions for the Implementation Climate Scale (ICS): 1) focus on EBP, 2) educational support for EBP, 3) recognition for EBP, 4) rewards for EBP, 5) selection for EBP, and 6) selection for openness. This structure was supported in the other half of the sample using CFA. Additional analyses supported the reliability and construct-based evidence of validity for the ICS, as well as the aggregation of the measure to the work group level.ConclusionsThe ICS is a very brief (18 item) and pragmatic measure of a strategic climate for EBP implementation. It captures six dimensions of the organizational context that indicate to employees the extent to which their organization prioritizes and values the successful implementation of EBPs. The ICS can be used by researchers to better understand the role of the organizational context on implementation outcomes and by organizations to evaluate their current climate as they consider how to improve the likelihood of implementation success

Validation of the Implementation Climate Scale (ICS) in substance use disorder treatment organizations.

BackgroundOne critical factor in the implementation of evidence-based practice (EBP) in substance use disorder treatment organizations is an inner organizational context that clearly supports implementation efforts. The Implementation Climate Scale (ICS) has been developed to allow researchers and organizations to assess climate for EBP implementation in health and allied health service organizations. The ICS consists of 18 items and measures six dimensions of implementation climate: focus on EBP, educational support for EBP, recognition for EBP, rewards for EBP, selection for EBP, and selection for openness. The ICS was initially developed in a mental health context; thus, the goal of this study was to provide initial validation of the ICS in substance use disorder (SUD) treatment settings.MethodsConfirmatory factor analysis (CFA) was used to assess the psychometric functioning of the ICS using survey data from 326 providers in 65 teams in SUD treatment programs. Cronbach's alpha was examined to assess internal consistency of the ICS, and individual and team level construct-based validity was examined by comparing its correlations with service climate, molar climate, and organizational change.ResultsWe found evidence for the reliability, factor structure, and validity of the ICS in SUD services. The psychometric functioning of the ICS in SUD treatment settings was comparable to that found in mental health contexts.ConclusionsThe ICS is a brief and pragmatic tool for researchers to better understand a critical antecedent for implementation effectiveness in SUD treatment and for organizational leaders in SUD treatment organizations to evaluate the extent to which providers perceive that their organization supports EBP implementation

Psychometric properties of the Dutch version of the Evidence-Based Practice Attitude Scale (EBPAS).

BackgroundThe Evidence-Based Practice Attitude Scale (EBPAS) was developed in the United States to assess attitudes of mental health and welfare professionals toward evidence-based interventions. Although the EBPAS has been translated in different languages and is being used in several countries, all research on the psychometric properties of the EBPAS within youth care has been carried out in the United States. The purpose of this study was to investigate the psychometric properties of the Dutch version of the EBPAS.MethodsAfter translation into Dutch, the Dutch version of the EBPAS was examined in a diverse sample of 270 youth care professionals working in five institutions in the Netherlands. We examined the factor structure with both exploratory and confirmatory factor analyses and the internal consistency reliability. We also conducted multiple linear regression analyses to examine the association of EBPAS scores with professionals' characteristics. It was hypothesized that responses to the EBPAS items could be explained by one general factor plus four specific factors, good to excellent internal consistency reliability would be found, and EBPAS scores would vary by age, sex, and educational level.ResultsThe exploratory factor analysis suggested a four-factor solution according to the hypothesized dimensions: Requirements, Appeal, Openness, and Divergence. Cronbach's alphas ranged from 0.67 to 0.89, and the overall scale alpha was 0.72. The confirmatory factor analyses confirmed the factor structure and suggested that the lower order EBPAS factors are indicators of a higher order construct. However, Divergence was not significantly correlated with any of the subscales or the total score. The confirmatory bifactor analysis endorsed that variance was explained both by a general attitude towards evidence-based interventions and by four specific factors. The regression analyses showed an association between EBPAS scores and youth care professionals' age, sex, and educational level.ConclusionsThe present study provides strong support for a structure with a general factor plus four specific factors and internal consistency reliability of the Dutch version of the EBPAS in a diverse sample of youth care professionals. Hence, the factor structure and reliability of the original version of the EBPAS seem generalizable to the Dutch version of the EBPAS

The implementation leadership scale (ILS): development of a brief measure of unit level implementation leadership

BACKGROUND: In healthcare and allied healthcare settings, leadership that supports effective implementation of evidenced-based practices (EBPs) is a critical concern. However, there are no empirically validated measures to assess implementation leadership. This paper describes the development, factor structure, and initial reliability and convergent and discriminant validity of a very brief measure of implementation leadership: the Implementation Leadership Scale (ILS). METHODS: Participants were 459 mental health clinicians working in 93 different outpatient mental health programs in Southern California, USA. Initial item development was supported as part of a two United States National Institutes of Health (NIH) studies focused on developing implementation leadership training and implementation measure development. Clinician work group/team-level data were randomly assigned to be utilized for an exploratory factor analysis (n = 229; k = 46 teams) or for a confirmatory factor analysis (n = 230; k = 47 teams). The confirmatory factor analysis controlled for the multilevel, nested data structure. Reliability and validity analyses were then conducted with the full sample. RESULTS: The exploratory factor analysis resulted in a 12-item scale with four subscales representing proactive leadership, knowledgeable leadership, supportive leadership, and perseverant leadership. Confirmatory factor analysis supported an a priori higher order factor structure with subscales contributing to a single higher order implementation leadership factor. The scale demonstrated excellent internal consistency reliability as well as convergent and discriminant validity. CONCLUSIONS: The ILS is a brief and efficient measure of unit level leadership for EBP implementation. The availability of the ILS will allow researchers to assess strategic leadership for implementation in order to advance understanding of leadership as a predictor of organizational context for implementation. The ILS also holds promise as a tool for leader and organizational development to improve EBP implementation

Atom-projected and angular momentum resolved density of states in the ONETEP code

Local and angular momentum projected densities of states (DOS) are invaluable sources of information that can be obtained from density functional theory calculations. In this work, we describe a theoretical framework within ONETEP's linear-scaling DFT formalism that allows the calculation of local (atom-projected) and angular momentum projected density of states l-p-DOS. We describe four different bases that can be used for projecting the DOS with angular momentum resolution and perform a set of tests to compare them. We validate the results obtained with ONETEP's l-p-DOS against the plane-wave DFT code CASTEP. Comparable results between ONETEP's and CASTEP's charge spilling parameters are observed when we use pseudo-atomic orbitals as the projection basis sets. In general, the charge spilling parameters show remarkably low values for projections using non-contracted spherical waves as the angular momentum resolved basis. We also calculate the d-band and d-band centres for Pt atoms in (1 1 1) facets of cuboctahedral Pt nanoparticles of increasing size, which is an example of l-p-DOS application commonly used as an electronic descriptor in heterogeneous catalysis. Interestingly, the different projection bases lead to similar conclusions, showing the reliability of the implemented method for such studies. The implementation of these methods in a linear-scaling framework such as ONETEP provides another tool for analysing the electronic structure of complex nanostructured materials

Mixed-method study of a conceptual model of evidence-based intervention sustainment across multiple public-sector service settings.

BackgroundThis study examines sustainment of an EBI implemented in 11 United States service systems across two states, and delivered in 87 counties. The aims are to 1) determine the impact of state and county policies and contracting on EBI provision and sustainment; 2) investigate the role of public, private, and academic relationships and collaboration in long-term EBI sustainment; 3) assess organizational and provider factors that affect EBI reach/penetration, fidelity, and organizational sustainment climate; and 4) integrate findings through a collaborative process involving the investigative team, consultants, and system and community-based organization (CBO) stakeholders in order to further develop and refine a conceptual model of sustainment to guide future research and provide a resource for service systems to prepare for sustainment as the ultimate goal of the implementation process.MethodsA mixed-method prospective and retrospective design will be used. Semi-structured individual and group interviews will be used to collect information regarding influences on EBI sustainment including policies, attitudes, and practices; organizational factors and external policies affecting model implementation; involvement of or collaboration with other stakeholders; and outer- and inner-contextual supports that facilitate ongoing EBI sustainment. Document review (e.g., legislation, executive orders, regulations, monitoring data, annual reports, agendas and meeting minutes) will be used to examine the roles of state, county, and local policies in EBI sustainment. Quantitative measures will be collected via administrative data and web surveys to assess EBI reach/penetration, staff turnover, EBI model fidelity, organizational culture and climate, work attitudes, implementation leadership, sustainment climate, attitudes toward EBIs, program sustainment, and level of institutionalization. Hierarchical linear modeling will be used for quantitative analyses. Qualitative analyses will be tailored to each of the qualitative methods (e.g., document review, interviews). Qualitative and quantitative approaches will be integrated through an inclusive process that values stakeholder perspectives.DiscussionThe study of sustainment is critical to capitalizing on and benefiting from the time and fiscal investments in EBI implementation. Sustainment is also critical to realizing broad public health impact of EBI implementation. The present study takes a comprehensive mixed-method approach to understanding sustainment and refining a conceptual model of sustainment

Systematic review of the Exploration, Preparation, Implementation, Sustainment (EPIS) framework

Background: Effective implementation of evidence-based practices (EBPs) remains a significant challenge. Numerous existing models and frameworks identify key factors and processes to facilitate implementation. However, there is a need to better understand how individual models and frameworks are applied in research projects, how they can support the implementation process, and how they might advance implementation science. This systematic review examines and describes the research application of a widely used implementation framework, the Exploration, Preparation, Implementation, Sustainment (EPIS) framework. Methods: A systematic literature review was performed to identify and evaluate the use of the EPIS framework in implementation efforts. Citation searches in PubMed, Scopus, PsycINFO, ERIC, Web of Science, Social Sciences Index, and Google Scholar databases were undertaken. Data extraction included the objective, language, country, setting, sector, EBP, study design, methodology, level(s) of data collection, unit(s) of analysis, use of EPIS (i.e., purpose), implementation factors and processes, EPIS stages, implementation strategy, implementation outcomes, and overall depth of EPIS use (rated on a 1-5 scale). Results: In total, 762 full-text articles were screened by four reviewers, resulting in inclusion of 67 articles, representing 49 unique research projects. All included projects were conducted in public sector settings. The majority of projects (73%) investigated the implementation of a specific EBP. The majority of projects (90%) examined inner context factors, 57% examined outer context factors, 37% examined innovation factors, and 31% bridging factors (i.e., factors that cross or link the outer system and inner organizational context). On average, projects measured EPIS factors across two of the EPIS phases (M = 2.02), with the most frequent phase being Implementation (73%). On average, the overall depth of EPIS inclusion was moderate (2.8 out of 5). Conclusion: This systematic review enumerated multiple settings and ways the EPIS framework has been applied in implementation research projects, and summarized promising characteristics and strengths of the framework, illustrated with examples. Recommendations for future use include more precise operationalization of factors, increased depth and breadth of application, development of aligned measures, and broadening of user networks. Additional resources supporting the operationalization of EPIS are available