Using Instructional Design, Analyze, Design, Develop, Implement, and Evaluate, to Develop e-Learning Modules to Disseminate Supported Employment for Community Behavioral Health Treatment Programs in New York State

BackgroundImplementation science lacks a systematic approach to the development of learning strategies for online training in evidence-based practices (EBPs) that takes the context of real-world practice into account. The field of instructional design offers ecologically valid and systematic processes to develop learning strategies for workforce development and performance support.ObjectiveThis report describes the application of an instructional design framework—Analyze, Design, Develop, Implement, and Evaluate (ADDIE) model—in the development and evaluation of e-learning modules as one strategy among a multifaceted approach to the implementation of individual placement and support (IPS), a model of supported employment for community behavioral health treatment programs, in New York State.MethodsWe applied quantitative and qualitative methods to develop and evaluate three IPS e-learning modules. Throughout the ADDIE process, we conducted formative and summative evaluations and identified determinants of implementation using the Consolidated Framework for Implementation Research (CFIR). Formative evaluations consisted of qualitative feedback received from recipients and providers during early pilot work. The summative evaluation consisted of levels 1 and 2 (reaction to the training, self-reported knowledge, and practice change) quantitative and qualitative data and was guided by the Kirkpatrick model for training evaluation.ResultsFormative evaluation with key stakeholders identified a range of learning needs that informed the development of a pilot training program in IPS. Feedback on this pilot training program informed the design document of three e-learning modules on IPS: Introduction to IPS, IPS Job development, and Using the IPS Employment Resource Book. Each module was developed iteratively and provided an assessment of learning needs that informed successive modules. All modules were disseminated and evaluated through a learning management system. Summative evaluation revealed that learners rated the modules positively, and self-report of knowledge acquisition was high (mean range: 4.4–4.6 out of 5). About half of learners indicated that they would change their practice after watching the modules (range: 48–51%). All learners who completed the level 1 evaluation demonstrated 80% or better mastery of knowledge on the level 2 evaluation embedded in each module. The CFIR was used to identify implementation barriers and facilitators among the evaluation data which facilitated planning for subsequent implementation support activities in the IPS initiative.ConclusionInstructional design approaches such as ADDIE may offer implementation scientists and practitioners a flexible and systematic approach for the development of e-learning modules as a single component or one strategy in a multifaceted approach for training in EBPs

Image_1_Using Instructional Design, Analyze, Design, Develop, Implement, and Evaluate, to Develop e-Learning Modules to Disseminate Supported Employment for Community Behavioral Health Treatment Programs in New York State.tiff

Background<p>Implementation science lacks a systematic approach to the development of learning strategies for online training in evidence-based practices (EBPs) that takes the context of real-world practice into account. The field of instructional design offers ecologically valid and systematic processes to develop learning strategies for workforce development and performance support.</p>Objective<p>This report describes the application of an instructional design framework—Analyze, Design, Develop, Implement, and Evaluate (ADDIE) model—in the development and evaluation of e-learning modules as one strategy among a multifaceted approach to the implementation of individual placement and support (IPS), a model of supported employment for community behavioral health treatment programs, in New York State.</p>Methods<p>We applied quantitative and qualitative methods to develop and evaluate three IPS e-learning modules. Throughout the ADDIE process, we conducted formative and summative evaluations and identified determinants of implementation using the Consolidated Framework for Implementation Research (CFIR). Formative evaluations consisted of qualitative feedback received from recipients and providers during early pilot work. The summative evaluation consisted of levels 1 and 2 (reaction to the training, self-reported knowledge, and practice change) quantitative and qualitative data and was guided by the Kirkpatrick model for training evaluation.</p>Results<p>Formative evaluation with key stakeholders identified a range of learning needs that informed the development of a pilot training program in IPS. Feedback on this pilot training program informed the design document of three e-learning modules on IPS: Introduction to IPS, IPS Job development, and Using the IPS Employment Resource Book. Each module was developed iteratively and provided an assessment of learning needs that informed successive modules. All modules were disseminated and evaluated through a learning management system. Summative evaluation revealed that learners rated the modules positively, and self-report of knowledge acquisition was high (mean range: 4.4–4.6 out of 5). About half of learners indicated that they would change their practice after watching the modules (range: 48–51%). All learners who completed the level 1 evaluation demonstrated 80% or better mastery of knowledge on the level 2 evaluation embedded in each module. The CFIR was used to identify implementation barriers and facilitators among the evaluation data which facilitated planning for subsequent implementation support activities in the IPS initiative.</p>Conclusion<p>Instructional design approaches such as ADDIE may offer implementation scientists and practitioners a flexible and systematic approach for the development of e-learning modules as a single component or one strategy in a multifaceted approach for training in EBPs.</p

Identifying clinically questionable psychotropic prescribing practices for medicaid recipients in New York state

OBJECTIVE: This study sought to identify and characterize rates of clinically questionable prescribing in New York State. METHODS: As part of a quality improvement initiative, 34 national psychopharmacology experts identified a set of questionable prescribing practices recognizable from pharmacy claims data. Indicators of such practices were applied to Medicaid claims data for 217,216 beneficiaries in New York State who had an active psychotropic prescription on April 1, 2008. RESULTS: A total of 156,103 (72%) of these beneficiaries had one or more continuing (\u3e90 days) prescriptions for a psychotropic. About 10% of adults were prescribed four or more psychotropics concurrently, and 13% of children and 2% of older adults were prescribed three or more concurrently. Prescribing an antipsychotic with a moderate-to-high risk of causing metabolic abnormalities approached 50% (46%) among individuals who had existing cardiometabolic conditions. Among beneficiaries prescribed second-generation antipsychotics with a moderate-to-high risk of causing metabolic abnormalities, over half (60%) had not received a metabolic screening test in the past year. Among women of reproductive age prescribed mood stabilizers, over one-quarter (30%) were prescribed a valproic acid-based formulation despite its potential for teratogenicity. Only 2% of youths under age 18 were prescribed benzodiazepines; however, about half (48%) had trials over 90 days\u27 duration. CONCLUSIONS: Examination of pharmacy claims from Medicaid beneficiaries in New York State indicated that prescribing practices deemed clinically questionable by pharmacology experts are common. Aggregated pharmacy claims data can identify such practices, and reviews of these data can be a core component of efforts to improve prescribing practices

Effectiveness of Switching From Antipsychotic Polypharmacy to Monotherapy

Objective:This randomized trial addressed the risks and benefits of staying on antipsychotic polypharmacy or switching to monotherapy. Method:Adult outpatients with schizophrenia taking two antipsychotics (127 participants across 19 sites) were randomly assigned to stay on polypharmacy or switch to monotherapy by discontinuing one antipsychotic. The trial lasted 6 months, with a 6-month naturalistic follow-up. Kaplan-Meier and Cox regression analyses examined time to discontinuation of assigned antipsychotic treatment, and random regression models examined additional outcomes over time. Results:Patients assigned to switch to monotherapy had shorter times to all-cause treatment discontinuation than those assigned to stay on polypharmacy. By month 6, 86% (N=48) of those assigned to stay on polypharmacy were still taking both medications, whereas 69% (N=40) of those assigned to switch to monotherapy were still taking the same medication. Most monotherapy discontinuations entailed returning to the original polypharmacy. The two groups did not differ with respect to psychiatric symptoms or hospitalizations. On average, the monotherapy group lost weight, whereas the polypharmacy group gained weight. Conclusions:Discontinuing one of two antipsychotics was followed by treatment discontinuation more often and more quickly than when both antipsychotics were continued. However, two-thirds of participants successfully switched, the groups did not differ with respect to symptom control, and switching to monotherapy resulted in weight loss. These results support the reasonableness of prescribing guidelines encouraging trials of antipsychotic monotherapy for individuals receiving antipsychotic polypharmacy, with the caveat that patients should be free to return to polypharmacy if an adequate trial on antipsychotic monotherapy proves unsatisfactory