RE-AIM is an evaluation framework that has been widely used by public health professionals for over 15 years and is well documented in public health literature. RE-AIM evaluates health promotion programs on five dimensions—reach, effectiveness, adoption, implementation, and maintenance. Although some Extension professionals have used the framework, it has been undocumented in Extension literature. To encourage wider use of RE-AIM in Extension, we briefly describe the framework and present an application of it to a two-state, grant-funded 4-H Healthy Living program. We conclude by identifying benefits of more widespread use of RE-AIM within Extension

Donaldson, Joseph L.

Downey, Laura H.

Hardman, Alisha

Peterson, Donna J.

English

Clemson University: TigerPrints

Journal of Extension Volume 55 Number 3 Article 25 6-1-2017 An Extension Application of the RE-AIM Evaluation Framework Laura H. Downey Mississippi State University Donna J. Peterson Mississippi State University Joseph L. Donaldson Mississippi State University Alisha Hardman Mississippi State University Recommended Citation Downey, L. H., Peterson, D. J., Donaldson, J. L., & Hardman, A. (2017). An Extension Application of the RE-AIM Evaluation Framework. Journal of Extension, 55(3), Article 25. https://tigerprints.clemson.edu/joe/vol55/iss3/25 This Ideas at Work is brought to you for free and open access by TigerPrints. It has been accepted for inclusion in Journal of Extension by an authorized editor of TigerPrints. For more information, please contact kokeefe@clemson.edu. June 2017Volume 55Number 3Article # 3IAW1Ideas at WorkAn Extension Application of the RE-AIM Evaluation FrameworkAbstractRE-AIM is an evaluation framework that has been widely used by public health professionals for over 15 yearsand is well documented in public health literature. RE-AIM evaluates health promotion programs on fivedimensions—reach, effectiveness, adoption, implementation, and maintenance. Although some Extensionprofessionals have used the framework, it has been undocumented in Extension literature. To encourage wideruse of RE-AIM in Extension, we briefly describe the framework and present an application of it to a two-state,grant-funded 4-H Healthy Living program. We conclude by identifying benefits of more widespread use of RE-AIMwithin Extension.   RE-AIM is an evaluation framework that has been widely used by public health professionals for over 15 years.RE-AIM identifies five evaluation dimensions—reach, effectiveness, adoption, implementation, andmaintenance—and is considered a comprehensive model of evaluation. The usefulness of RE-AIM has been wellestablished in public health journals but has remained undocumented in journals for which Extensionprofessionals are the target audience (Folta et al., 2015; Glasgow, Vogt, & Boles, 1999).With the purpose of informing Extension professionals about RE-AIM, we briefly describe and present anapplication of the framework. We conclude with insights about the potential of applying RE-AIM for evaluationacross Extension programming.A Description of RE-AIMRE-AIM was developed for the purpose of evaluating health behavior research but has evolved into a toolpractitioners use to plan and assess programs and report program results (Gaglio, Shoup, & Glasgow, 2013).This framework allows programs to be evaluated at the individual or participant level (reach, effectiveness, andmaintenance) and at the organizational level (adoption, implementation, and maintenance) (National CancerInstitute, 2012; RE-AIM, 2013).Reach is the number, proportion, and representativeness of intervention participants.Effectiveness (or efficacy) is the impact of the intervention relative to important outcomes.Laura H. DowneyEvaluation SpecialistMississippi StateUniversityMississippi State,Mississippilaura.downey@msstate.eduDonna J. PetersonEvaluation SpecialistMississippi StateUniversityMississippi State,Mississippidonna.peterson@msstate.eduJoseph L. DonaldsonAssistant ProfessorUniversity ofTennesseeKnoxville, Tennesseejldonaldson@tennessee.eduAlisha HardmanEvaluation SpecialistMississippi StateUniversityMississippi State,Mississippia.hardman@msstate.eduAdoption is the number, proportion, and representativeness of entities or intervention agents willing toinitiate the intervention.Implementation is the fidelity to various elements of the intervention's protocol.Maintenance at the individual level is the long-term impact of the intervention in terms of program outcomesand at the organizational level is the extent to which the intervention becomes institutionalized.Application of RE-AIM to a 4-H Health Promotion ProgramThe U.S. Department of Agriculture provided funding to expand 4-H Healthy Living programming to youths inselected rural counties in Mississippi and Tennessee. The Jump into Foods and Fitness curriculum (JIFF) (Baird,Branta, Mark, & Seremba, 2003), developed by Michigan State University Extension Service, was adopted forreplication in these counties because evidence suggested that JIFF increases 4-H'ers frequencies of eatingbreakfast daily, vegetable intake levels, amounts of moderate physical activity, and enjoyment of beingphysically active (Downey, Peterson, Le Menestrel, Leatherman, & Lang, 2014). Also, there was evidence ofstates having adopted JIFF as a research-based educational curriculum (Gossett, 2012).Prior to implementation of JIFF in the selected counties, 4-H professionals and volunteers who would beimplementing the program received face-to-face training to enhance their relevant knowledge and skills. Partof the training focused on the project's evaluation protocol, which was based on the five RE-AIM dimensions.Because JIFF had not been implemented previously in Mississippi and Tennessee, RE-AIM was used as astandardized approach for evaluating the program in the participating counties, settings that greatly differ fromthe one in which the program was developed. Specifically, the RE-AIM framework was used for measuringprogram reach, effectiveness, adoption, implementation, and maintenance as follows:Reach was measured by the percentage of youths who participated in JIFF, based on a valid denominator oftotal target population. For example, the number of third graders who participated in JIFF was compared tothe total number of third graders at a participating institution. Sign-in sheets were used for obtainingnumbers of youth participants, and secondary data sources (i.e., school records) were used for documentingthe total number of youths eligible to participate.Effectiveness was assessed at two levels. First, the effectiveness of the training for 4-H professionals andvolunteers in Mississippi and Tennessee was assessed through a posttest survey completed at the end of thetraining and through an observation checklist completed as each 4-H professional or volunteer taught a JIFFlesson at the training. Secondly, effectiveness of JIFF on youth participants was assessed through the use ofa pretest/posttest based on 4-H Common Measures (National 4-H Council, 2015). The pretest wasimplemented prior to the start of the program and repeated as a posttest at the end of the program.Adoption was assessed by comparing the number of settings, for example schools or Boys and Girls Clubs,approached to participate with the number of settings eligible to participate (i.e., the number of elementaryschools and out-of-school programs for children in each county).Implementation was assessed by documenting adherence to the JIFF protocol through a 4-H professionalIdeas at Work An Extension Application of the RE-AIM Evaluation Framework JOE 55(3)©2017 Extension Journal Inc. 1and volunteer readiness assessment completed after the training, an implementation checklist for 4-Hprofessionals and volunteers to use to self-report on implementation, and an observation checklist forevaluation specialists to use on periodic site visits during program implementation.Maintenance was assessed at the individual level through a follow-up survey administered to youths 3months after the posttest. It is too early to assess maintenance at the organizational level because projectfunding continues. Maintenance at the organizational level will be demonstrated if JIFF continues to beimplemented after the grant ends.ConclusionsOur experience with the RE-AIM framework suggests that it is a feasible tool for informing programmodifications for enhanced success and sustainability. For example, examining the checklists used for theimplementation dimension of the evaluation could help identify adaptations made by individual educators thatwould be beneficial across all sites. Evaluation based on RE-AIM is a straightforward approach for comparingpublic health outcomes across multiple states or sites. The framework allowed us to develop a detailedevaluation protocol, on which agents were trained so that they were prepared to properly collect evaluationdata. Consistent data collected across sites made comparisons in outcomes possible. Although we used theframework to evaluate only one health promotion program, it could be used to evaluate multiple programs forthe purpose of identifying those that are most effective. Use of 4-H Common Measures could make thispossible. If RE-AIM were used for evaluating several programs, effectiveness could inform decisions aboutresource distribution toward future program implementation across one state or multiple states.RE-AIM could serve these same evaluative purposes for other Extension programs as well. It is most intuitiveto apply RE-AIM to other health-related programs in community resource and economic development andfamily and consumer sciences. Yet the dimensions of RE-AIM are applicable to any educational program,including those in agriculture and natural resources. It will always be important for Extension educators toknow whether they have reached their target audiences and whether their programs have achieved theintended outcomes.AcknowledgmentFunding for the 4-H Healthy Living program described in this article was provided by the U.S. Department ofAgriculture through the Rural Health and Safety Education Competitive Grants Program.ReferencesBaird, K., Branta, C., Mark, C., & Seremba, D. (2003). Jump into foods and fitness. East Lansing, MI: MichiganState University Extension, 4-H Youth Development, Family & Consumer Sciences.Downey, L. H., Peterson, D. J., Le Menestrel, S., Leatherman, J., & Lang, J. (2014). 4-H Healthy Livingprograms with impact: A national environmental scan. New Directions for Youth Development, 143, 13–24.doi:10.1002/yd.20101Folta, S. C., Lichtenstein, A. H., Seguin, R. A., Goldberg, J. P., Corbin, M. A., Wiker, N., . . . Nelson, M. E.(2015). The Strong Women-Healthy Hearts program in Pennsylvania: RE-AIM analysis. TranslationalBehavioral Medicine, 5(1), 94–102. doi:10.1007/s13142-014-0286-8Ideas at Work An Extension Application of the RE-AIM Evaluation Framework JOE 55(3)©2017 Extension Journal Inc. 2Gaglio, B., Shoup, J. A., & Glasgow, R. E. (2013). The RE-AIM framework: A systematic review of use overtime. American Journal of Public Health, 103(6), E38–E46. doi:10.2105/AJPH.2013.301299Glasgow, R. E., Vogt, T. M., & Boles, S. M. (1999). Evaluating the public health impact of health promotioninterventions: The RE-AIM framework. American Journal of Public Health, 89(9), 1322–1327.doi:10.2105/AJPH.89.9.1322Gossett, L. S. (2012). EFNEP reaches refugee youth using a mobile van. Journal of Extension, 50(1), Article1IAW2. Available at: http://www.joe.org/joe/2012february/iw2.phpNational 4-H Council. (2015). Common Measures. Retrieved from http://4-h.org/professionals/common-measures/National Cancer Institute. (2012). Measuring the use of the RE-AIM model dimension items-checklist.Retrieved from http://www.re-aim.org/forms/re-aim_framework_constructs_checklist.pdfRE-AIM. (2013). What is RE-AIM? Retrieved from http://www.re-aim.hnfe.vt.edu/about_re-aim/what_is_re-aim/index.htmlCopyright © by Extension Journal, Inc. ISSN 1077-5315. Articles appearing in the Journal become the propertyof the Journal. Single copies of articles may be reproduced in electronic or print form for use in educational ortraining activities. Inclusion of articles in other publications, electronic sources, or systematic large-scaledistribution may be done only with prior electronic or written permission of the Journal Editorial Office, joe-ed@joe.org.If you have difficulties viewing or printing this page, please contact JOE Technical SupportIdeas at Work An Extension Application of the RE-AIM Evaluation Framework JOE 55(3)©2017 Extension Journal Inc. 3

An Extension Application of the RE-AIM Evaluation Framework

https://tigerprints.clemson.edu/cgi/viewcontent.cgi?article=1788&context=joe

An Extension Application of the RE-AIM Evaluation Framework

Abstract

Similar works

Full text

Available Versions

Clemson University: TigerPrints