Extension educators are expected to more fully evaluate programs. In response, evaluation capacity building (ECB) is a necessary component of Extension educator professional development. One size rarely fits all; ECB more likely succeeds if it is well aligned with the educator\u27s evaluation needs and the type of educational effort. Four approaches to Extension work have been documented—service, facilitation, content transmission, and transformative education. These approaches require different evaluation measures and therefore different forms of ECB

Archibald, Thomas

Franz, Nancy K.

English

Franz, Nancy

Clemson University: TigerPrints

Journal of Extension Volume 56 Number 4 Article 3 8-1-2018 Four Approaches to Building Extension Program Evaluation Capacity Nancy Franz Iowa State University Thomas Archibald Virginia PolytechnicInstitute and StateUniversity Recommended Citation Franz, N., & Archibald, T. (2018). Four Approaches to Building Extension Program Evaluation Capacity. Journal of Extension, 56(4). Retrieved from https://tigerprints.clemson.edu/joe/vol56/iss4/3 This Tools of the Trade is brought to you for free and open access by TigerPrints. It has been accepted for inclusion in Journal of Extension by an authorized editor of TigerPrints. For more information, please contact kokeefe@clemson.edu. August 2018Volume 56Number 4Article # 4TOT5Tools of the TradeFour Approaches to Building Extension Program EvaluationCapacityAbstractExtension educators are expected to more fully evaluate programs. In response, evaluation capacity building (ECB)is a necessary component of Extension educator professional development. One size rarely fits all; ECB more likelysucceeds if it is well aligned with the educator's evaluation needs and the type of educational effort. Fourapproaches to Extension work have been documented—service, facilitation, content transmission, andtransformative education. These approaches require different evaluation measures and therefore different forms ofECB.Keywords: evaluation capacity building, program, education, evaluative thinking, educational approaches Extension professionals are increasingly expected to more fully evaluate their programs (Nichols, Blake, Chazdon,& Radhakrishna, 2015; Rennekamp & Engle, 2008). In response, evaluation capacity building (ECB) is anecessary component of Extension educator professional development (Taylor-Powell & Boyd, 2008). One sizerarely fits all; ECB is more likely to succeed when it is well aligned with the educator’s evaluation needs, derivedfrom the type of educational effort he or she is using.Four approaches to Extension work have been documented—service, facilitation, content transmission, andtransformative education (Franz, 2013; Franz & Townson, 2008). These approaches require different evaluationmeasures and therefore different forms of ECB (Figure 1).Figure 1.Extension Educational Approaches and Their Implications for Evaluation Capacity BuildingNancy FranzProfessor EmeritusSchool of EducationIowa State UniversityAmes, Iowanfranz@iastate.edu@NancyfranzThomas ArchibaldAssistant Professorand ExtensionSpecialistDepartment ofAgricultural,Leadership, andCommunity EducationVirginia PolytechnicInstitute and StateUniversityBlacksburg, Virginiatgarch@vt.edu@tgarchibaldIntentional Program ECBAdapting the four quadrants proposed by Franz and Townson (2008) to include guidance on the types of ECBrequired by educators can help one develop intentional ECB for each type of educational approach. For example,if an educator engages in content transmission (IIa), an ECB approach limited to measuring customer satisfaction(Ib) would be inappropriate and ineffective. This differs from many past ECB approaches focused solely ongeneral evaluation concepts such as engaging stakeholders, focusing the evaluation, collecting and analyzingdata, and reporting evaluation results. Tailored ECB matched with educational approaches also moves away fromimplementing one-size-fits-all evaluation whereby the Extension professional seeks a survey used by anothereducator that he or she can simply adopt with few or no changes without determining first whether the surveyfits the situation at hand.Service-Focused ECBExtension systems as publicly funded entities have the responsibility or mandate to determine whether theireducational services benefit tax payers (Franz, 2011). This context often requires evaluating the quality ofcustomer service provided by Extension professionals. In some Extension systems, standardized surveys havebeen implemented across services and geography to aggregate customer service feedback statewide to report tolegislators and other stakeholders. ECB related to gaining useful customer service feedback should addressthe rationale for collecting service feedback from customers,intended uses of the data (i.e., local service quality, legislative mandate, Extension professional performance,Tools of the Trade Four Approaches to Building Extension Program Evaluation Capacity JOE 56(4)©2018 Extension Journal Inc. 2etc.),alignment of feedback collection methods and analysis with the evaluation rationale and uses,identification of customer feedback catalyzers and inhibitors to improve the amount and quality/authenticity offeedback, andways to integrate the collecting and using of customer service feedback into processes for providing services.Facilitation-Focused ECBExtension professionals have increasingly become known for their ability to facilitate group processes (French &Morse, 2015). This approach to Extension education requires evaluation that measures the success of facilitatedprocesses used and related outputs to help the group reach its goals. ECB related to determining theeffectiveness of facilitated education should includeways to determine whether facilitation contributed toward the group's reaching its goals,the effectiveness of group process and engagement techniques,determination of whether the makeup or perspectives of the group are appropriate for the group to reach itsgoals (i.e., inclusion or exclusion of particular views or experiences),the unique roles or value of Extension in facilitating group processes, andthe value of outputs created by facilitation (i.e., participation, inclusion, products).Content Transmission–Focused ECBThis type of ECB is most common in Extension focused on a specific discipline, topic, communication method, orprogram. With this type of evaluation practice, evaluators have relied mostly on conducting preprogram andpostprogram surveys to determine knowledge gain from an educational opportunity. Sometimes the intent tochange behavior by the learner is measured. ECB related to determining successful content transmission focusesonreasons for collecting content dissemination feedback (i.e., to improve information quality, to improvedissemination methods, to change opinions or behavior, to fulfill funding requirements, to change conditions,etc.) and aligning evaluation accordingly;the degree to which information is accessible (methods, language, etc.) and used;what happens when learners use the information provided;timing of information dissemination for enhanced use of information;effectiveness and value of dissemination methods; andTools of the Trade Four Approaches to Building Extension Program Evaluation Capacity JOE 56(4)©2018 Extension Journal Inc. 3the value of Extension's unique content (i.e., research based, best practices, etc.).Transformative Education–Focused ECBMany Extension professionals are increasingly interested in articulating the impact of their educational efforts oneconomic, environment, and social condition changes at the individual, family, group, business, or communitylevel. ECB related to determining impact on condition change requires that Extension professionals adoptevaluative thinking (Buckley, Archibald, Hargraves, & Trochim, 2015) as part of program design, implementation,and evaluation as this type of change requires attention to both Extension educational processes and content.ECB related to measuring condition change through transformative education addressesstrong needs assessment that determines the current situation, the desired condition, and ways Extensioneducation can bridge the gap;the building, maintaining, and evaluating of strong program evaluation partnerships (i.e., condition changerequires a multipronged approach involving many educational efforts, often by many entities);integration of evaluative thinking into all aspects of the Extension program development model;gathering of impact data at multiple points in time to show when and for how long change takes place; andcreation and sharing of condition change stories.SummaryThe types of evaluation conducted by Extension professionals need to match the educational approaches theyuse. Achieving this goal requires that ECB for Extension professionals focus more specifically on fit witheducational approaches and less on basic evaluation steps or procedures. More intentional ECB should result inmore effective and efficient program evaluation for all evaluation stakeholders. One-size ECB does not fit allExtension professionals or their approaches to improving program quality or determining program impact.ReferencesBuckley, J., Archibald, T., Hargraves, M., & Trochim, W. M. (2015). Defining and teaching evaluative thinking:Insights from research on critical thinking. American Journal of Evaluation, 36(3), 375–388.doi:10.1177/1098214015581706Franz, N. (2011). Advancing the public value movement: Sustaining Extension during tough times. Journal ofExtension, 49(2), Article 2COM2. Available at: https://www.joe.org/joe/2011april/comm2.phpFranz, N. (2013). Improving Extension programs: Putting public value stories and statements to work. Journal ofExtension, 51(3), Article 3TOT1. Available at: http://www.joe.org/joe/2013june/tt1.phpFranz, N., & Townson, L. (2008). The nature of complex organizations: The case of Cooperative Extension. In M.T. Braverman, M. Engle, M. E. Arnold, & R. A. Rennekamp (Eds.), Program evaluation in a complex organizationalsystem: Lessons from Cooperative Extension (pp. 5–14). New Directions for Evaluation, 120. San Francisco, CA:Jossey-Bass. doi:10.1002/ev.272Tools of the Trade Four Approaches to Building Extension Program Evaluation Capacity JOE 56(4)©2018 Extension Journal Inc. 4French, C., & Morse, G. (2015). Extension stakeholder engagement: An exploration of two cases exemplifying21st century adaptations. Journal of Human Sciences and Extension, 3(2), 108–131.Nichols, A., Blake, S., Chazdon, S., & Radhakrishna, R. (2015). From farm results demonstrations to multistateimpact designs: Cooperative Extension navigates its way through evaluation pathways. Journal of HumanSciences and Extension, 3(2), 83–107.Rennekamp, R. A., & Engle, M. (2008). Case study for change: Evaluation in Cooperative Extension. In M. T.Braverman, M. Engle, M. E. Arnold, & R. A. Rennekamp (Eds.), Program evaluation in a complex organizationalsystem: Lessons from Cooperative Extension (pp. 15–26). New Directions for Evaluation, 120. San Francisco, CA:Jossey-Bass. doi:10.1002/ev.273Taylor-Powell, E., & Boyd, H. (2008). Evaluation capacity building in complex organizations. In M. T. Braverman,M. Engle, M. E. Arnold, & R. A. Rennekamp (Eds.), Program evaluation in a complex organizational system:Lessons from Cooperative Extension (pp. 55–69). New Directions in Evaluation, 120. San Francisco, CA: Jossey-Bass. doi: 10.1002/ev.276Copyright © by Extension Journal, Inc. ISSN 1077-5315. Articles appearing in the Journal become the property ofthe Journal. Single copies of articles may be reproduced in electronic or print form for use in educational ortraining activities. Inclusion of articles in other publications, electronic sources, or systematic large-scaledistribution may be done only with prior electronic or written permission of the Journal Editorial Office, joe-ed@joe.org.If you have difficulties viewing or printing this page, please contact JOE Technical SupportTools of the Trade Four Approaches to Building Extension Program Evaluation Capacity JOE 56(4)©2018 Extension Journal Inc. 5

Four Approaches to Building Extension Program Evaluation Capacity

https://lib.dr.iastate.edu/cgi/viewcontent.cgi?article=1119&amp;context=edu_pubs

Four Approaches to Building Extension Program Evaluation Capacity

Abstract

Similar works

Full text

Available Versions

Clemson University: TigerPrints