The Cooperative Extension organization could be categorized as a participatory organization because of our focus on local needs in driving our work. The pandemic moved a lot of our work to online platforms. This article reflects on use of online tools for program evaluation and needs assessment. Online focus groups were successful in gathering high quality data from a diverse and geographically dispersed population for program evaluation and as part of a needs assessment process. Coupling the focus groups with a modified World Café approach allowed us to go beyond identification of needs to prioritization and development of potential solutions

Vines, Karen A

English

Clemson University: TigerPrints

The Journal of Extension Volume 60 Number 3 Summer 2022 Article 14 9-21-2022 Participatory Evaluation and Needs Assessment Tools for Cooperative Extension in a Virtual Environment Karen A. Vines Virginia Tech, kvines@vt.edu This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 License. Recommended Citation Vines, K. A. (2022). Participatory Evaluation and Needs Assessment Tools for Cooperative Extension in a Virtual Environment. The Journal of Extension, 60(3), Article 14. https://doi.org/10.34068/joe.60.03.14 This Tools of the Trade is brought to you for free and open access by the Conferences at TigerPrints. It has been accepted for inclusion in The Journal of Extension by an authorized editor of TigerPrints. For more information, please contact kokeefe@clemson.edu. Participatory Evaluation and Needs Assessment Tools for Cooperative Extension in a Virtual Environment Cover Page Footnote Thank you to members of the research teams for these two projects. They include Sarah Baughman, Neil Clark, Cynthia Gregg, Jane Henderson, Lonnie Johnson, Subrato Kuri, Ruth Wallace. Thanks also to national Epsilon Sigma Phi for providing a Professional Development grant to support the Professional Development Needs of VCE Mid-Career Extension Professionals. Finally, thank you to the participants in the focus groups. This tools of the trade is available in The Journal of Extension: https://tigerprints.clemson.edu/joe/vol60/iss3/14 Journal of Extension          Tools of the Trade Volume 60, Issue 3, 2022Participatory Evaluation and Needs Assessment Tools for Cooperative Extension in a Virtual EnvironmentKaren Vines1AUTHORS: 1Virginia Tech.USING ONLINE FOCUS GROUPS TO GATHER DATA FOR COOPERATIVE EXTENSIONFocus groups have been used in Extension to improve (Vanderford et al., 2014) and evaluate (White et al., 2008) programs, and to conduct needs assessment (Vines et al., 2018). Face-to-face focus groups generally limit participants based on their willingness to travel. This condition was further complicated by the COVID-19 pandemic, which resulted in travel bans. However, access to funds and time can also create challenges regarding travel. Moving face-to-face focus group protocol to an online environment removes geographic limitations and eliminates the need for travel for program participants and hosts. This article highlights two examples of online focus group approaches used with internal Extension audiences. Examples include use of online focus groups for program evaluation and needs assessment.EVALUATION EXAMPLEIn fall 2019, I conducted online focus groups to gather information about the experiences of Virginia Cooperative Extension (VCE) agents who hosted interns the previous summer (VT IRB – 17-1149). I sent an email to agents with details of the study and a link to a survey that they could use to indicate their availability. Three dates were identified to provide a minimum of six participants for each focus group. Participants received the script, consent form, expectations, and purpose for the focus group by email at least one week prior to the session. Online focus groups were conducted using Zoom and each session was recorded. A graduate student and I served as the research team. We took turns facilitating and serving as notetaker for the focus groups.During the focus groups, the questions were asked, and participants answered questions in the same order under the direction of the facilitator. Participants were encouraged to interject with comments as needed. This did occur regularly, as one participant’s answer would often spark the memory of a previous respondent. As with traditional, in-person focus groups, the participants regularly referred to previous responses as they spoke.The graduate student and I transcribed, cleaned, and coded the focus group sessions independently. We then discussed our codes to develop final coding prior to data analysis.Abstract. The Cooperative Extension organization could be categorized as a participatory organization because of our focus on local needs in driving our work. The pandemic moved a lot of our work to online platforms. This arti-cle reflects on use of online tools for program evaluation and needs assessment. Online focus groups were success-ful in gathering high quality data from a diverse and geographically dispersed population for program evaluation and as part of a needs assessment process. Coupling the focus groups with a modified World Café approach allowed us to go beyond identification of needs to prioritization and development of potential solutions.Journal of Extension  Volume 60, Issue 3 (2022)  VinesA NEEDS ASSESSMENT APPROACHIn spring 2021, we used online focus groups again to identify needs of mid-career Extension agents and specialists (VT IRB – not human subjects research). We used purposive sampling to gather a breadth of experiences. The sampling frame of Extension faculty with 3–7 years of service was provided by human resources. This was then divided to provide two focus groups each for agents who had and had not yet been promoted, and one focus group each for specialists who had and had not yet received tenure. Potential participants were randomly selected from each group to obtain a goal of six to eight participants in each focus group. These individuals received an email explaining the purpose and process for the study and a link to a Doodle poll to indicate their availability to participate in a focus group. After the deadline, the dates for the focus groups were selected. Individuals were invited to participate in the sessions. As individuals declined, other individuals in their category were invited to participate. The goal was to have six to eight participants in each focus group.This study was supported in part by a professional development grant from Epsilon Sigma Phi (ESP). Leaders of Virginia ESP and two members of the VCE leadership group joined me as members of the research team. I facilitated all of the focus groups except one, and a member of the research team served as the notetaker. Two members of the team served as a facilitator and notetaker for the group whose session I could not fully attend.Participants received the questions and information about the study one to three weeks prior to their focus group session. They received a reminder email within 24 hours of the session. In the sessions, the order of respondents varied for each question (i.e., the second respondent for question 1 became the first respondent in question 2 while the first respondent became last, etc.). Sessions were recorded and transcribed. The notetaker for each session and I worked to clean up the transcripts. They were then shared with the focus group participants for member checking. I developed codes that were guided by the focus group questions to analyze findings. In addition, the research team met to discuss observations and findings. We used guiding questions to facilitate this discussion (Table 1). The follow-up research team meetings were recorded and considered as data for analysis.1. What did you hear that you weren’t expecting to hear?2. As you reflect on the content of your group as a whole, what action items would you recommend for us or others in the organization?3. Are there areas where you feel we need to follow up to learn more?Table 1. Guiding Questions for Discussion by the Research TeamFinally, findings documents were developed that provided a general overview and identified themes from the focus groups while also providing quotes, sharing the participants’ view in their own words. Findings documents were provided for the following topics: competencies, sources of stress, needs assessment, onboarding and mentoring, professional associations, and organizational support. These documents were shared with focus group participants prior to a final feedback session. A modified World Café Approach (http://theworldcafe.com/key-concepts-resources/world-cafe-method/) was used in this online session. In World Café, an initial round of participants brainstorms responses to a set of questions. Subsequent rounds of participants add to or modify responses. These have generally been done in face-to-face environments, but this session was conducted online using breakout rooms to provide the additional rounds of participants. Participants identified topic areas based on the questions included in the focus groups and were asked to review the findings document to refresh their memories and allow them time to develop ideas for solving issues. Breakout rooms were used, allowing participants to answer three questions (Table 2) around three topics. Breakout sessions one and two lasted 20 minutes, and the final breakout session lasted 15 minutes. Each breakout room had a member of the research team serving as a facilitator and a notetaker. Notes were taken using a Google Doc for each topic and participants were granted access so they could add or revise information. Members of the first breakout developed the initial content for the topic. Participants in breakout sessions two and three reviewed previous content and then added their thoughts. The Google Docs were left open for a week following the session to allow continued revision by participants or allow those who were not able to attend an opportunity to share their thoughts.Journal of Extension  Volume 60, Issue 3 (2022)  Online Participatory Evaluation and Needs Assessment ToolsCONCLUSIONThere are benefits to gathering data from Extension personnel across the organization in a virtual environment. First, we were able to assemble agents and specialists who were geographically dispersed without travel expense. Second, the quality of the responses from the online focus groups were as superior as those experienced in previous face-to-face focus group sessions. Finally, the benefit of not having to travel also reduced the time commitment and expense for the participants and the research team. When the COVID-19 pandemic emerged in 2020, online focus groups were valuable in allowing us to continue the needs assessment project.One negative aspect of the virtual sessions was the kind of interruptions that occurred, such as when someone scheduled another meeting adjacent to the session and had to leave early, or when someone who was keeping the Extension office open with reduced personnel had to leave because of clientele knocking on the door. In another case, a participant was driving during the focus group, so they were unable to remain fully present. However, these individuals may not have been able to participate at all if the sessions had been held face-to-face.Working with a research team of experienced Extension educators helped reduce the amount of time required for cleaning up transcripts. It was also helpful getting their responses as part of the analysis process since they provided unique insights based on their experience. Sharing findings and having focus group participants assist in priority setting and developing potential solutions to issues helps to verify areas in which we need to focus and see solutions from different contexts. I believe this approach was not only useful within the organization but can also be used in various community settings. The modified World Café approach was effective, although a common complaint was that people did not have enough time in the breakout rooms. Recording discussions in Google Docs helped encourage ongoing review and revision by participants.REFERENCESVanderford, E. F., Gordon, J. S., Londo, A. J., & Munn, I. A. (2014). Using focus groups to assess educational programming needs in forestry. Journal of Extension, 52(3), 1–9. https://tigerprints.clemson.edu/joe/vol52/iss3/32/Vines, K. A., Cletzer, D. A., Westfall-Rudd, D., Lambur, M., Hunnings, J., & Vines, N. T. (2018). Identifying needs and implementing organizational change to improve retention of early-career agents. Journal of Extension, 56(1). https://tigerprints.clemson.edu/joe/vol56/iss1/23/White, D. J., Arnold, M. E., & Lesmeister, M. K. (2008). Using focus groups to evaluate youth development program direction. Journal of Extension, 46(6). https://archives.joe.org/joe/2008december/rb3.php1. Is there anything you feel is missing or should be added to the findings document? (This data was collected a year ago, so some things may have changed, and participants may have thought of other items since then.)2. As a group, list the items you see as top priority that need to be addressed in order to improve (VCE).3. Work through the items identified in #2 to make recommendations in terms of further dis-cussion or study, policy, practice, professional development topics, etc. to address each of these in order to strengthen VCE.Table 2. Guiding Questions for Discussion During the Feedback Session with Focus Group Participants

Participatory Evaluation and Needs Assessment Tools for Cooperative Extension in a Virtual Environment

https://tigerprints.clemson.edu/cgi/viewcontent.cgi?article=3896&context=joe

Participatory Evaluation and Needs Assessment Tools for Cooperative Extension in a Virtual Environment

Abstract

Similar works

Full text

Available Versions

Clemson University: TigerPrints