Framing Anticolonialism in Evaluation: Bridging Decolonizing Methodologies and Culturally Responsive Evaluation

Background: Evaluation is grounded in academically imperialistic research methodologies, paradigms, and epistemologies, which have lasting effects on communities of study. Confronting Westernized evaluation's monoculturalism, scholars call for decolonization, to produce locally-determined, strengths-based, culturally-situated, and valid understandings. This endeavor is complicated, requiring a paradigm shift for Westernized evaluators.  Purpose: In this paper, we describe the anticolonial culturally responsive framework occurring in the intersections between culturally responsive (CRE) and decolonizing (DF) approaches. Anticolonialism honors decolonizing without displacing the authority of Indigeneity, simultaneously foregrounding the interweaving of evaluator, evaluand, and disciplinary culture. Interrogating academic imperialism through anticolonialism, confronts the social processes and cultural ideologies that produce and reproduce social inequality in evaluations. Setting: Not applicable. Data Collection and Analysis: We draw on scholars and scholarship who have advanced culturally responsive, decolonizing, and anticolonial evaluation and methodological fields.   Findings: The anticolonial culturally responsive framework is an invitation for evaluators trained in imperialistic Westernized approaches or who embody the colonial world through our race, language, knowledge, and culture. Our goal is not to displace the primacy and urgency of vitalizing Indigenous and decolonizing frameworks. Instead, we offer a tentative approach committed to pluriversality, justice, self-determination, and the possibility of collaboration between knowledge systems and knowers

Equity “On the Sideline”: A Mixed Methods Study of New England Evaluation Practice in 2020

Background: Centering equity in evaluations is increasingly recognized as an important professional responsibility of evaluators. While some theoretical and practical guidance exists, the evaluation field has limited empirical research on equity within evaluation practice. Purpose: This paper explores whether and how evaluators address inequities and advance equity throughout evaluation phases drawing on select findings from a larger study. Setting: The study focuses on American Evaluation Association-affiliated evaluators in the New England region of the United States who work in a variety of areas (e.g., health, education).   Intervention: Not applicable Research Design: The study uses a complementarity, sequential mixed methods design comprised of a researcher-developed online questionnaire administered to a census and snowball sample of practicing evaluators (n=82) and individual, semi-structured interviews with a subset of this sample selected to maximize variation (n=21). Quantitative data were analyzed using descriptive statistics (i.e., means and standard deviations, frequencies). Qualitative data were analyzed using a collaborative process of deductive and inductive coding followed by thematic analysis. Findings: Eight overarching findings suggest that despite evaluators’ attempts to center equity, it remains largely “on the sideline.” This is due to evaluators’ need to work against some conventional professional and methodological norms, within contractual and contextual constraints, and with limited professional preparation. &nbsp

Serving Their Needs: A Qualitative Examination of Nutrition Policy Implementation in the Early Care and Education Setting

Background: Childhood obesity is a growing problem in the United States and results in increased risk for chronic diseases such as diabetes, heart disease, and hypertension. Thirteen percent of youth in Georgia are obese. Identifying strategies to assist children in establishing healthy habits is essential to reduce the risk of childhood obesity. The Early Care and Education (ECE) setting is ideal for the implementation of obesity prevention practices. However, there are barriers present for implementing nutrition policies in this setting. This report explores the implementation of food and beverage best practices in the ECE setting and highlights barriers to and facilitators for adopting these policies. Methods: We conducted 24 interviews and 6 focus groups with ECE program directors and teachers in 6 regions in Georgia. The statewide sample included directors from child care learning centers, family child care homes, and license-exempt programs. A trained qualitative researcher facilitated focus groups and interviews. Data were audio recorded and transcribed verbatim. Qualitative data analysis software, NVivo 10, was used to code data and identify emergent themes. Results: Several key themes related to barriers to food and beverage policy implementation emerged including the need for: 1) enhanced parent communication, 2) resources to limit juice consumption, and 3) financial support to decrease food costs. Facilitators of nutrition policy implementation included: 1) ease of access to water, 2) children’s preferences for fruits and vegetables, and 3) availability of existing nutrition resources. Findings will inform the development of resources to support nutrition policy implementation as well as policy training for ECE teachers in Georgia. Conclusions: Study themes may provide insight about how to improve current resources and develop new solutions to improve adoption and implementation of nutrition policies in the ECE setting in the future

Equity “On the Sideline”

Background: Centering equity in evaluations is increasingly recognized as an important professional responsibility of evaluators. While some theoretical and practical guidance exists, the evaluation field has limited empirical research on equity within evaluation practice. Purpose: This paper explores whether and how evaluators address inequities and advance equity throughout evaluation phases drawing on select findings from a larger study. Setting: The study focuses on American Evaluation Association-affiliated evaluators in the New England region of the United States who work in a variety of areas (e.g., health, education).   Intervention: Not applicable Research Design: The study uses a complementarity, sequential mixed methods design comprised of a researcher-developed online questionnaire administered to a census and snowball sample of practicing evaluators (n=82) and individual, semi-structured interviews with a subset of this sample selected to maximize variation (n=21). Quantitative data were analyzed using descriptive statistics (i.e., means and standard deviations, frequencies). Qualitative data were analyzed using a collaborative process of deductive and inductive coding followed by thematic analysis. Findings: Eight overarching findings suggest that despite evaluators’ attempts to center equity, it remains largely “on the sideline.” This is due to evaluators’ need to work against some conventional professional and methodological norms, within contractual and contextual constraints, and with limited professional preparation.

Educational Accountability: Investigating the Collective Capacity of a Middle School

181 p.Thesis (Ph.D.)--University of Illinois at Urbana-Champaign, 2008.The main findings of this instrumental mixed methods case study were that, in this context, external accountability led to increased attention to test scores, curriculum, and instruction. External accountability influenced the school's system of internal accountability by fostering better alignment between instruction and state curriculum standards. External accountability also influenced the school's collective capacity, particularly with respect to leadership and teachers' ability to provide more academic support to students and a more nurturing environment. At the same time, teachers had varied student expectations that deterred them from achieving even higher levels of collective instructional capacity.U of I OnlyRestricted to the U of I community idenfinitely during batch ingest of legacy ETD

Values engagement in evaluation: Ideas, illustrations, and implications. American Journal of Evaluation. Advance online publication. doi: 10.1177/1098214011422592

Abstract Values-engagement in evaluation involves both describing stakeholder values and prescribing certain values. Describing stakeholder values is common practice in responsive evaluation traditions. Prescribing or advocating particular values is only explicitly part of democratic, culturally responsive, critical, and other openly ideological traditions in evaluation, but we argue that it is implicit in all evaluation approaches and practices. In this article, we discuss various conceptualizations of values-engagement in evaluation. We further present a specific form of values-engaged evaluation that is committed to descriptive and prescriptive valuing, with an emphasis on its prescriptive advancement of the values of inclusion and equity. Examples from field experience illustrate these two countenances and underscore the multiple challenges invoked by intentional engagement with the values dimensions of evaluation. The examples come from evaluations of science, technology, engineering, and mathematics (STEM) educational programs