Rubrics: A Method for Surfacing Values and Improving the Credibility of Evaluation

Background: The challenges of valuing in evaluation have been the subject of much debate; on what basis do we make judgments about performance, quality, and effectiveness? And according to whom? (Julnes, 2012b). There are many ways identified in the literature for carrying out assisted valuation (Julnes, 2012c). One way of assisting the valuation process is the use of evaluative rubrics. This practice-based article unpacks the learnings of a group of evaluators who have used evaluative rubrics to grapple with this challenge. Compared to their previous practice, evaluative rubrics have allowed them to surface and deal with values in a more transparent way. In their experience when evaluators and evaluation stakeholders get clearer about values, evaluative judgments become more credible and warrantable. Purpose: Share practical lessons learned from working with rubrics. Setting: Aotearoa (New Zealand). Intervention: Not applicable. Research Design: Not applicable. Data Collection and Analysis: Not applicable. Findings: They have found that while evaluative rubrics look beguilingly simple they are hard to do well. However, when done well, evaluative rubrics can substantially increase the use and credibility of evaluation.Keywords: Rubrics; values; valuation; stakeholder; validity; credibility; utilit

Rubrics: A Method for Surfacing Values and Improving the Credibility of Evaluation

Background: The challenges of valuing in evaluation have been the subject of much debate; on what basis do we make judgments about performance, quality, and effectiveness? And according to whom? (Julnes, 2012b). There are many ways identified in the literature for carrying out assisted valuation (Julnes, 2012c). One way of assisting the valuation process is the use of evaluative rubrics. This practice-based article unpacks the learnings of a group of evaluators who have used evaluative rubrics to grapple with this challenge. Compared to their previous practice, evaluative rubrics have allowed them to surface and deal with values in a more transparent way. In their experience when evaluators and evaluation stakeholders get clearer about values, evaluative judgments become more credible and warrantable. Purpose: Share practical lessons learned from working with rubrics. Setting: Aotearoa (New Zealand). Intervention: Not applicable. Research Design: Not applicable. Data Collection and Analysis: Not applicable. Findings: They have found that while evaluative rubrics look beguilingly simple they are hard to do well. However, when done well, evaluative rubrics can substantially increase the use and credibility of evaluation.Keywords: Rubrics; values; valuation; stakeholder; validity; credibility; utilit

What Does it Take to do Evaluation in Communities and Cultural Contexts Other Than Our Own?

Background: This paper builds on the growing body of evaluation literature around the importance of culture and cultural context in evaluation (e.g. Greene, 2005; Hood, Hopson and Frierson 2005; Hopson, 2009; Kirkhart, 1995 and 2005; La France, 2001). Purpose: The place of language, culture, cultural context, and leadership roles in evaluation is explored through consideration of the question, “What does it take to do evaluation in communities and cultural contexts other than our own?”  Setting: Not applicable. Intervention: Not applicable. Research Design:  Not applicable. Findings: Attention to the location of power and privilege in evaluation, and to community engagement and ‘sense-making’ processes are the conversational starting points to begin to explore what it takes to do evaluation in communities, where the language, culture, and cultural context are different from one’s own.  Keywords: culture, context, diversity, power, indigenous communities, communities of color, values, ethics, cultural competence, validit

Strengthening prevention in communities through systems change: lessons from the evaluation of Healthy Families NZ

Abstract This article presents the findings from the first 3 years of the evaluation of Healthy Families NZ, a systems-change intervention to prevent chronic diseases in 10 communities. The initiative, which builds on existing prevention activities, aims to strengthen the health prevention system through evidence-driven action to enable people to make good food choices, be physically active, smoke-free and free from alcohol-related harm. Key investment areas are a dedicated systems thinking and acting health promotion workforce, and activating leaders who can influence transformational change. The evaluation to date has found the initiative is being implemented with integrity. Evidence indicates a shift towards greater action on prevention, and the prevention system being strengthened. Māori ownership has been enabled, and prioritizing equity has led teams to utilize methods that amplify diverse local perspectives. There is progress on developing a flexible workforce through adaptive learning, flexible resources, professional development and a responsive National team. There is also progress in activating local leadership and empowering local teams. The initiative design has explicitly taken into account the context of complexity within which it is being implemented. It has evolved to focus on action that can accelerate sharing information and practices within communities, and between policy and decision-makers. Healthy Families NZ and its evaluation have been refunded to 2022. This provides an important opportunity to gather further insight into effective ways to strengthen the community agency and trust needed to promote and deliver evidence-based action on prevention