research

Meta-evaluations in government and government institutions: A case study example from the Australian Centre for International Agricultural Research

Abstract

In this paper we draw on impact assessment work of the Australian Centre for International Agricultural Research (ACIAR) to present an example of meta-evaluation – an evaluation of evaluations – in an agricultural research, development and extension setting. We explore quality issues relating to evaluation studies in the context of government institutions. Program evaluation standards (PES) are divided into categories of utility, feasibility, propriety and accuracy to provide a framework for the meta-evaluation. The PES are presented as a universal measure of evaluation study quality. The intent of using them here is to judge the adequacy of PES as a universal quality measure or meta-evaluation base and to extract useful insights from ACIAR program evaluation activities when developing a meta-evaluation model for the Limpopo Department of Agriculture (LDA). Our meta-evaluation is undertaken of 63 impact assessment reports. First, the literature guiding the conduct of a meta-evaluation is reviewed. Second, an assessment (the meta-evaluation) of the evaluation studies is carried out for 19 sampled reports from a population of relevant reports fitting the dimension of the analysis, and results are presented and discussed. Also, lessons learned are presented, using the framework provided by the meta-evaluation criteria. Third, taking into account the lessons learned, implications are drawn for a proposed systematic meta-evaluation of the LDA. Finally, we conclude that all the PES cannot be equally emphasized in a meta-evaluation model. At ACIAR, 70% of the standards were at least partially addressed. Therefore, we succeeded in using the PES in judging the ACIAR evaluation quality. As such, they can be an important base when developing an evaluation model but should be applied in a contextualized manner.Meta-evaluation, Evaluation Quality, Program Evaluation Standards, Evaluation Model, Australian Centre for International Agricultural Research, Limpopo Department of Agriculture (South Africa), International Development,

Similar works

Full text

thumbnail-image

Research Papers in Economics

Provided a free PDF
Last time updated on 7/6/2012View original full text link

This paper was published in Research Papers in Economics.

Having an issue?

Is data on this page outdated, violates copyrights or anything else? Report the problem now and we will take corresponding actions after reviewing your request.