The objective of this symposium is to establish some conceptual and operational links between participatory approaches to evaluation research (PAER) principles and organizational change processes. In order to discover some of these critical connections we propose to open a discussion to explore the paradigmatic dimensions of PAER.

Essentially, PAER start out from a recognition that evaluation research develops within a pluralistic society and allows evaluation projects to be built upon the ideas, values and aspirations of those taking part at all levels and throughout the whole evaluation process (Diez et al., 2005). In this sense, Gregory (2000) points out how participation allows people to be aware of the rationality behind the evaluation process, and in doing so, increases their involvement in the implementation of an evaluation research project, both in an effective and efficient way. In other words, PAER is a Capacity Building to Evaluation (CBE) strategy.

CBE is a context-dependent, organized action system of guided processes and practices for bringing about and sustaining a state of affairs in which quality evaluation and its appropriate uses are ordinary and ongoing practices within and/or between one or more organizations. Evaluation here is understood as a learning process around a social project evaluated from the perspective of all the stakeholders. Therefore, evaluation becomes an exercise stimulating the appearance of an organizational learning (OL) process. The very essence of OL is to adapt to and to take part of organizational change processes

Kagan, Carolyn

Springett, Jane

Suarez, Jose C.

E-space: Manchester Metropolitan University's Research Repository

Conference of the UK Evaluation Society: Evaluation in Society: Critical ConnectionsJoint International Conference4-6 October 2006LondonCritical connections between participatory approaches to evaluation research and organizational change processesSuárez, J.C.1, Springett, J.2 & Kagan, C.31. José Carlos Suárez Herrera (Ph.D. Candidate)Department of Health AdministrationPhone: 	+1 (514) 596-2554Fax: 		+1 (514) 343-2207E-mail:	jc.suarez.herrera@umontreal.ca (​mailto:jc.suarez.herrera@umontreal.ca​) University of MontrealQuébec (Canada)2. Jane Springett, Ph.D. (Professor)Institute for HealthPhone: 	+44(0)151 231 4115Fax: 		+44(0)151 258 4218E-mail:	R.J.Springett@livjm.ac.uk (​mailto:R.J.Springett@livjm.ac.uk​) Liverpool John Moores University 70 Great Crosshall Street,Liverpool L3 2ABEngland3. Carolyn Kagan, Ph.D. (Professor)Research Institute for Health and Social Change Phone: 	+44 (0) 161 247 2563Fx: 		+44 (0)161 247 6842E-mail: 	c.kagan@mmu.ac.uk (​mailto:c.kagan@mmu.ac.uk​) The Manchester Metropolitan University Elizabeth Gaskell Campus Hathersage Road Manchester, M13 0JA England 	Plan of Symposium-	Introduction: Some critical connections-	“State of the arts” of participatory approaches to evaluation research -	Doing evaluation to build capacities and building capacities to do evaluation-	The double face of organizational learning -	Organized systems of social action and organizational change-	Workshop: o	Paradigmatic dimensions of participatory approaches to evaluation research o	Walking the talk… participatory approaches to evaluation research and organizational change! Introduction: Some critical connectionsThe objective of this symposium is to establish some conceptual and operational links between participatory approaches to evaluation research (PAER) principles and organizational change processes. In order to discover some of these critical connections we propose to open a discussion to explore the paradigmatic dimensions of PAER.Essentially, PAER start out from a recognition that evaluation research develops within a pluralistic society and allows evaluation projects to be built upon the ideas, values and aspirations of those taking part at all levels and throughout the whole evaluation process (Diez et al., 2005). In this sense, Gregory (2000) points out how participation allows people to be aware of the rationality behind the evaluation process, and in doing so, increases their involvement in the implementation of an evaluation research project, both in an effective and efficient way. In other words, PAER is a Capacity Building to Evaluation (CBE) strategy.CBE is a context-dependent, organized action system of guided processes and practices for bringing about and sustaining a state of affairs in which quality evaluation and its appropriate uses are ordinary and ongoing practices within and/or between one or more organizations. Evaluation here is understood as a learning process around a social project evaluated from the perspective of all the stakeholders. Therefore, evaluation becomes an exercise stimulating the appearance of an organizational learning (OL) process. The very essence of OL is to adapt to and to take part of organizational change processes.PAER ---» CBE ---» OL ---» Organizational Change “State of the arts” of participatory approaches to evaluation researchThroughout the years, the evaluation field has focused on the use of evaluation findings as one indicator that an evaluation has been successful (Preskill, Zuckerman & Matthews, 2003). And, during the last several years increasing interest in involving stakeholders in various phases of an evaluation has led to a greater number of evaluations that employ participatory approaches (collaborative, empowerment and/or learning-based approaches) (Cousins & Whitmore, 1998; Fetterman, 2000; Patton, 1997; Preskill & Caracelli, 1997; Preskill & Torres, 1999; Torres & Preskill, 2001; Weiss, 1998). We believe that such approaches will: (a) contribute to participants’ sense of ownership of, and commitment to the evaluation, (b) provide participants with opportunities for learning about effective evaluation practice, (c) result in more useful recommendations, and (d) enhance the use of evaluation findings.What do we mean when we say participatory approaches to evaluation research (PAER)? Clearly, participatory activity in evaluation research has a wide range of meanings. We think that looking first briefly at participatory practices in research realms helps to lay a foundation for looking at PAER.In research realm, participation can be a continuum going from an activity as limited as simply answering a questionnaire or being part of an interview, to a course of action as extensive as full, active involvement in all phases of the research process. Whyte (1991) has defined participatory action research (PAR) as a form of applied research, where the researcher becomes a facilitator in helping those being studied to also become actively engaged in the quest for information and ideas to guide future efforts. Widely used in the developing world, the PAR concept was originally conceived as a means of helping small farmers assess and solve problems, as well as a means for westerners to learn more about locally adaptive agricultural practices. It is in the extension of PAR to evaluation that PAER found its earliest expression (Garaway, 1995).Cousins & Earl (1992: 399) define participatory evaluation as, “applied social research that involves a partnership between trained evaluation personnel and practice-based decision makers, organization members with program responsibility or people with a vital interest in the program.” They differentiate participatory evaluation from PAR and other forms of action research by maintaining that PAR is limited to a normative and ideological research orientation rather than an evaluative one. ___________________________________________________________________Springett (1995, p. 83) expose that “a participatory approach attempts to involve those who are at stake in the outcome in order to take action and effect change… Its philosophical and epistemological base lies in a hermeneutical tradition of knowledge creation. Its methodological and ideological roots lie in participatory action research.”More recently, Diez et al (2005: 4) argue that, essentially, “participatory evaluation starts out from a recognition that evaluation develops within a pluralistic society and allows evaluation to be built upon the ideas, values and aspirations of those taking part at all levels and throughout the whole evaluation process.” In PAER, the evaluation design is not imposed from outside, but gradually takes shape through the collaboration of all the stakeholders and their active participation in the analytical evaluation process. This focus considerably increases the probability that the process and the results achieved by the evaluation will be used in an effective way to improve the policy, since it allows the actors in the programme to make the actual evaluation process and its results their own, transforming the evaluation into a collective learning process which, in a certain sense, belongs to them. In this sense, Gregory (2000) points out how participation in decision making allow people to be aware of the rationality behind the decision making, and in doing so, makes higher their implication in the implementation of the policy, both in an effective and efficient way. In other words, PAER are a capacity-building to evaluation (CBE) strategy.Doing evaluation to build capacities and building capacities to do evaluationWe identified two principal trends of evaluation use associated with capacity-building strategies. Any given evaluation project, we suggest, would be characterized by a primary emphasis on one or a combination of these two trends. First is the pragmatic trend, and evaluation is conceived as a means. We are talking about evaluation for capacity-building. Here evaluation is purported to lead to instrumental consequences and to increase the usefulness of the knowledge that is created. In this sense, evaluation takes on a problem-solving orientation. Members of the community of practice engage with evaluators to produce knowledge that bears upon identifiable practical problems. To the extent that the evaluation project is grounded in the context for use and thereby rendered meaningful to those responsible for problem solving, the knowledge produced will be of greater use. A second trend is political, and is ideologically rooted in normative conceptions of social justice and the democratic process. Evaluation is conceived as an end. We are talking about capacity-building to evaluation (CBE). The primary interest of evaluation that subscribes to such political aims is to promote fairness through the involvement of individuals associated with all groups with a stake in the research (e.g., applied study, evaluation). Through direct involvement and participation in the research process, social actors that do not normally have a voice in policy or programme decision making are now provided with such opportunities. It is in that sense that we argue that PAER are a CBE strategy. CBE is a context-dependent, organized action system of guided processes and practices for bringing about and sustaining a state of affairs in which quality evaluation and its appropriate uses are ordinary and ongoing practices within and/or between one or more organizations. Evaluation here is therefore understood as an organizational learning process around a social program evaluated from the perspective of all the stakeholders.The double face of organizational learningOrganizational learning (OL) is a phenomenon that takes place in various forms within an organization. According to Argyris (1992) OL is any process of change in organizational structures, codes or practices that is triggered or reinforced by new experiences, new interactions or new information. Furthermore, Argyris and Schön (1996) describe two types of OL: single-loop and double loop learning. Each entails a certain kind of organizational behaviour and each engenders a specific outcome. In simple terms, single-loop learning deals with strategic changes and more or less maintains the status quo of the organization. Double-loop learning deals with declared and underlying goals and engenders basic changes in organizational outlook and behaviour.Single-loop learning seems to be present when goals, values, frameworks and, to a significant extent, strategies are taken for granted. The emphasis is on “techniques and making techniques more efficient” (Usher & Bryant: 1989: 87). Any reflection is directed toward making the strategy more effective. Double-loop learning, in contrast, “involves questioning the role of the framing and learning systems which underlie actual goals and strategies”. In many respects the distinction at work here is the one used by Aristotle, when exploring technical and practical thought. The former involves following routines and some sort of preset plan – and is both less risky for the individual and the organization, and affords greater control. The latter is more creative and reflexive, and involves consideration notions of the good. Reflection here is more fundamental: “the basic assumptions behind ideas or policies are confronted… hypotheses are publicly tested… processes are disconfirmable not self-seeking” (Argyris 1982: 103-4).We are confident that by taking a learning approach within organizations, both, single-loop and double-loop learning, can significantly support efforts to learn, grow, and take appropriate action. Through OL we re-create ourselves (Senge, 1990) by a transformational process. Through OL we become able to do something we never were able to do. Through OL we re-perceive the world and our relationship to it. We extend our capacity to create, to be part of the generative process of life. There is within each of us a deep hunger for this type of learning. This kind of learning is essential to change creation. The very essence of OL is to adapt to and to take part of organizational change processes.Organized systems of social action and organizational changeSo OL can play a key role in change processes. Unfamiliar major changes almost always generate fear and anxiety in people, often requiring them to radically shift their thinking, feelings, beliefs and behaviours. Consequently, the more individuals understand and accept about a change, the more comfortable and committed they tend to become to it. Such understanding gives people a sense of control over the change or a greater ability to anticipate relative to the change, contributing to their sense of comfort and security and lessening their resistance to the change (Conner, 1993). These ideas and OL principles, as conceived earlier, open a complex social construction process for an organized system of social action (OSSA). Although the assimilation of this complex system does not guarantee that resistance to change will be eliminated and that a desired change will be accomplished, its proper (re)production does significantly should improve the chances for success with a change effort. Notice that this OSSA implies no surprises, since it requires that OL must precede any change. For major institutional initiatives, often there must be significant local learning preceding global change, such as several planned, multidimensional, many-level iterations of OL over a substantial time period. The likelihood of people reacting favourably to change and assisting with it will be enhanced greatly if time is taken to provide a basis of learning and understanding about the change. Simply put, OL requires change and significant change in OSSA requires OL. PAER ---» CBE ---» OL ---» Organizational ChangeWorkshop: Paradigmatic dimensions of participatory approaches to evaluation researchPAER ---» CBE ---» OL ---» Organizational ChangeThe notion of paradigm or worldview as an overarching framework which organizes our whole approach to being in the world has become commonplace since Kuhn published The Structure of Scientific Revolutions in 1962. In contrast to the view that a paradigm is, beyond definition and the grasp of the human mind, we believe that the mind, by its very nature, is more extensive than any worldview on which it takes its current cognitive stance. Hence it is possible and essential to expand our awareness to articulate any fundamental way in which we frame PAER, through consideration of their ontological, epistemological, methodological, and teleological dimensions and the interactions between them. I. Ontological dimension of PAERThe word ‘ontology’ seems to generate a lot of controversy in discussions about scientific process. The term is borrowed from philosophy, where Ontology is a systematic account of Existence. In the context of PAER, we use the term ontology to mean an explicit specification of a conceptualization. That is, ontological dimension of PAER is a description of the concepts and relationships between these concepts that can exist for an agent or a community of agents regarding these approaches. This conception is consistent with the use of ontology as set-of-concepts, but more general. This set-of-concepts, and the relationships among them, are reflected in the representational vocabulary with which a community of practice represents knowledge. Thus, ontological dimension of PAER implies a description of a set of their representational and unique terms by an explicit specification of their distinctive components (participation, evaluation and research) and of the interactions between them.II. Epistemological dimension of PAER‘Epistemology’ is the branch of philosophy that deals with questions concerning the nature, scope, and sources of knowledge. A central question of the area is: Under what conditions does a subject or a group of subjects know something? Epistemology seeks to understand the origin, processes and limitations of observation including such operations as drawing distinctions, establishing relations, creating constructs and all consequences for knowledge resulting from communication between a subject and any given object and within a community of subject who may observe each other. Regarding PAER, we consider the subject and the object as parts of the same complex system and as an emergent property of the interaction process. We ask not "what is", or "what can we know" but "how do we come to know". Thus, we would like to explore how (by what processes) a group of subjects might be involved in determining and reviewing the evaluation’s purpose, key questions, and data collection instruments in order to explore a given object by using a project. We call this relationship an epistemological triad.III. Methodological dimension of PAER‘Methodology’ refers to more than a simple set of methods; rather it refers to the rationale and the philosophical assumptions that underlie a particular study. A methodological perspective needs to be one which draws on that epistemological triad in such a way that critical subjectivity is enhanced by critical intersubjectivity. Hence a collaborative form of inquiry, in which all involved engage together in democratic dialogue as co-researchers and as co-subjects (Reason & Heron, 1995; Heron, 1996). In our articulation of this with PAER, people collaborate to define the questions they wish to explore and the combination of methods to use for that exploration. Thus, we would like to explore this methodological synergism, in order to understand how co-researchers engage their methods together in cycling several times through the different forms of knowing and how they enrich their methodological congruence, that is, the way they choose their methods and deepen the complementary way they know the world.IV. Teleological dimension of PAER‘Teleology’ refers to the understanding of the fact or character attributed to nature or natural processes of being directed toward an end or shaped by a purpose. In this sense, teleology can be considered as the understanding of process use oriented to a specific end. Regarding with PAER, ‘process use’ is the learning that occurs from one’s participation in an evaluation process (Preskill et al., 2003). Patton (1997: 90) defines process use as: “Individual changes in thinking and behaviour, and program or organizational changes in procedures and culture, that occur among those involved in evaluation as a result of the learning that occurs during the evaluation process.”Process use reflects constructivist learning theory in that it focuses on how groups of people make meaning as they conduct an evaluation. Engaging in such learning processes requires collaborative and participatory forms of evaluation. Thus, we would like to explore how we could provide participants with opportunities for learning about effective evaluation practice and how we could contribute to develop a participant’s sense of ownership of, and commitment to the evaluation project in order to build together more useful recommendations and enhance the use of evaluation findings. PAGE  9

Critical connections between participatory approaches to evaluation research

https://e-space.mmu.ac.uk/24954/1/jose%20carlos%20ees.doc

Critical connections between participatory approaches to evaluation research

Abstract

Similar works

Full text

Available Versions

E-space: Manchester Metropolitan University's Research Repository