Hendrickx, Saskia C.J.

Swaans, S.

Taye, H.

CGSpace

ILRI Research Brief — September 2014 1Using outcome mapping as a monitoring and management tool in a small ruminant value chain projectHailemichael Taye, Kees Swaans and Saskia HendrickxIO.LRI RESEARCH BRIEF  NDate  yearSummaryThe imGoats project (‘small ruminant value chains to in-crease income and food security in India and Mozambique’) was designed to increase incomes and food security in a sustainable manner by enhancing pro-poor small ruminant value chains in the two countries. This brief shares experiences of the project in using out-come mapping (OM) to improve planning, monitoring and evaluation (M&E) of a goat value chain development project in the two countries.Results show that OM was very useful to monitor behav-ioural changes among value chain actors. It also contributed to effective management of the project. The approach suited both a relatively small project area in Mozambique (>500 households), with a strong emphasis on qualitative data, as well as a larger project area in India (>2500 house-holds) where it helped improve data collection and analysis, and stimulate feedback mechanisms. Data collection and analysis was resource intensive, and substantial adaptations were required based on organiza-tional culture and capacity as well as project scale. Outcome mappingThe OM methodology was developed by the International Development Research Centre (IDRC) in 2001. It is an ap-proach to M&E that defines results as changes in behaviour of an intervention’s direct partners. OM starts from the view that development results from complex interactions between different actors, forces and trends.  As it is difficult to attribute development impact to interventions directly, OM focuses on contributions of interventions towards developmental results. In doing so, it focuses on people. OM shifts the focus of development from bringing “changes in states” to ‘‘changes in behaviour, relationships, activities, or actions among those a project interacts directly with (boundary partners).OM consists of twelve steps organized in three stages, corresponding to designing, monitoring and evaluating an intervention (see Figure 1). Often though, not all stages and steps are used in practice. Various elements and tools can be used separately or in conjunction with other processes.     ILRI RESEARCH BRIEF  18                    September 2014ILRI Research Brief — September 20142OM in the imGoats projectThe imGoats project was implemented from January 2011 to June 2013, with the aim to transform goat production and marketing into profitable enterprises, largely controlled by and benefiting disadvantaged and vulnerable groups while preserving the natural resource base. The overall project was managed by the International Live-stock Research Institute (ILRI) while the implementation was done by two non-governmental organizations (NGOs): Bharatiya Agro Industry Foundation (BAIF) in India and CARE in Mozambique. OM was used as a participatory M&E framework to assess changes in behaviours, practices and capacities of value chain actors.Project sites :The specific project area in India was       Rajasthan State with 2600 target households in Jhadol and Sarada blocks of Udaipur district. These households consti-tuted about 240 goat keepers groups, with 10-15 families per group. In Mozambique, the project targeted 500 house-holds in Inhassoro district of Inhambane Province, of which 38% were female headed households; this represented about 3800 direct beneficiaries in 18 villages. Goat keepers were organized in 23 producer groups. Key characteristics of the project sites are described in Table 1.Figure 1: Three stages and twelve steps of OM (Earl et al., 2001)Table 1: Key characteristics of the project sitesTopic Udaipur district, Rajasthan State  – India* Inhassoro district, Inhambane Province  – Mozambique Population density 196/km2 11/km2Project households About 2600 524  Literacy levels 58.62% 51% (for Mozambique)Average annual rainfall 600mm 600–800 mmLivelihoods Small land and livestock holdings (subsistence agriculture); wage labour important source of incomeSmall land and livestock holdings (subsistence agriculture); crop production main occupation; cattle numbers very lowMain crops Maize, wheat, barley, chickpea, rape and mustardMaize, groundnuts, beans, cassava, milletAverage goat herd size 6.2 (range 1-16) 8.4 (range 1-30)Marketing practices During main festive period (October to December) and ad hoc throughout the year to meet household demandsDuring festive period (December) and ad hoc throughout the year to meet household demandsNearest goat market 50Km (Udaipur) 200Km (Massinga) Main goat value chain constraintsLack of improved bucks; limited access to animal health services; low number of goats available for sale; limited knowledge about improved husbandry practicesLow number of goats; limited access to animal health services; lack of organization of producers; lack of infrastructure; limited knowledge about improved husbandry practicesMain value chain actors Producers; CAHWs; local traders/butchers; long distance traders; local pharmacist; Animal Husbandry Department; BAIF; research (ILRI, veterinary college)**Producers; CAHWs; local traders/butchers; local retailer; District (SDAE) and Provincial (SPP) Veterinary Services; CARE; research (ILRI)***The imGoats project worked in 2 of 12 blocks of Udaipur district; **CAHW = community animal health workerILRI Research Brief — September 2014 3Design of the OM: The OM framework was designed by the project teams from both countries. During a three day workshop in February 2011, facilitated by an external OM consultant, the overall vision and mission were formulated and boundary partners identified. Four categories of boundary partners were defined: producers (goat keepers and CAHWs), input and service providers (CAHWs, retailer/pharmacist and veterinary ser-vices), post production actors (mainly traders) and enabling agencies (community leaders, government agencies, NGOs, research and private investors). For each boundary partner outcome challenges were formulated while progress mark-ers were defined for three categories: expect to see, like to see and love to see (see Figure 2). Mid-2011, the consultant worked with each country team to adapt the progress markers to the specific project con-text and each progress marker was assigned a subjective measure of what constitutes a high level of achievement. In addition, existing M&E systems of implementing organiza-tions were reviewed to identify information that would support the progress markers, and outcome journals were designed for additional data collection. To further enhance cross learning between the two coun-tries and improve implementation, a mid-term evaluation was organized with the OM consultant in July 2012 to discuss challenges and lessons learned to date. Based on the review, progress indicators were slightly adapted, while some country specific challenges in data collection and analysis were addressed.Figure 2: Intervention logic of the imGoats project based on OM principles (Key: OC=outcome challenge; PM= progress marker; PAs=production actors; PPAs=post production actors; ISPs=input and service providers) (after Deprez, 2013)OM implementation: Due to the short project period and inexperience of the project team with OM, it was decided to implement only some of the components of OM: step 1-5 (vision, mission, boundary partners, outcome challenges, and progress markers) of the Intentional Design stage and step 9 (outcome journals) of the Outcome and Performance Monitoring stage (see figure 1). OM was initially intended to monitor project progress and to use the information for reporting, accountability, and decision making purposes for the project as a whole. Over time, OM became a management tool for planning, feed back and learning in both countries even if the way OM was implemented was very different due to the scale of operation and the existing M&E system in place. In the case of India, the implementing NGO had a M&E system whereby significant amounts of quantitative data was collected. As this system seemed to capture most of the information required for the progress markers, no specific OM journal was designed. Data was collected by 26 CAHWs on a monthly basis (CAHWs are goat keep-ers with minimal education). Each CAHW was responsible for collecting information from 100 households. The data collection forms were adapted to better feed into the progress markers. As part of this process, a data entry operator was hired.In Mozambique, the NGO also had its own M&E system but information (mostly qualitative) was    lacking for some progress markers. OM journals for the extension officers were designed to be filled in on a weekly basis. However, ILRI Research Brief — September 20144Enabling agencies (EA)EA engaged in dialogue with value chain actorsPublic, private and NGO sector promoting and/or developing the goat sectorEA invest in advanced technologiesEA stimulate joint public private sector investmentPublic sector developing infrastructure/facilities Post-production actors  (PPAs)PPAs meet with other value chain actorsShare market information with value chain actorsPPAs take action on decisions made by value chain actorsActing on decisions made at value chain actors meetingsUsing shared information and involved in joint action with other value chain actorsProvision of goats in required quantity and quality based on consumer preference and market demandPPAs establish processing unitInput and service providers (ISPs)ISPs meet with other value chain actorsCAHWs receive trainingService available to producersISP taking actions on decisions made at meetings with value chain actorsDissemination of market informationISPs provide quality services   Provision of new services startedWomen complete CAHWs trainingWomen are involved as input and service providersProduction actors(PAs)Mature goat producer groupsPAs meet with other value chain actorsPAs treat their animalsIncreased goat production and salesProducer groups meet and address value chain issuesProducer groups act on decisions made by value chain actorsPAs get good prizes for goatsPAs introduce improved breedsPAs continue to meet without project involvementPAs actively seeking servicesExpect to see Like to see Love to seeFigure 3: Intended changes and level of achievement (high          , medium          and low         ) (the indicated progress markers are a synthesis of those for India and Mozambique).Beneficiaries and outcomesThere are different groups of beneficiaries: the project team, the boundary partners and the target population. In both countries, OM proved to be a flexible tool for planning, and M&E. It enhanced learning, participation, and accountability within the project teams and among project partners. Besides being used to measure and bring about intended changes, it was also sensitive to unintended changes. Flexibility: One of the qualities of OM is its flexibility and complementarity with other methodologies and existing M&E systems. Some of its stages and steps were applied and adapted based on the context and the data collection tools could easily be transformed to suit the needs in each country. OM allowed for review and adjustment of prog-ress markers during the project implementation process. Participation, learning and accountability: The strong involvement of the project teams in the data collection and analysis contributed to the success of the use of OM. As the entire project team was somehow involved in data col-lection and feedback was provided, the approach resulted in increased involvement and commitment among staff. It also led to increased mutual accountability among the project team members (especially in Mozambique, as result of regular OM meetings) and organizational accountability (for example in India by monitoring the performance of CAHWs). as the extension officers were not familiar with collecting qualitative data, it was decided to hold OM team meet-ings (at an interval of 1-2 months). The ‘OM journals’ were transformed into an ‘OM facilitator guide’ for the meetings and detailed meeting reports were made. As indicated, data was also used for management pur-poses. In India, data were analysed on a monthly basis by the project team for feedback to the CAHWs and project management. In Mozambique OM meetings were used to assess progress and validate information. The outcomes against progress markers were reviewed during a mid-term evaluation and at the end of the project. ILRI Research Brief — September 2014 5Intended and unintended changes: Figure 3 shows the level of achievement in terms of progress markers for the different boundary partners. We see two trends: first that the highest level of achievement has been reached for prog-ress markers in the category ‘expect to see’, followed by ‘like to see’ and ‘love to see’; second, the impact on bound-ary partners is higher among ‘producers’ and ‘input and service providers’ compared to ‘post-production actors’ and ‘enabling agencies’. As the timeframe of the project was limited (30 months), it is expected that the level of achieve-ment will further rise over time.The process of visioning, including setting progress markers along the value chain based on ‘expect to see’, ‘like to see’ and ‘love to see’, created a spectrum of level of achieve-ment and ownership among those involved. This helped to motivate them to achieve and even move beyond the expected; it also helped to identify and address persis-tent problems in the value chain during the course of the project. The flexibility of OM further helped to capture unintended changes the project brought about, such as the systematic fecal sampling in India for better targeted animal health interventions and the development of communal grazing areas to address feed shortages in Mozambique, which was not envisioned at the start of the project. OM also brought about unintended changes in terms of partners’ M&E and management practices.  For instance, in India, the OM has led to improved use of existing data, with potential constructive changes in the modus of operandi (from production towards a more market focus). Challenges and lessons Despite the positive experiences with OM, there were some challenges:Behavioural change is a slow process: While the cat-egorization of behavioural changes in ‘expect, like and love to see’, helps to measure incremental changes over time, a 30-month project period is a short time to realize a stable and irreversible trend.  Monitoring change among enabling agencies: Due to the focus on behavioural change, it was more difficult to assess changes among (some of the) enabling agencies. The project team did not regularly interact with enabling agencies which made it difficult to assess (observable) behavioural change over time. This issue was not sufficiently addressed during the project and should be considered for other projects. Capacity building: As OM is different from traditional M&E systems, additional capacity building of project staff is required.  An internal OM support system may need to be developed to provide technical backstopping. Organizational culture: OM promotes strategic thinking and its relfective nature can help organizations respond to issues that emerge. This may be a challenge in rigid or target-oriented organizations. However, if there are cham-pions among those involved in the project who promote the use of OM, its focus on outcomes and participation r may help to change attitudes among project partners. Involvement of boundary partners: For imGoats it was decided to only involve the project implementing organizations and ILRI in the OM process. This was to test the usefulness of OM in a livestock value chain project and because we did not have sufficient experience with the approach to also involve boundary partners. For future projects we may wish to include more boundary partners. It will be a challenge to involve more distant stakeholders such as traders and enabling agencies in the process as im-mediate benefits from the project are less clear to them. Resource intensity: OM may be more resource inten-sive than traditional M&E systems especially when it is used in larger projects. It requires time, skilled manpower and other logistics to collect and analyse data. Hence, the investment needs to be carefully balanced against the ben-efits. Mechanisms should be designed and tested to make it less resource intensive. ILRI Research Brief — September 20146The research brief has a Creative Commons licence. You are free to re-use or distribute this work, provided credit is given to ILRI.           September 2014ilri.org Better lives through livestock ILRI is a member of the CGIAR ConsortiumBox 30709, Nairobi 00100, Kenya Phone: + 254 20 422 3000 Fax: +254 20 422 3001 Email: ILRI-Kenya@cgiar.orgBox 5689,  Addis Ababa, Ethiopia Phone: +251 11 617 2000  Fax: +251 11 617 2001 Email: ILRI-Ethiopia@cgiar.orgContact: Saskia HendrickxInternational Livestock Research Institute (ILRI)Caixa Postal 2100MaputoMozambiqueEmail: s.hendrickx@cgiar.orghttp://www.imgoats.orgReferencesBendapudi, R. and Hendrickx, S.C.J. 2013. Characterization of small-holder goat production and marketing systems in Udaipur district, Rajasthan State, India: Results of a baseline study. ILRI Research Brief. Nairobi, Kenya: ILRI. Available from: http://cgspace.cgiar.org/handle/10568/21698Boogaard, B.K., Hendrickx, S.C.J. and Swaans, K. 2012. Characterization of smallholder goat production and marketing systems in Inhassoro District, Mozambique: Results of a baseline study. ILRI Research Brief. Nairobi, Kenya: ILRI. Available from: http://cgspace.cgiar.org/handle/10568/32699Deprez, S. 2013. The use of outcome mapping in value-chain development programmes. OM ideas No.7, April 2013. Available from: http://www.outcomemapping.ca/resource/resource.php?id=418Earl, S., Carden, F., and Smutylo, T. 2001 Outcome mapping: Building learn-ing and reflection into development programs. Ottawa: International Development and Research Center (IDRC). IIRR. 2012. Outcome Mapping: Reflecting and learning from application in Eastern Africa. Nairobi, Kenya: International Institute of Rural Recon-struction (IIRR).Smith, R., Mauremootoo, J., and Rassmann, K. 2012. Ten years of Outcome Mapping adaptations and support: An analysis of how and where Out-come Mapping has been applied, how users have experienced OM and the support options available and required for its use. Available from: http://www.outcomemapping.ca/resource/resource.php?id=379Areas for further research•	 How OM can be used successfully in large scale projects given the strong emphasis on qualitative data which needs to be collected and analysed in a differ-ent way than quantitative data; •	 It seems obvious that the involvement of bound-ary partners in the OM process will help to achieve desired behavioural changes. But this may not always be easy as actors may have different objectives. More research is needed to explore to what extent and how boundary partners can best be involved;•	 More insight is required on how log frames and OM can be used effectively together. Acknowledgements This research was supported by the European Com-mission through the International Fund for Agricultural Development - grant C-ECG_45-ILRI

Using outcome mapping as a monitoring and management tool in a small ruminant value chain project

https://cgspace.cgiar.org/bitstream/10568/45946/1/Research_brief_18.pdf

Using outcome mapping as a monitoring and management tool in a small ruminant value chain project

Abstract

Similar works

Full text

Available Versions

CGSpace