Lanzanova, D.

Luedeling, Eike

Whitney, C.W.

CGSpace

     Enabling Decision-Making for Agricultural Interventions  TOOL KIT  Lanzanova, D., Center for Development Research, University of Bonn, Germany Luedeling, E., World Agroforestry Centre (ICRAF), Kenya Whitney, C., Center for Development Research, University of Bonn, Germany  Scope and objective Agricultural systems are influenced by a range of environmental, economic and socio-cultural factors. Due to this complexity, policymakers in agricultural development need science-based guidance to make decisions. Classical research approaches, using data-driven models, often fail to inform intervention decisions due to insufficient and low quality data. Given their multifaceted and complex nature, agricultural issues require the integration of knowledge and systems thinking from beyond the discipline-specific approaches that are often used.  Enabling Decision-Making for Agricultural Interventions can provide a new approach and numerical tools to support practical decisions on agricultural systems in the face of risk and imperfect information. This holistic, transdisciplinary methodology allows decision makers:  To carry out rapid assessment of agricultural development interventions when precise information is not available  To explicitly incorporate risk and uncertainty in the analysis through probabilistic simulation of decision outcomes  To identify priorities for further investigations and efficiently allocate limited budgets for measurements with the aim to reduce decision uncertainty  To make rational recommendations on which decision(s) should be taken, given the available information  Description The Enabling Decision-Making for Agricultural Interventions approach can be broken down into nine steps:  1. Decision framing and stakeholder identification The method starts by framing the decision to be modeled. Gaining clarity on specifics of the decision and its context is a key step providing useful advice to decision-makers. Once the decision is clarified, all relevant stakeholders can be identified. A small group of experts (decision-makers, stakeholders, advisers) with a good understanding of the decision is selected. The modeling team consists of this group of experts and the decision analysts.  2. Participatory analysis of the decision problem  During a workshop, the expert group engages in a participatory analysis of the decision problem. Participants discuss the decision’s objectives and identify any factors they deem important for the proposed interventions, such as costs, benefits or project risks that may affect the outcome.  3. Development of the conceptual model  The information collected earlier in the participatory analysis of the decision problem is assembled into a conceptual, graphical model. This is constructed as a causal impact pathway with causal relationships based on experts' expectations, gathered during brainstorming sessions.     Conceptual model development aims to capture the “big picture” of the decision by gathering all system dynamics and relevant issues, without taking constraints of measurement into account.  4. Development of the mathematical model The conceptual model is then translated into a mathematical model and coded as a function for the R programming language (R Development Core Team, 2017). It is represented by a set of equations that reflect as much as possible the experts’ understanding of the decision. For the computer modeling, R’s decisionSupport package can be used (available online; Luedeling and Göhring, 2016).  5. Calibration of experts All experts are required to undergo ‘calibration training’, which teaches them how to make estimates as reliably as possible. The training consists of a series of procedures, grounded on research findings in cognitive psychology. Basically, participants learn how to assess their state of uncertainty and reduce errors of judgement through exercises that reveal to them their personal biases (overconfidence or underconfidence). To this end, they compare their performance in responding to trivia questions to the correct answers to these questions. Rather than providing ‘best guesses’, participants are requested to provide two numbers, for which they are 90% sure that the correct answer is between these numbers. Perfectly ‘calibrated’ estimators should get 90% of their range estimates correct. Once exposed to their biases (most people are initially overconfident), experts are instructed in a set of mental techniques that has been shown to improve people’s ability to provide accurate estimates. More information can be found in Hubbard (2014).  6. Parameterization of the model Where no reliable data are available, experts’ knowledge is the main source of information. At this stage, the team is requested to provide estimations of the model’s input variables. Estimates consist of a probability distribution and a confidence interval. The confidence interval (defined by the upper and lower bounds) has a predefined chance (e.g. 90%) of containing the right value. In practice, if respondents do not feel confident in selecting the most appropriate probability distribution, the normal distribution is selected and only the confidence interval is requested from the participants.  7. Simulation of the decision Once the model is parameterized, the decision model is run a large number of times (normally 10,000 times) as a probabilistic Monte Carlo simulation. Each run provides one possible outcome. The totality of all model runs generates a probability distribution illustrating the outcomes that are plausible given the experts’ current state of uncertainty. R’s decisionSupport package (Luedeling and Göhring, 2016) is an efficient tool to generate probabilistic outcomes of the decision.   8. Analysis of results and identification of the most important knowledge gaps The output of the Monte Carlo simulations can reveal a clearly preferred option (e.g. a specific intervention in a group of possible interventions). However, high uncertainty in the input variables (e.g. wide confidence intervals) may result in a range of potential outcomes that is too wide to give effective guidance as to which decision option is preferable. In such a case, when no clearly preferred option emerges, Partial Least Squares (PLS) analysis, in particular its Variable Importance in Projection (VIP) metric, which is provided in R’s decisonSupport package (Luedeling and Göhring, 2016), allows for identification of variables that most affect the uncertainty of the overall outcome of the model. In addition to this sensitivity analysis, decision analysts also compute the Expected Value of Perfect Information (EVPI), which can be interpreted as a rational     willingness-to-pay to gain access to perfect information. Rather than referring to the absolute value of the decision outcome, the EVPI is concerned only with whether the outcome is positive or negative, because this is the only criterion that decides on which option should be preferred by a rational decision-maker. Measurements of input variables with the highest EVPI, which can be used to update the decision model, can help to narrow uncertainty about how the decision should be taken.   9. Model refinement  The process described in step 8 (additional measurements and modeling procedure) is repeated until the best option is determined. For decision support activities to be effective, it is highly desirable that decision-makers perceive they can make a well-informed decision in order to ensure that the model results are considered in the decision-making process (Fig. 1).   Graphic from Luedeling and Shepherd (2016) Figure 1. Illustration of the Decision Analysis process  Example from Lagdwenda, Burkina-Faso The approach has been implemented to study intervention alternatives to prevent sedimentation in the reservoir of Lagdwenda, Burkina-Faso. The Lagdwenda reservoir is used for the irrigation of crops (vegetables and rice) and is critical for the livelihoods of local communities. An intervention decision model was developed to identify the best of several intervention options to secure agricultural production.  With this aim in mind, a workshop was help to engage local experts (key stakeholders and decision-makers) in a participatory process. The participants discussed the sedimentation issue and co-designed three interventions for sedimentation control: 1) dredging along the main inlet, 2) building rock dams along the streams upstream and 3) implementing a buffer protection scheme around the reservoir. They then collaboratively built a model that attempted to project the impacts of these interventions on agricultural production.  In a second phase of the Lagdwenda case, the conceptual model developed by the team was converted into computer code as a Monte Carlo-based decision model. The model was parameterized by the team of experts and used to assess interventions or combinations of interventions. Simulation results show that the preferred option (out of seven) is a combination of the three interventions. The outcome for this option is presented in Fig. 2.     Graphical representations of the decision analysis consist of four illustrations. The Net Present Value (upper left) and the cash flow (bottom left) refer to measures that determine the rate of return of the investment. The NPV gives the number of times (“frequency”) that each outcome of the distribution (bar of the histogram) was realized when the model was simulated. The cash flow is a series of monetary values, either negative (e.g. initial investment costs of interventions) or positive (e.g. marginal revenue generated by the interventions in a specific year) over a time period. On the figure, uncertainty on the value of the cash flow is represented by quantiles around the median. The Expected Value of Perfect Information (top right) and the Variable Importance in the Projection (bottom right) regard the value of information analysis that seeks to assign a value to reduction of uncertainty about specific variables.    Figure 2. Simulated outcomes of implementing all three interventions EVPI is the difference between the expected value of a decision made with perfect information and the expected value of the decision with current imperfect information. For example, results show that the profit generated per ton of vegetables presents a non-zero information value. This means that a rational decision-maker should be willing to pay up to the value indicated by the analysis to get full information on this variable.  The VIP gives the level of significance of a variable for explaining variation in the simulated outcome. On the figure, the threshold for considering a variable important is represented by the vertical black straight line. Above this threshold, variables positively correlated with the outcome are shown in green and in red if the correlation is negative. In the example, the analysis of the VIP scores revealed     that the most influential variable is the profit per ton of vegetables. The second most important uncertainty is the discount rate.  Recommendations  State the objectives As stated above, it is fundamentally important that the decision to be addressed is clearly described. Without clear objectives, the effectiveness of an intervention cannot be assessed and no preferred option will appear. Without clear objectives, the benefit of better information for a specific input variable would be difficult to evaluate, since it is directly related to its impact on the uncertainty of the overall outcome of the model.    Include decision-makers The inclusion of decision-makers in the team of experts is crucial. This can help to close the gap between research and practice. It allows the decision team to explicitly take decision-makers’ preferences into account and strengthen the analysis’ impact by increasing the probability that research outcomes will be considered in relevant decisions.   Beware of decision analysts’ biases  The conceptual model begins as a graphical representation of causal relationships between variables and is later translated into equations and converted into computer code by the decision analysts. In so doing, they attempt to interpret the experts’ perception of the impact pathway, but there is a possibility that they can introduce their own biases and perceptions into the decision model. Therefore, it is important that the team (experts and decision analysts) discusses all points in detail during the workshop to clearly understand what is meant by each risk specified in the model. It is also critical to obtain feedback from experts on the resulting model.    Do not look for perfection Research able to deliver site-specific decision support actually rarely exists. The proposed approach is intended to be put into practice on concrete decision cases with limited resources dedicated to field research. Thus, it has advantages over resource-intensive research projects, which often aim to produce knowledge on a system rather than targeted information for decision-making.    Integrate diversity of risks and uncertainties It would be hard to know ALL the risks and uncertainties that exist in complex agricultural systems. However, decision-analysts should attempt to think beyond the variables that seem the most important to them. They should also consider risk and uncertainty around external variables, e.g. include effects of institutional or behavioral factors, when considering ecological effects such as climate variability. There is no way to know in advance which risks and uncertainties are most critical for the success of an intervention. Efficient and holistic workshop facilitation (that allows elicitation of relevant information) can safeguard against the temptation to focus only on the most obvious ones.  References Hubbard, DW (2014). How to Measure Anything-Finding the Value of Intangibles in Business. Hoboken, NJ: Wiley and Sons.  Luedeling E, Oord AL, Kiteme B, Ogalleh S, Malesu M, Shepherd KD and De Leeuw J (2015). Fresh groundwater for Wajir—ex-ante assessment of uncertain benefits for multiple stakeholders in a water supply project in Northern Kenya. Front. Environ. Sci. 3:16. doi: 10.3389/fenvs.2015.00016      Luedeling E and Göhring L (2016). decisionSupport: Quantitative Support of Decision Making under Uncertainty. CRAN archive; https://cran.r-project.org/web/packages/decisionSupport  Luedeling E, Shepherd K (2016). Decision-Focused Agricultural Research. Solutions 7(5):46-54   R Development Core Team, 2017. R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria.  Shepherd K, Hubbard D, Fenton N, Claxton K, Luedeling E and de Leeuw J (2015). Development goals should enable decision-making. Nature 523:152-154. doi:10.1038/523152a                                         We would like to thank all donors who supported this research through their contributions to the CGIAR Fund.  

Enabling decision-making for agricultural interventions

https://cgspace.cgiar.org/bitstream/10568/83037/5/WLEmemo_DecisionAnalysis_method%20guide.pdf

Enabling decision-making for agricultural interventions

Abstract

Similar works

Full text

Available Versions

CGSpace