Gamifying and evaluating problem structuring: A card game workshop for generating decision objectives

Serious games, gamification, or game-based interventions are increasingly used as tools to aid participatory decision-making processes, but their evaluation is often not very rigorous. Therefore, it is still unclear whether game-based interventions are really beneficial. We focused on the following overarching question: how effective are game-based interventions specifically designed to support decision-making processes. We used an illustrative case to reflect on this question. Using a published framework proposing that design processes of game-based interventions and their evaluation are intertwined, we designed simultaneously (1) a game-based intervention, specifically a card game and a workshop structure in which this card game is to be used, and (2) its evaluation procedure, formulating evaluation questions and proposing measure instruments based on the literature. We pre-tested the evaluation procedure in a small pilot study with 10 students. We illustrate the use of the design framework for an intervention to generate objectives in a decision-making process about sustainable wastewater management. Through our illustrative case, we identify future research opportunities about designing game-based interventions and evaluating their effectiveness. We found that it is possible to address the dual challenge of game-based interventions for participatory decision-making processes: (1) designing an informative and engaging game-based intervention without telling participants what to think and (2) designing a tailored evaluation procedure. Designing the game-based intervention and its evaluation simultaneously is valuable, because both are strongly intertwined. However, conducting the evaluation is demanding and requires the collaborative efforts of scientists, including across disciplinary boundaries. For instance, the data collection effort could be distributed among different research groups to increase sample size. This would allow including control treatment(s) and covering the variation span of the confounding factors more broadly. All material is made openly available to foster collaborative future research

Structuring problems for Multi-Criteria Decision Analysis in practice : a literature review of method combinations

Structuring problems for Multi-Criteria Decision Analysis (MCDA) has attracted increasing attention over the past 20 years from both a conceptual and a practical perspective. This is reflected in a significant growth in the number of published applications which use a formal approach to problem structuring in combination with an analytic method for multi-criteria analysis. The problem structuring approaches (PSMs) include general methodologies such as Checkland's Soft Systems Method (SSM), Eden and Ackermann's Strategic Options Design and Analysis (SODA) and other methods that focus on a particular aspect. We carried out a literature review that covers eight PSMs (Cognitive and Causal Maps, DPSIR, Scenario Planning, SSM, Stakeholder Analysis, Strategic Choice Approach, SODA and SWOT) and seven MCDA methods (AHP, ANP, ELECTRE, MAUT, MAVT, PROMETHEE and TOPSIS). We first identified and analysed 333 articles published during 2000-2015, then selected 68 articles covering all PSM-MCDA combinations, which were studied in detail to understand the associated processes, benefits and challenges. The three PSMs most commonly combined with MCDA are SWOT, Scenario Planning and DPSIR. AHP was by far the most commonly applied MCDA method. Combining PSMs with MCDA produces a richer view of the decision situation and enables more effective support for different phases of the decision-making process. Some limitations and challenges in combining PSMs and MCDA are also identified, most importantly relating to building a value tree and assigning criteria weights

Barriers to evidence use for sustainability: Insights from pesticide policy and practice.

Calls for supporting sustainability through more and better research rest on an incomplete understanding of scientific evidence use. We argue that a variety of barriers to a transformative impact of evidence arises from diverse actor motivations within different stages of evidence use. We abductively specify this variety in policy and practice arenas for three actor motivations (truth-seeking, sense-making, and utility-maximizing) and five stages (evidence production, uptake, influence on decisions, effects on sustainability outcomes, and feedback from outcome evaluations). Our interdisciplinary synthesis focuses on the sustainability challenge of reducing environmental and human health risks of agricultural pesticides. It identifies barriers resulting from (1) truth-seekers' desire to reduce uncertainty that is complicated by evidence gaps, (2) sense-makers' evidence needs that differ from the type of evidence available, and (3) utility-maximizers' interests that guide strategic evidence use. We outline context-specific research-policy-practice measures to increase evidence use for sustainable transformation in pesticides and beyond

Barriers to evidence use for sustainability: Insights from pesticide policy and practice

Calls for supporting sustainability through more and better research rest on an incomplete understanding of scientific evidence use. We argue that a variety of barriers to a transformative impact of evidence arises from diverse actor motivations within different stages of evidence use. We abductively specify this variety in policy and practice arenas for three actor motivations (truth-seeking, sense-making, and utility-maximizing) and five stages (evidence production, uptake, influence on decisions, effects on sustainability outcomes, and feedback from outcome evaluations). Our interdisciplinary synthesis focuses on the sustainability challenge of reducing environmental and human health risks of agricultural pesticides. It identifies barriers resulting from (1) truth-seekers’ desire to reduce uncertainty that is complicated by evidence gaps, (2) sense-makers’ evidence needs that differ from the type of evidence available, and (3) utility-maximizers’ interests that guide strategic evidence use. We outline context-specific research–policy–practice measures to increase evidence use for sustainable transformation in pesticides and beyond

Operational Research: Methods and Applications

Throughout its history, Operational Research has evolved to include a variety of methods, models and algorithms that have been applied to a diverse and wide range of contexts. This encyclopedic article consists of two main sections: methods and applications. The first aims to summarise the up-to-date knowledge and provide an overview of the state-of-the-art methods and key developments in the various subdomains of the field. The second offers a wide-ranging list of areas where Operational Research has been applied. The article is meant to be read in a nonlinear fashion. It should be used as a point of reference or first-port-of-call for a diverse pool of readers: academics, researchers, students, and practitioners. The entries within the methods and applications sections are presented in alphabetical order

Four Common Simplifications of Multi-Criteria Decision Analysis do not hold for River Rehabilitation

River rehabilitation aims at alleviating negative effects of human impacts such as loss of biodiversity and reduction of ecosystem services. Such interventions entail difficult trade-offs between different ecological and often socio-economic objectives. Multi-Criteria Decision Analysis (MCDA) is a very suitable approach that helps assessing the current ecological state and prioritizing river rehabilitation measures in a standardized way, based on stakeholder or expert preferences. Applications of MCDA in river rehabilitation projects are often simplified, i.e. using a limited number of objectives and indicators, assuming linear value functions, aggregating individual indicator assessments additively, and/or assuming risk neutrality of experts. Here, we demonstrate an implementation of MCDA expert preference assessments to river rehabilitation and provide ample material for other applications. To test whether the above simplifications reflect common expert opinion, we carried out very detailed interviews with five river ecologists and a hydraulic engineer. We defined essential objectives and measurable quality indicators (attributes), elicited the experts´ preferences for objectives on a standardized scale (value functions) and their risk attitude, and identified suitable aggregation methods. The experts recommended an extensive objectives hierarchy including between 54 and 93 essential objectives and between 37 to 61 essential attributes. For 81% of these, they defined non-linear value functions and in 76% recommended multiplicative aggregation. The experts were risk averse or risk prone (but never risk neutral), depending on the current ecological state of the river, and the experts´ personal importance of objectives. We conclude that the four commonly applied simplifications clearly do not reflect the opinion of river rehabilitation experts. The optimal level of model complexity, however, remains highly case-study specific depending on data and resource availability, the context, and the complexity of the decision problem

Operational research for, with, and by citizens : an overview

Interest in citizen participation is increasing generally. Almost all operational research (OR) is engaged with clients, but it is mainly in the areas of Soft and Community OR that wider stakeholder and citizen participation has been a significant focus. It is the involvement of citizens that is the subject of this paper. We surveyed OR literature and compiled a corpus of 62 studies, the earliest from 1970, to systematically characterize the involvement of citizens in OR processes. Our review produced three findings: First, some fields of OR have embraced citizen participation, but this is not yet a major concern outside the field of Community OR. Second, citizen participation in OR processes is often driven by a moral rationale. Third, progress in information and communication technology (ICT) enables broad participation, but traditional processes requiring physical presence can also be participatory. From these insights, we formulate research opportunities for OR. (1) OR may join Community OR's endeavor to engage with and empower citizens who have so far rarely been involved in OR processes. (2) OR may identify benefits and drawbacks of digital OR processes in empirical studies. (3) OR may determine whether involving large numbers of citizens is suitable for the societal scale. (4) OR may research building and maintaining trust. (5) OR may join efforts for data protection of participants. (6) OR may systematically report and reflect on participatory OR processes. (7) OR should continue researching the fair aggregation of individual inputs. Citizen participation in OR is topical and challenging. Pursuing these research opportunities will contribute to OR fulfilling its mandate of better decision-making in close cooperation with all affected stakeholders

Stakeholder analysis combined with social network analysis provides fine-grained insights into water infrastructure planning processes

Environmental policy and decision-making are characterized by complex interactions between different actors and sectors. As a rule, a stakeholder analysis is performed to understand those involved, but it has been criticized for lacking quality and consistency. This lack is remedied here by a formal social network analysis that investigates collaborative and multi-level governance settings in a rigorous way. We examine the added value of combining both elements. Our case study examines infrastructure planning in the Swiss water sector. Water supply and wastewater infrastructures are planned far into the future, usually on the basis of projections of past boundary conditions. They affect many actors, including the population, and are expensive. In view of increasing future dynamics and climate change, a more participatory and long-term planning approach is required. Our specific aims are to investigate fragmentation in water infrastructure planning, to understand how actors from different decision levels and sectors are represented, and which interests they follow. We conducted 27 semi-structured interviews with local stakeholders, but also cantonal and national actors. The network analysis confirmed our hypothesis of strong fragmentation: we found little collaboration between the water supply and wastewater sector (confirming horizontal fragmentation), and few ties between local, cantonal, and national actors (confirming vertical fragmentation). Infrastructure planning is clearly dominated by engineers and local authorities. Little importance is placed on longer-term strategic objectives and integrated catchment planning, but this was perceived as more important in a second analysis going beyond typical questions of stakeholder analysis. We conclude that linking a stakeholder analysis, comprising rarely asked questions, with a rigorous social network analysis is very fruitful and generates complementary results. This combination gave us deeper insight into the socio-political-engineering world of water infrastructure planning that is of vital importance to our well-being

Can online interfaces enhance learning for public decision-making? : eliciting citizens’ preferences for multicriteria decision analysis

Innovative online interfaces informing and consulting citizens about their preferences for multicriteria decision analysis (MCDA) could make public decision-making more participatory. We propose a three-faceted learning for decision-making framework and used it to test newly-designed online weight elicitation interfaces. We investigated two features meant to enhance learning: fully-fledged gamification with a narrative, interaction with nonplayer characters, and ambient music, and learning loops (LL) using consistency checks of elicited weights and the challenge to resolve inconsistencies. We operationalized our framework with a novel systematic set of measure instruments providing complementary data types. We designed a 2 × 2 between-subject experiment with pre- and postquestionnaires. Answers from 769 respondents, representative of the Swiss population in age and gender, indicated that the interfaces successfully raised awareness about wastewater management. Gamification was helpful: respondents performed better in the factual learning test, and unexpected social learning occurred. However, gamification lowered the perception of process understanding. The LL were beneficial: objectively, respondents performed better in the factual learning test. However, respondents perceived the LL as cognitively demanding and their factual learning as lower. Our structured assessment highlighted the need for further research to investigate, for instance, high interpersonal variability and the disparities between tested and perceived learning. Measuring preference construction remains challenging; and social learning should be added to the assessment framework. Applying such structured assessment of learning outcomes to more traditional operational research interventions would provide a baseline for future comparison