Towards Best Practice Framing of Uncertainty in Scientific Publications: A Review of Water Resources Research Abstracts

Uncertainty is recognized as a key issue in water resources research, amongst other sciences. Discussions of uncertainty typically focus on tools and techniques applied within an analysis, e.g. uncertainty quantification and model validation. But uncertainty is also addressed outside the analysis, in writing scientific publications. The language that authors use conveys their perspective of the role of uncertainty when interpreting a claim —what we call here “framing” the uncertainty. This article promotes awareness of uncertainty framing in four ways. 1) It proposes a typology of eighteen uncertainty frames, addressing five questions about uncertainty. 2) It describes the context in which uncertainty framing occurs. This is an interdisciplinary topic, involving philosophy of science, science studies, linguistics, rhetoric, and argumentation. 3) We analyze the use of uncertainty frames in a sample of 177 abstracts from the Water Resources Research journal in 2015. This helped develop and tentatively verify the typology, and provides a snapshot of current practice. 4) Provocative recommendations promote adjustments for a more influential, dynamic science. Current practice in uncertainty framing might be described as carefully-considered incremental science. In addition to uncertainty quantification and degree of belief (present in ~5% of abstracts), uncertainty is addressed by a combination of limiting scope, deferring to further work (~25%) and indicating evidence is sufficient (~40%) – or uncertainty is completely ignored (~8%). There is a need for public debate within our discipline to decide in what context different uncertainty frames are appropriate. Uncertainty framing cannot remain a hidden practice evaluated only by lone reviewers

Towards Trust-Aware Human-Automation Interaction: An Overview of the Potential of Computational Trust Models

Several computational models have been proposed to quantify trust and its relationship to other system variables. However, these models are still under-utilised in human-machine interaction settings due to the gap between modellers’ intent to capture a phenomenon and the requirements for employing the models in a practical context. Our work amalgamates insights from the system modelling, trust, and human-autonomy teaming literature to address this gap. We explore the potential of computational trust models in the development of trust-aware systems by investigating three research questions: 1- At which stages of development can trust models be used by designers? 2- how can trust models contribute to trust-aware systems? 3- which factors should be incorporated within trust models to enhance models’ effectiveness and usability? We conclude with future research directions

Autonomous Swarm Shepherding Using Curriculum-Based Reinforcement Learning

Autonomous shepherding is a bio-inspired swarm guidance approach, whereby an artificial sheepdog guides a swarm of artificial or biological agents, such as sheep, towards a goal. While the success in this guidance depends on the set of behaviours exhibited by the sheepdog, the main source of complexity for learning effective behaviours lies within the highly non-linear dynamics featured among the swarm members as well as between the swarm and the sheepdog. Attempts to apply reinforcement learning (RL) to shepherding have so far relied greatly on rule-based algorithms for calculating waypoints to guide the RL algorithm. In this paper, we propose a curriculum-based approach for RL that does not rely on any external algorithm to pre-determine waypoints for the sheepdog. Instead, the approach uses task decomposition by formulating shepherding in terms of two sub-tasks: (1) pushing an agent from a start to a target location and (2) selecting between collecting scattered agents or driving the biggest cluster of agents to the goal. Simple-to-complex curriculum learning is used to accelerate the learning of each sub-task. For the first sub-task, the complexity is gradually increased over training time, whereas for the second sub-task a simplified environment is designed for initial learning before proceeding with the main environment. The proposed approach results in high-performance shepherding with a success rate of about 96%. While curriculum learning was found to expedite the learning of the first sub-task, it was not as efficient for the second sub-task. Our analyses highlight the need for the careful design of the curriculum to ensure that skills acquired in intermediate tasks are useful for the main tasks

Machine Education: Designing semantically ordered and ontologically guided modular neural networks

The literature on machine teaching, machine education, and curriculum design for machines is in its infancy with sparse papers on the topic primarily focusing on data and model engineering factors to improve machine learning. In this paper, we first discuss selected attempts to date on machine teaching and education. We then bring theories and methodologies together from human education to structure and mathematically define the core problems in lesson design for machine education and the modelling approaches required to support the steps for machine education. Last, but not least, we offer an ontology-based methodology to guide the development of lesson plans to produce transparent and explainable modular learning machines, including neural networks.Comment: IEEE Symposium Series on Computational Intelligence, 201

Toward best practice framing of uncertainty in scientific publications: A review of Water Resources Research abstracts

Uncertainty is recognized as a key issue in water resources research, among other sciences. Discussions of uncertainty typically focus on tools and techniques applied within an analysis, e.g., uncertainty quantification and model validation. But uncertainty is also addressed outside the analysis, in writing scientific publications. The language that authors use conveys their perspective of the role of uncertainty when interpreting a claim—what we call here “framing” the uncertainty. This article promotes awareness of uncertainty framing in four ways. (1) It proposes a typology of eighteen uncertainty frames, addressing five questions about uncertainty. (2) It describes the context in which uncertainty framing occurs. This is an interdisciplinary topic, involving philosophy of science, science studies, linguistics, rhetoric, and argumentation. (3) We analyze the use of uncertainty frames in a sample of 177 abstracts from the Water Resources Research journal in 2015. This helped develop and tentatively verify the typology, and provides a snapshot of current practice. (4) We make provocative recommendations to achieve a more influential, dynamic science. Current practice in uncertainty framing might be described as carefully considered incremental science. In addition to uncertainty quantification and degree of belief (present in ∼5% of abstracts), uncertainty is addressed by a combination of limiting scope, deferring to further work (∼25%) and indicating evidence is sufficient (∼40%)—or uncertainty is completely ignored (∼8%). There is a need for public debate within our discipline to decide in what context different uncertainty frames are appropriate. Uncertainty framing cannot remain a hidden practice evaluated only by lone reviewers.J. H. A. Guillaume was funded by the Academy of Finland project WASCO (grant 305471) and Emil Aaltonen Foundation funded project eat-lesswater. C. Helgeson received funding from the Arts and Humanities Research Council (AHRC) project Managing Severe Uncertainty (AH/ J006033/1) and L’Agence Nationale de la Recherche (ANR) project DUSUCA (ANR-14-CE29–0003-01)

The effects of climate change on ecologically-relevant flow regime and water quality attributes

The management of freshwater ecosystems is usually targeted through the regulation of water quantity (limiting diversions and providing environmental flows) and regulation of water quality (setting limits or targets for constituent concentrations). Clima

A framework for characterising and evaluating the effectiveness of environmental modelling

Environmental modelling is transitioning from the traditional paradigm that focuses on the model and its quantitative performance to a more holistic paradigm that recognises successful model-based outcomes are closely tied to undertaking modelling as a social process, not just as a technical procedure. This paper redefines evaluation as a multi-dimensional and multi-perspective concept, and proposes a more complete framework for identifying and measuring the effectiveness of modelling that serves the new paradigm. Under this framework, evaluation considers a broader set of success criteria, and emphasises the importance of contextual factors in determining the relevance and outcome of the criteria. These evaluation criteria are grouped into eight categories: project efficiency, model accessibility, credibility, saliency, legitimacy, satisfaction, application, and impact. Evaluation should be part of an iterative and adaptive process that attempts to improve model-based outcomes and foster pathways to better futures

Eight grand challenges in socio-environmental systems modeling

Modeling is essential to characterize and explore complex societal and environmental issues in systematic and collaborative ways. Socio-environmental systems (SES) modeling integrates knowledge and perspectives into conceptual and computational tools that explicitly recognize how human decisions affect the environment. Depending on the modeling purpose, many SES modelers also realize that involvement of stakeholders and experts is fundamental to support social learning and decision-making processes for achieving improved environmental and social outcomes. The contribution of this paper lies in identifying and formulating grand challenges that need to be overcome to accelerate the development and adaptation of SES modeling. Eight challenges are delineated: bridging epistemologies across disciplines; multi-dimensional uncertainty assessment and management; scales and scaling issues; combining qualitative and quantitative methods and data; furthering the adoption and impacts of SES modeling on policy; capturing structural changes; representing human dimensions in SES; and leveraging new data types and sources. These challenges limit our ability to effectively use SES modeling to provide the knowledge and information essential for supporting decision making. Whereas some of these challenges are not unique to SES modeling and may be pervasive in other scientific fields, they still act as barriers as well as research opportunities for the SES modeling community. For each challenge, we outline basic steps that can be taken to surmount the underpinning barriers. Thus, the paper identifies priority research areas in SES modeling, chiefly related to progressing modeling products, processes and practices

Effective modeling for integrated water resource management: a guide to contextual practices by phases and steps and future opportunities

The effectiveness of Integrated Water Resource Management (IWRM) modeling hinges on the quality of practices employed through the process, starting from early problem definition all the way through to using the model in a way that serves its intended purpose. The adoption and implementation of effective modeling practices need to be guided by a practical understanding of the variety of decisions that modelers make, and the information considered in making these choices. There is still limited documented knowledge on the modeling workflow, and the role of contextual factors in determining this workflow and which practices to employ. This paper attempts to contribute to this knowledge gap by providing systematic guidance of the modeling practices through the phases (Planning, Development, Application, and Perpetuation) and steps that comprise the modeling process, positing questions that should be addressed. Practice-focused guidance helps explain the detailed process of conducting IWRM modeling, including the role of contextual factors in shaping practices. We draw on findings from literature and the authors’ collective experience to articulate what and how contextual factors play out in employing those practices. In order to accelerate our learning about how to improve IWRM modeling, the paper concludes with five key areas for future practice-related research: knowledge sharing, overcoming data limitations, informed stakeholder involvement, social equity and uncertainty management. © 2019 Elsevier Lt

Investigating Gender Differences in Human Interactions with a Transparent Swarm

Research suggests that different operationalisations of automation transparency can influence team collaboration and performance. Yet, little is known about how gender differences can affect humans’ ability to process the information provided by their automation teammate when the automation is transparent. The significance of this research question stems from the fact that the broader areas of judgment and decision making show that females and males deploy different information processing strategies. The aim of this paper is to explore whether gender differences exist in the way people interact with a transparent swarm. We conducted a user study in which 33 subjects (15 males and 18 females) interacted with a simulated swarm under two conditions: with and without transparency. While no significant differences were detected under the control condition, results indicate that when transparency was added, males were able to utilise transparency early on, while females needed significantly more time to benefit from it. Besides, under the transparency condition, females exhibited significantly longer response times than males. How-ever, as time progresses and towards the last time window of the experiment, females could achieve slightly higher decision accuracy than males. These findings stress the need to consider gender differences when designing transparent human-machine interaction and its training protocols.</p