Sensitivity-driven simulation development: a case study in forced migration

© 2021 The Authors. This paper presents an approach named sensitivity-driven simulation development (SDSD), where the use of sensitivity analysis (SA) guides the focus of further simulation development and refinement efforts, avoiding direct calibration to validation data. SA identifies assumptions that are particularly pivotal to the validation result, and in response model ruleset refinement resolves those assumptions in greater detail, balancing the sensitivity more evenly across the different assumptions and parameters. We implement and demonstrate our approach to refine agent-based models of forcibly displaced people in neighbouring countries. Over 70.8 million people are forcibly displaced worldwide, of which 26 million are refugees fleeing from armed conflicts, violence, natural disaster or famine. Predicting forced migration movements is important today, as it can help governments and NGOs to effectively assist vulnerable migrants and efficiently allocate humanitarian resources. We use an initial SA iteration to steer the simulation development process and identify several pivotal parameters. We then show that we are able to reduce the relative sensitivity of these parameters in a secondary SA iteration by approximately 54% on average. This article is part of the theme issue 'Reliability and reproducibility in computational science: implementing verification, validation and uncertainty quantification in silico'.European Union Horizon 2020 research and innovation programme VECMA and HiDALGO projects under grant agreement nos 800925 and 824115

Tutorial applications for Verification, Validation and Uncertainty Quantification using VECMA toolkit

The VECMA toolkit enables automated Verification, Validation and Uncertainty Quantification (VVUQ) for complex applications that can be deployed on emerging exascale platforms and provides support for software applications for any domain of interest. The toolkit has four main components including EasyVVUQ for VVUQ workflows, FabSim3 for automation and tool integration, MUSCLE3 for coupling multiscale models and QCG tools to execute application workflows on high performance computing (HPC). A more recent addition to the VECMAtk is EasySurrogate for various types of surrogate methods. In this paper, we present five tutorials from different application domains that apply these VECMAtk components to perform uncertainty quantification analysis, use surrogate models, couple multiscale models and execute sensitivity analysis on HPC. This paper aims to provide hands-on experience for practitioners aiming to test and contrast with their own applications

Estimation of Model Error Using Bayesian Model-Scenario Averaging with Maximum a Posterori-Estimates

International audienceThe lack of an universal modelling approach for turbulence in Reynolds-Averaged Navier–Stokes simulations creates the need for quantifying the modelling error without additional validation data. Bayesian Model-Scenario Averaging (BMSA), which exploits the variability on model closure coefficients across several flow scenarios and multiple models, gives a stochastic, a posteriori estimate of a quantity of interest. The full BMSA requires the propagation of the posterior probability distribution of the closure coefficients through a CFD code, which makes the approach infeasible for industrial relevant flow cases. By using maximum a posteriori (MAP) estimates on the posterior distribution, we drastically reduce the computational costs. The approach is applied to turbulent flow in a pipe at Re= 44,000 over 2D periodic hills at Re=5600, and finally over a generic falcon jet test case (Industrial challenge IC-03 of the UMRIDA project)

Tutorial applications for Verification, Validation and Uncertainty Quantification using VECMA toolkit

Copyright © 2021 The Author(s). The VECMA toolkit enables automated Verification, Validation and Uncertainty Quantification (VVUQ) for complex applications that can be deployed on emerging exascale platforms and provides support for software applications for any domain of interest. The toolkit has four main components including EasyVVUQ for VVUQ workflows, FabSim3 for automation and tool integration, MUSCLE3 for coupling multiscale models and QCG tools to execute application workflows on high performance computing (HPC). A more recent addition to the VECMAtk is EasySurrogate for various types of surrogate methods. In this paper, we present five tutorials from different application domains that apply these VECMAtk components to perform uncertainty quantification analysis, use surrogate models, couple multiscale models and execute sensitivity analysis on HPC. This paper aims to provide hands-on experience for practitioners aiming to test and contrast with their own applications.This work was supported by the VECMA project, which has received funding from the European Union Horizon 2020 research and innovation programme under grant agreement No. 800925. The development of MUSCLE3 and its respective description was supported by the Netherlands eScience Center and NWO under the e-MUSC project. The development of ISR3D was supported by the InSilc project and the In Silico World (ISW) project (European Union Horizon 2020 research and innovation programme grant agreements #777119 and #101016503 respectively). The calculations were performed in the Poznan Supercomputing and Networking Center

VECMAtk: A Scalable Verification, Validation and Uncertainty Quantification toolkit for Scientific Simulations

© 2021 The Authors. We present the VECMA toolkit (VECMAtk), a flexible software environment for single and multiscale simulations that introduces directly applicable and reusable procedures for verification, validation (V&V), sensitivity analysis (SA) and uncertainty quantification (UQ). It enables users to verify key aspects of their applications, systematically compare and validate the simulation outputs against observational or benchmark data, and run simulations conveniently on any platform from the desktop to current multi-petascale computers. In this sequel to our paper on VECMAtk which we presented last year, we focus on a range of functional and performance improvements that we have introduced, cover newly introduced components, and applications examples from seven different domains such as conflict modelling and environmental sciences. We also present several implemented patterns for UQ/SA and V&V, and guide the reader through one example concerning COVID-19 modelling in detail.European Union Horizon 2020 research and innovation programme under grant agreement No 800925; The development of the migration and coronavirus modelling applications was supported by the EU-funded HiDALGO project (grant agreement No 824115)