Open TURNS: An industrial software for uncertainty quantification in simulation

The needs to assess robust performances for complex systems and to answer tighter regulatory processes (security, safety, environmental control, and health impacts, etc.) have led to the emergence of a new industrial simulation challenge: to take uncertainties into account when dealing with complex numerical simulation frameworks. Therefore, a generic methodology has emerged from the joint effort of several industrial companies and academic institutions. EDF R&D, Airbus Group and Phimeca Engineering started a collaboration at the beginning of 2005, joined by IMACS in 2014, for the development of an Open Source software platform dedicated to uncertainty propagation by probabilistic methods, named OpenTURNS for Open source Treatment of Uncertainty, Risk 'N Statistics. OpenTURNS addresses the specific industrial challenges attached to uncertainties, which are transparency, genericity, modularity and multi-accessibility. This paper focuses on OpenTURNS and presents its main features: openTURNS is an open source software under the LGPL license, that presents itself as a C++ library and a Python TUI, and which works under Linux and Windows environment. All the methodological tools are described in the different sections of this paper: uncertainty quantification, uncertainty propagation, sensitivity analysis and metamodeling. A section also explains the generic wrappers way to link openTURNS to any external code. The paper illustrates as much as possible the methodological tools on an educational example that simulates the height of a river and compares it to the height of a dyke that protects industrial facilities. At last, it gives an overview of the main developments planned for the next few years

Epistemological issues concerning computer simulations in science and their implications for science education

Computers and simulations represent an undeniable aspect of daily scientific life, the use of simulations being comparable to the introduction of the microscope and the telescope, in the development of knowledge. In science education, simulations have been proposed for over three decades as useful tools to improve the conceptual understanding of students and the development of scientific capabilities. However, various epistemological aspects that relate to simulations have received little attention. Although the absence of this discussion is due to various factors, among which the relatively recent interest in the analysis of longstanding epistemological questions concerning the use of simulations, the inclusion of this discussion on the research agenda in science education appears relevant, if we wish to educate scientifically literate students in a vision of the nature of science closer to the work conducted by researchers today. In this paper we review some contemporary thoughts emerging from philosophy of science about simulations in science and set out questions that we consider of relevance for discussion in science education, in particular related with model-based learning and experimental wor

The Development of the Action Concept Inventory

An Action Concept Inventory was created to measure student understanding of the principles that underpin the principle of stationary action. International Action experts were consulted to create a list of 10 concepts that underpin the principle of least action. This list was then used to create the action concept inventory, an eighteen question multiple choice conceptual test which measures student understanding of the principle of least action. 77 ANU first year students studying action physics took the Action Concept Inventory as both pre-test and post-test. 61 physics honours students, PhD students and academics from the ANU also completed the action concept inventory. This inventory was validated through student interviews, expert review and using statistical analysis. In addition to one on one think aloud interviews, students were interviewed in pairs, this made the interview process more comfortable for the students and decreased the frequency with which students misinterpreted questions. It was found that 15 of the 18 questions were valid measures of students' understanding of action concepts, while three of the questions needed to be revised before the inventory is published. A number of 'Explore All Paths' misconceptions were discovered and as a result it is recommended that one more question is added to the inventory to probe understanding of this concept

Psychometrics in Practice at RCEC

A broad range of topics is dealt with in this volume: from combining the psychometric generalizability and item response theories to the ideas for an integrated formative use of data-driven decision making, assessment for learning and diagnostic testing. A number of chapters pay attention to computerized (adaptive) and classification testing. Other chapters treat the quality of testing in a general sense, but for topics like maintaining standards or the testing of writing ability, the quality of testing is dealt with more specifically.\ud All authors are connected to RCEC as researchers. They present one of their current research topics and provide some insight into the focus of RCEC. The selection of the topics and the editing intends that the book should be of special interest to educational researchers, psychometricians and practitioners in educational assessment

An investigation into hazard-centric analysis of complex autonomous systems

This thesis proposes a hypothesis that a conventional, and essentially manual, HAZOP process can be improved with information obtained with model-based dynamic simulation, using a Monte Carlo approach, to update a Bayesian Belief model representing the expected relations between cause and effects – and thereby produce an enhanced HAZOP. The work considers how the expertise of a hazard and operability study team might be augmented with access to behavioural models, simulations and belief inference models. This incorporates models of dynamically complex system behaviour, considering where these might contribute to the expertise of a hazard and operability study team, and how these might bolster trust in the portrayal of system behaviour. With a questionnaire containing behavioural outputs from a representative systems model, responses were collected from a group with relevant domain expertise. From this it is argued that the quality of analysis is dependent upon the experience and expertise of the participants but this might be artificially augmented using probabilistic data derived from a system dynamics model. Consequently, Monte Carlo simulations of an improved exemplar system dynamics model are used to condition a behavioural inference model and also to generate measures of emergence associated with the deviation parameter used in the study. A Bayesian approach towards probability is adopted where particular events and combinations of circumstances are effectively unique or hypothetical, and perhaps irreproducible in practice. Therefore, it is shown that a Bayesian model, representing beliefs expressed in a hazard and operability study, conditioned by the likely occurrence of flaw events causing specific deviant behaviour from evidence observed in the system dynamical behaviour, may combine intuitive estimates based upon experience and expertise, with quantitative statistical information representing plausible evidence of safety constraint violation. A further behavioural measure identifies potential emergent behaviour by way of a Lyapunov Exponent. Together these improvements enhance the awareness of potential hazard cases

Using Interacting Multiple Model Filters to Indicate Program Risk

Technology development has increased exponentially. Program managers are pushed to accelerate development. There are many resources available to program managers that enable acceleration, such as: additional resources in the form of funding, people and technology. There are also negative impacts to acceleration, such as: inclusion, inexperience program managers, and communication. This research seeks to identify the limit to which a program or project can be accelerated before the program manager begins to accept an unacceptable amount of pre-determined risk. This research will utilize estimation algorithms used by sensor systems to estimate the current and future state of objects in space. The most common estimation algorithm used is the Kalman filter developed by Kalman (Bar-Shalom, Rong Li, & Kirubarajan, 2001). This research will examine the use of two Kalman filters in for the form of an Interacting Multiple Model (IMM) in order to predict the future state of the program. Traditional multiple model filters use Bayesian technique to adaptively switch between different motion models implemented in the filter structure (USA Patent No. 7030809, 2005). These logic designs rely upon a predefined Markov Switching Matrix (MSM). If the future state approaches a predetermined acceptable level of risk, the MSM will indicate to the program manager that the project has potentially reached a level of unacceptable risk

New improvements in the use of dependence measures for sensitivity analysis and screening

International audiencePhysical phenomena are commonly modeled by numerical simulators. Such codes can take as input a high number of uncertain parameters and it is important to identify their influences via a global sensitivity analysis (GSA). However, these codes can be time consuming which prevents a GSA based on the classical Sobol' indices, requiring too many simulations. This is especially true as the number of inputs is important. To address this limitation, we consider recent advances in dependence measures, focusing on the distance correlation and the Hilbert-Schmidt independence criterion (HSIC). Our objective is to study these indices and use them for a screening purpose. Numerical tests reveal some differences between dependence measures and classical Sobol' indices, and preliminary answers to "What sensitivity indices to what situation?" are derived. Then, two approaches are proposed to use the dependence measures for a screening purpose. The first one directly uses these indices with independence tests; asymptotic tests and their spectral extensions exist and are detailed. For a higher accuracy in presence of small samples, we propose a non-asymptotic version based on bootstrap sampling. The second approach is based on a linear model associating two simulations, which explains their output difference as a weighed sum of their input differences. From this, a bootstrap method is proposed for the selection of the influential inputs. We also propose a heuristic approach for the calibration of the HSIC Lasso method. Numerical experiments are performed and show the potential of these approaches for screening when many inputs are not influential

Comparing the structural uncertainty and uncertainty management in four common Land Use Cover Change (LUCC) model software packages

Research on the uncertainty of Land Use Cover Change (LUCC) models is still limited. Through this paper, we aim to globally characterize the structural uncertainty of four common software packages (CA_Markov, Dinamica EGO, Land Change Modeler, Metronamica) and analyse the options that they offer for uncertainty management. The models have been compared qualitatively, based on their structures and tools, and quantitatively, through a study case for the city of Cape Town. Results proved how each model conceptualised the modelled system in a different way, which led to different outputs. Statistical or automatic approaches did not provide higher repeatability or validation scores than user-driven approaches. The available options for uncertainty management vary depending on the model. Communication of uncertainties is poor across all models.Spanish GovernmentEuropean Commission INCERTIMAPS PGC2018-100770-B-100Spanish Ministry of Economy and Competitiveness and the European Social Fund [Ayudas para contratos predoctorales para la formacion de doctores 2014]University of Granada [Contratos Puente 2018]Spanish Ministry of Science and Innovation [Ayudas para contratos Juan de la Cierva-for-macion] 2019-FJC2019-040043University of Cape Town (Centre for Transport Studies