A Manifesto for the Equifinality Thesis.

This essay discusses some of the issues involved in the identification and predictions of hydrological models given some calibration data. The reasons for the incompleteness of traditional calibration methods are discussed. The argument is made that the potential for multiple acceptable models as representations of hydrological and other environmental systems (the equifinality thesis) should be given more serious consideration than hitherto. It proposes some techniques for an extended GLUE methodology to make it more rigorous and outlines some of the research issues still to be resolved

Understanding predictive uncertainty in hydrologic modeling: The challenge of identifying input and structural errors

Meaningful quantification of data and structural uncertainties in conceptual rainfall-runoff modeling is a major scientific and engineering challenge. This paper focuses on the total predictive uncertainty and its decomposition into input and structural components under different inference scenarios. Several Bayesian inference schemes are investigated, differing in the treatment of rainfall and structural uncertainties, and in the precision of the priors describing rainfall uncertainty. Compared with traditional lumped additive error approaches, the quantification of the total predictive uncertainty in the runoff is improved when rainfall and/or structural errors are characterized explicitly. However, the decomposition of the total uncertainty into individual sources is more challenging. In particular, poor identifiability may arise when the inference scheme represents rainfall and structural errors using separate probabilistic models. The inference becomes ill‐posed unless sufficiently precise prior knowledge of data uncertainty is supplied; this ill‐posedness can often be detected from the behavior of the Monte Carlo sampling algorithm. Moreover, the priors on the data quality must also be sufficiently accurate if the inference is to be reliable and support meaningful uncertainty decomposition. Our findings highlight the inherent limitations of inferring inaccurate hydrologic models using rainfall‐runoff data with large unknown errors. Bayesian total error analysis can overcome these problems using independent prior information. The need for deriving independent descriptions of the uncertainties in the input and output data is clearly demonstrated.Benjamin Renard, Dmitri Kavetski, George Kuczera, Mark Thyer, and Stewart W. Frank

Practical identifiability analysis of environmental models

Identifiability of a system model can be considered as the extent to which one can capture its parameter values from observational data and other prior knowledge of the system. Identifiability must be considered in context so that the objectives of the modelling must also be taken into account in its interpretation. A model may be identifiable for certain objective functions but not others; its identifiability may depend not just on the model structure but also on the level and type of noise, and may even not be identifiable when there is no noise on the observational data. Context also means that non-identifiability might not matter in some contexts, such as when representing pluralistic values among stakeholders, and may be very important in others, such as where it leads to intolerable uncertainties in model predictions. Uncertainty quantification of environmental systems is receiving increasing attention especially through the development of sophisticated methods, often statistically-based. This is partly driven by the desire of society and its decision makers to make more informed judgments as to how systems are better managed and associated resources efficiently allocated. Less attention seems to be given by modellers to understand the imperfections in their models and their implications. Practical methods of identifiability analysis can assist greatly here to assess if there is an identifiability problem so that one can proceed to decide if it matters, and if so how to go about modifying the model (transforming parameters, selecting specific data periods, changing model structure, using a more sophisticated objective function). A suite of relevant methods is available and the major useful ones are discussed here including sensitivity analysis, response surface methods, model emulation and the quantification of uncertainty. The paper also addresses various perspectives and concepts that warrant further development and use

Influence of error terms in Bayesian calibration of energy system models

Calibration represents a crucial step in the modelling process to obtain accurate simulation results and quantify uncertainties. We scrutinise the statistical Kennedy & O’Hagan framework, which quantifies different sources of uncertainty in the calibration process, including both model inputs and errors in the model. In specific, we evaluate the influence of error terms on the posterior predictions of calibrated model inputs. We do so by using a simulation model of a heat pump in cooling mode. While posterior values of many parameters concur with the expectations, some parameters appear not to be inferable. This is particularly true for parameters associated with model discrepancy, for which prior knowledge is typically scarce. We reveal the importance of assessing the identifiability of parameters by exploring the dependency of posteriors on the assigned prior knowledge. Analyses with random datasets show that results are overall consistent, which confirms the applicability and reliability of the framework

Development and application of a framework for model structure evaluation in environmental modelling

In a fast developing world with an ever rising population, the pressures on our natural environment are continuously increasing, causing problems such as floods, water- and air pollution, droughts,... Insight in the driving mechanisms causing these threats is essential in order to properly mitigate these problems. During the last decades, mathematical models became an essential part of scientific research to better understand and predict natural phenomena. Notwithstanding the diversity of currently existing models and modelling frameworks, the identification of the most appropriate model structure for a given problem remains a research challenge. The latter is the main focus of this dissertation, which aims to improve current practices of model structure comparison and evaluation. This is done by making individual model decisions more transparent and explicitly testable. A diagnostic framework, focusing on a flexible and open model structure definition and specifying the requirements for future model developments, is described. Methods for model structure evaluation are documented, implemented, extended and applied on both respirometric and hydrological models. For the specific case of lumped hydrological models, the unity between apparently different models is illustrated. A schematic representation of these model structures provides a more transparent communication tool, while meeting the requirements of the diagnostic approach

Characterization of process-oriented hydrologic model behavior with temporal sensitivity analysis for flash floods in Mediterranean catchments

This paper presents a detailed analysis of 10 flash flood events in the Mediterranean region using the distributed hydrological model MARINE. Characterizing catchment response during flash flood events may provide new and valuable insight into the dynamics involved for extreme catchment response and their dependency on physiographic properties and flood severity. The main objective of this study is to analyze flash-flood-dedicated hydrologic model sensitivity with a new approach in hydrology, allowing model outputs variance decomposition for temporal patterns of parameter sensitivity analysis. Such approaches enable ranking of uncertainty sources for nonlinear and nonmonotonic mappings with a low computational cost. Hydrologic model and sensitivity analysis are used as learning tools on a large flash flood dataset. With Nash performances above 0.73 on average for this extended set of 10 validation events, the five sensitive parameters of MARINE process-oriented distributed model are analyzed. This contribution shows that soil depth explains more than 80% of model output variance when most hydrographs are peaking. Moreover, the lateral subsurface transfer is responsible for 80% of model variance for some catchment-flood events’ hydrographs during slow-declining limbs. The unexplained variance of model output representing interactions between parameters reveals to be very low during modeled flood peaks and informs that model parsimonious parameterization is appropriate to tackle the problem of flash floods. Interactions observed after model initialization or rainfall intensity peaks incite to improve water partition representation between flow components and initialization itself. This paper gives a practical framework for application of this method to other models, landscapes and climatic conditions, potentially helping to improve processes understanding and representation

Optimal experimental design and its applications to biochemical engineering systems

This work is motivated by challenges in data-based modelling of complex systems due to limited information of sparse and noisy experimental data. Optimal experimental design (OED) techniques, which aim at devising necessary experiments to generate informative measurement data to facilitate model identification, have been investigated comprehensively.The limitations of existing experimental design approaches have been extensively discussed, based on which advanced experimental design methods and efficient numerical strategies have been developed for improved solutions.;Two case study biochemical systems have been used through the research investigation, one is an enzyme reaction system, the other one is a lab-scale enzymatic biodiesel production system. The main contributions of this PhD work can be summarised as follows:;Single objective experimental designs by considering one type of design factors, i.e. input intensity, measurement set selection, sampling profile design, respectively, has been formulated and numerical strategies to solve these optimisation problems have been described in detail. Implementations of these design methods to biochemical systems have demonstrated its efficiency in reducing parameter estimation errors.;A new OED strategy has been proposed to cope with OED problems including multiple design factors in one optimisation framework. An iterative two-layer design structure is developed. In the lower layer for observation design, the sampling profile and the measurement set selection are combined and formulated as a single integrated observation design problem, which is relaxed to a convex optimization problem that can be solved with a local method.;Thus the measurement set selection and the sampling profile can be determined simultaneously. In the upper layer for input design, the optimisation of input intensities is obtained through stochastic global searching. In this way, the multi-factor optimisation problem is solved through the integration of a stochastic method, for the upper layer, and a deterministic method, for the lower layer.;A new enzyme reaction model has been established which represents a typical class of enzymatic kinetically controlled synthesis process. This model contains important kinetic reaction features, moderate complexity, and complete model information. It can be used as a benchmark problem for development and comparison of OED algorithms. Systematic analysis has been performed in order to examine the system behaviours, and the dependence on model parameters, initial operation conditions.;Structural identifiability and practical identifiability of this system have been analysed and identifiable parameters determined. The design of experiment for the enzyme reactionsystem by considering different types of design variables have been investigated. The parameter estimation precision can be improved significantly by using the proposed OED techniques, compared to the non-designed condition.;The OED techniques are numerically investigated based on a lab-scale biodiesel production process with real experimental data through research collaboration with DTU in Denmark. The OED applications on this real system model allow to examine the effectiveness and efficiency of those new proposed OED methods. The measurement set selection and the sampling design of this system are developed which provide detailed instructions on how to improve experiments through OED.;Also, the sensitivity analysis and parameter identifiability analysis are conducted; and their impacts to experimental design are clearly identified.This work is motivated by challenges in data-based modelling of complex systems due to limited information of sparse and noisy experimental data. Optimal experimental design (OED) techniques, which aim at devising necessary experiments to generate informative measurement data to facilitate model identification, have been investigated comprehensively.The limitations of existing experimental design approaches have been extensively discussed, based on which advanced experimental design methods and efficient numerical strategies have been developed for improved solutions.;Two case study biochemical systems have been used through the research investigation, one is an enzyme reaction system, the other one is a lab-scale enzymatic biodiesel production system. The main contributions of this PhD work can be summarised as follows:;Single objective experimental designs by considering one type of design factors, i.e. input intensity, measurement set selection, sampling profile design, respectively, has been formulated and numerical strategies to solve these optimisation problems have been described in detail. Implementations of these design methods to biochemical systems have demonstrated its efficiency in reducing parameter estimation errors.;A new OED strategy has been proposed to cope with OED problems including multiple design factors in one optimisation framework. An iterative two-layer design structure is developed. In the lower layer for observation design, the sampling profile and the measurement set selection are combined and formulated as a single integrated observation design problem, which is relaxed to a convex optimization problem that can be solved with a local method.;Thus the measurement set selection and the sampling profile can be determined simultaneously. In the upper layer for input design, the optimisation of input intensities is obtained through stochastic global searching. In this way, the multi-factor optimisation problem is solved through the integration of a stochastic method, for the upper layer, and a deterministic method, for the lower layer.;A new enzyme reaction model has been established which represents a typical class of enzymatic kinetically controlled synthesis process. This model contains important kinetic reaction features, moderate complexity, and complete model information. It can be used as a benchmark problem for development and comparison of OED algorithms. Systematic analysis has been performed in order to examine the system behaviours, and the dependence on model parameters, initial operation conditions.;Structural identifiability and practical identifiability of this system have been analysed and identifiable parameters determined. The design of experiment for the enzyme reactionsystem by considering different types of design variables have been investigated. The parameter estimation precision can be improved significantly by using the proposed OED techniques, compared to the non-designed condition.;The OED techniques are numerically investigated based on a lab-scale biodiesel production process with real experimental data through research collaboration with DTU in Denmark. The OED applications on this real system model allow to examine the effectiveness and efficiency of those new proposed OED methods. The measurement set selection and the sampling design of this system are developed which provide detailed instructions on how to improve experiments through OED.;Also, the sensitivity analysis and parameter identifiability analysis are conducted; and their impacts to experimental design are clearly identified