A goodness of fit measure related to r² for model performance assessment

Checking the predictive worth of an environmental model inevitably includes a goodness of fit metric to quantify the degree of matching to recorded data, thereby giving a measure of model performance. Considerable analysis and discussion have taken place over fit indices in hydrology, but a neglected aspect is the degree of communicability to other disciplines. It is suggested that a fit index is best communicated to colleagues via reference to models giving unbiased predictions, because unbiased environmental models are a desirable goal across disciplines. That is, broad recognition of a fit index is aided if it simplifies in the unbiased case to a familiar and logical expression. This does not hold for the Nash–Sutcliffe Efficiency E which reduces to the somewhat awkward unbiased expression E = 2 – 1/r², where r² is the coefficient of determination. A new goodness of fit index V is proposed for model validation as V =  r²/(2-E), which simplifies to the easily communicated V = r4 in the unbiased case. The index is defined over the range 0 ≤ V ≤ 1, and it happens that V < E for larger values of E. Some synthetic and recorded data sets are used to illustrate characteristics of V in comparison to

A Global lake ecological observatory network (GLEON) for synthesising high-frequency sensor data for validation of deterministic ecological models

A Global Lake Ecological Observatory Network (GLEON; www.gleon.org) has formed to provide a coordinated response to the need for scientific understanding of lake processes, utilising technological advances available from autonomous sensors. The organisation embraces a grassroots approach to engage researchers from varying disciplines, sites spanning geographic and ecological gradients, and novel sensor and cyberinfrastructure to synthesise high-frequency lake data at scales ranging from local to global. The high-frequency data provide a platform to rigorously validate processbased ecological models because model simulation time steps are better aligned with sensor measurements than with lower-frequency, manual samples. Two case studies from Trout Bog, Wisconsin, USA, and Lake Rotoehu, North Island, New Zealand, are presented to demonstrate that in the past, ecological model outputs (e.g., temperature, chlorophyll) have been relatively poorly validated based on a limited number of directly comparable measurements, both in time and space. The case studies demonstrate some of the difficulties of mapping sensor measurements directly to model state variable outputs as well as the opportunities to use deviations between sensor measurements and model simulations to better inform process understanding. Well-validated ecological models provide a mechanism to extrapolate high-frequency sensor data in space and time, thereby potentially creating a fully 3-dimensional simulation of key variables of interest

Predictive Modeling the Free Hydraulic Jumps Pressure through Advanced Statistical Methods

Pressure fluctuations beneath hydraulic jumps potentially endanger the stability of stilling basins. This paper deals with the mathematical modeling of the results of laboratory-scale experiments to estimate the extreme pressures. Experiments were carried out on a smooth stilling basin underneath free hydraulic jumps downstream of an Ogee spillway. From the probability distribution of measured instantaneous pressures, pressures with different probabilities could be determined. It was verified that maximum pressure fluctuations, and the negative pressures, are located at the positions near the spillway toe. Also, minimum pressure fluctuations are located at the downstream of hydraulic jumps. It was possible to assess the cumulative curves of pressure data related to the characteristic points along the basin, and different Froude numbers. To benchmark the results, the dimensionless forms of statistical parameters include mean pressures (P*m), the standard deviations of pressure fluctuations (σ*X), pressures with different non-exceedance probabilities (P*k%), and the statistical coefficient of the probability distribution (Nk%) were assessed. It was found that an existing method can be used to interpret the present data, and pressure distribution in similar conditions, by using a new second-order fractional relationships for σ*X, and Nk%. The values of the Nk% coefficient indicated a single mean value for each probability

Towards Best Practice Framing of Uncertainty in Scientific Publications: A Review of Water Resources Research Abstracts

Uncertainty is recognized as a key issue in water resources research, amongst other sciences. Discussions of uncertainty typically focus on tools and techniques applied within an analysis, e.g. uncertainty quantification and model validation. But uncertainty is also addressed outside the analysis, in writing scientific publications. The language that authors use conveys their perspective of the role of uncertainty when interpreting a claim —what we call here “framing” the uncertainty. This article promotes awareness of uncertainty framing in four ways. 1) It proposes a typology of eighteen uncertainty frames, addressing five questions about uncertainty. 2) It describes the context in which uncertainty framing occurs. This is an interdisciplinary topic, involving philosophy of science, science studies, linguistics, rhetoric, and argumentation. 3) We analyze the use of uncertainty frames in a sample of 177 abstracts from the Water Resources Research journal in 2015. This helped develop and tentatively verify the typology, and provides a snapshot of current practice. 4) Provocative recommendations promote adjustments for a more influential, dynamic science. Current practice in uncertainty framing might be described as carefully-considered incremental science. In addition to uncertainty quantification and degree of belief (present in ~5% of abstracts), uncertainty is addressed by a combination of limiting scope, deferring to further work (~25%) and indicating evidence is sufficient (~40%) – or uncertainty is completely ignored (~8%). There is a need for public debate within our discipline to decide in what context different uncertainty frames are appropriate. Uncertainty framing cannot remain a hidden practice evaluated only by lone reviewers

A systematic review of the treatment of phosphorus in biogeochemical and ecological models

Aquatic biogeochemical and ecological models have become increasingly detailed, both in resolution and in the number of processes and components represented. It is timely to pause and take stock of these models. Do these models accurately reflect our current understanding of the biogeochemistry of aquatic systems? Is their predictive performance improving? Are they improving in the range of system properties and behaviours that can be predicted? Where there is variety in approaches or algorithms, can we demonstrate that one method is better than another and should be preferred? This review begins this process, focusing on how phosphorus cycles are represented in models of aquatic systems. A systematic review of 71 distinct published biogeochemical and ecological models of aquatic systems in 167 applications finds: Lake models and marine models are very similar, while river and catchment models differ. This appears largely to be due to different traditions within modelling communities rather than real differences in biogeochemical processes. River models tend to have simpler representations of phosphorus than lake or marine models. The performance of river models is usually quantitatively assessed using standard metrics (77% of publications include some performance metrics), while this is usually not the case for lake models (35% include some quantitative metric) or marine models (only 29%). Across all three domains, models are becoming more complex over time (Figure 1), but there is no clear evidence that this is improving the predictive performance of models. The appropriate degree of model complexity depends on a number of factors, including the resources and data available to support the model and the purpose for which the model is being developed. Simpler models can be more rigorously calibrated and evaluated and may have just as much predictive capacity, while physiologically-based models, which are usually more complex, may be more generalisable and better able to anticipate unexpected responses and emergent properties of the system. A range of phosphorus papers not included in current models is discussed, including the biogeochemistry of organic phosphorus, the biogeochemical implications of flow through sediments, and sediment drying and re-wetting. In some aquatic environments, these processes are likely to be important. It is concluded that a broader community toolkit of model algorithms is required

An evaluation framework for input variable selection algorithms for environmental data-driven models

Abstract not availableStefano Galelli, Greer B. Humphrey, Holger R. Maier, Andrea Castelletti, Graeme C. Dandy, Matthew S. Gibb