Reply to comment by K. Beven et al. on "Pursuing the method of multiple working hypotheses for hydrological modelling"

Martyn P. Clark, Dmitri Kavetski, and Fabrizio Fenici

Evaluating post-processing approaches for monthly and seasonal streamflow forecasts

Streamflow forecasting is prone to substantial uncertainty due to errors in meteorological forecasts, hydrological model structure, and parameterization, as well as in the observed rainfall and streamflow data used to calibrate the models. Statistical streamflow post-processing is an important technique available to improve the probabilistic properties of the forecasts. This study evaluates post-processing approaches based on three transformations – logarithmic (Log), log-sinh (Log-Sinh), and Box–Cox with λ=0.2 (BC0.2) – and identifies the best-performing scheme for post-processing monthly and seasonal (3-months-ahead) streamflow forecasts, such as those produced by the Australian Bureau of Meteorology. Using the Bureau's operational dynamic streamflow forecasting system, we carry out comprehensive analysis of the three post-processing schemes across 300 Australian catchments with a wide range of hydro-climatic conditions. Forecast verification is assessed using reliability and sharpness metrics, as well as the Continuous Ranked Probability Skill Score (CRPSS). Results show that the uncorrected forecasts (i.e. without post-processing) are unreliable at half of the catchments. Post-processing of forecasts substantially improves reliability, with more than 90 % of forecasts classified as reliable. In terms of sharpness, the BC0.2 scheme substantially outperforms the Log and Log-Sinh schemes. Overall, the BC0.2 scheme achieves reliable and sharper-than-climatology forecasts at a larger number of catchments than the Log and Log-Sinh schemes. The improvements in forecast reliability and sharpness achieved using the BC0.2 post-processing scheme will help water managers and users of the forecasting service make better-informed decisions in planning and management of water resources.Fitsum Woldemeskel, David McInerney, Julien Lerat, Mark Thyer, Dmitri Kavetski, Daehyok Shin, Narendra Tuteja and George Kuczer

High-quality probabilistic predictions for existing hydrological models with common objective functions

Conference theme 'Digital Water.'Probabilistic predictions describe the uncertainty in modelled streamflow, which is a critical input for many environmental modelling applications. A residual error model typically produces the probabilistic predictions in tandem with a hydrological model that predicts the deterministic streamflow. However, many objective functions that are commonly used to calibrate the parameters of the hydrological model make (implicit) assumptions about the errors that do not match the properties (e.g. of heteroscedasticity and skewness) of those errors. The consequence of these assumptions is often low-quality probabilistic predictions of errors, which reduces the practical utility of probabilistic modelling. Our study has two aims: Firstly, to evaluate the impact of objective function inconsistency on the quality of probabilistic predictions; Secondly, to demonstrate how a simple enhancement to a residual error model can rectify the issues identified with inconsistent objective functions in Aim 1, and thereby improve probabilistic predictions in a wide range of scenarios. Our findings show that the enhanced error model enables high-quality probabilistic predictions to be obtained for a range of catchments and objective functions, without requiring any changes to the hydrological modelling or calibration process. This advance has practical benefits that are aimed at increasing the uptake of probabilistic predictions in real-world applications, in that the methods are applicable to existing hydrological models that are already calibrated, simple to implement, easy to use and fast. Finally, these methods are available as an open-source R-shiny application and an R-package function.Jason Hunter, Mark Thyer, David McInerney, Dmitri Kavetsk

High-quality probabilistic predictions for existing hydrological models with common objective functions

Session HS1.3.1Probabilistic predictions describe the uncertainty in modelled streamflow, which is a critical input for many environmental modelling applications. A residual error model typically produces the probabilistic predictions in tandem with a hydrological model that predicts the deterministic streamflow. However, many objective functions that are commonly used to calibrate the parameters of the hydrological model make (implicit) assumptions about the errors that do not match the properties (e.g. of heteroscedasticity and skewness) of those errors. The consequence of these assumptions is often low-quality probabilistic predictions of errors, which reduces the practical utility of probabilistic modelling. Our study has two aims: 1. Evaluate the impact of objective function inconsistency on the quality of probabilistic predictions; 2. To demonstrate how a simple enhancement to a residual error model can rectify the issues identified with inconsistent objective functions in Aim 1, and thereby improve probabilistic predictions in a wide range of scenarios. Our findings show that the enhanced error model enables high-quality probabilistic predictions to be obtained for a range of catchments and objective functions, without requiring any changes to the hydrological modelling or calibration process. This advance has practical benefits that are aimed at increasing the uptake of probabilistic predictions in real-world applications, in that the methods are applicable to existing hydrological models that are already calibrated, simple to implement, easy to use and fast. Finally, these methods are available as an open-source R-shiny application and an R-package function.Mark Thyer, Jason Hunter, David McInerney, and Dmitri Kavetsk

Practical approaches to produce high-quality probabilistic predictions and improve risk-based design making

Conference theme 'Digital Water.'Probabilistic predictions provide crucial information regarding the uncertainty of hydrological predictions, which are a key input for risk-based decision-making. High-quality probabilistic predictions provide reliable estimates of water resource system risks – avoiding a false sense of security. However, probabilistic predictions are not widely used in hydrological modelling applications because they are perceived to be difficult to construct and interpret. We present a software tool that provides an easy-to-use and simple approach to produce high-quality probabilistic streamflow predictions. The approach integrates the recommendations from multiple research papers over multiple years to provide guidance on selection of robust descriptions of uncertainty (residual error models) for a wide range of hydrological applications. This guidance includes the choice of transformation to handle common features of residual errors (heteroscedasticity, skewness, persistence) and techniques that handles a wide range of common objective functions. A case study illustrating the practical benefits of uncertainty analysis for risk-based decision- making is provided. The case study evaluates fish health in two catchments (Mt. McKenzie and Upper Jacobs) in Barossa Valley, South Australia. The streamflow predictions of environmental flow metrics are combined with a simplified environmental response model to estimate fish health. The outcomes obtained using deterministic streamflow predictions are contrasted to the outcomes obtained from probabilistic predictions. In general, probabilistic predictions provide greater confidence in the predictions of fish health because the uncertainty ranges recognise the differences at the two sites between the quality of hydrological predictions. The uncertainty ranges were generally high, in the range 40-60% (Mt McKenzie) or 4-20% (Upper Jacobs) for predictions of the frequency of years with poor (or worse) fish health. This analysis provides a richer source of information for risk averse decision-makers than the single values provided by deterministic predictions.Mark Thyer, David McInerney, Dmitri Kavetski, Jason Hunte

Pitfalls and improvements in the joint inference of heteroscedasticity and autocorrelation in hydrological model calibration

Residual errors of hydrological models are usually both heteroscedastic and autocorrelated. However, only a few studies have attempted to explicitly include these two statistical properties into the residual error model and jointly infer them with the hydrological model parameters. This technical note shows that applying autoregressive error models to raw heteroscedastic residuals, as done in some recent studies, can lead to unstable error models with poor predictive performance. This instability can be avoided by applying the autoregressive process to standardized residuals. The theoretical analysis is supported by empirical findings in three hydrologically distinct catchments. The case studies also highlight strong interactions between the parameters of autoregressive residual error models and the water balance parameters of the hydrological model. ©2013. American Geophysical Union. All Rights Reserved.Guillaume Evin, Dmitri Kavetski, Mark Thyer, and George Kuczer

Probabilistic streamflow prediction and uncertainty estimation in ephemeral catchments

Conference theme 'Digital Water.'Probabilistic streamflow predictions at the daily scale are of major practical interest for environmental management and planning, including risk assessment as part of reservoir management operations. Ephemeral catchments, where streamflow is frequently zero or negligible, pose particularly stark challenges in this context, due to asymmetry of the error distribution and the discrete (rather than continuous) nature of zero flows. In this work, our focus is on two practical error modelling approaches where predictive uncertainty is approximated by a (transformed) Gaussian error model. The first approach, termed "pragmatic", does not distinguish between zero and positive flows during calibration, but sets negative flows to zero when making predictions. The second approach, termed "explicit", applies a "censored" Gaussian assumption in both calibration and prediction. We report a comparison of these two approaches over 74 Australian catchments with diverse hydroclimatology, using multiple performance metrics. The performance of the approaches depended on the catchment type as follows: (1) "mid-ephemeral" catchments, where 5-50% of days have zero flows, are best modelled using the "explicit" approach in combination with the Box-Cox streamflow transformation with a power parameter of 0.2; (2) "low-ephemeral" catchments, with fewer than 5% zero flow days, can be modelled using the pragmatic approach with (relatively) little loss of predictive performance; (3) "high-ephemeral" catchments, with more than 50% zero flow days, prove challenging to both approaches, and require more specialised techniques. The findings provide practical guidance towards improving probabilistic streamflow predictions in ephemeral catchments. Previous chapter Next chapterDmitri Kavetski, David McInerney, Mark Thyer, Julien Lerat and George Kuczer

The MuTHRE Model for High Quality Sub-seasonal Streamflow Forecasts

Conference theme 'Digital Water.'Sub-seasonal streamflow forecasts, with lead times up to 30 days, can provide valuable information for water management, including reservoir operation to meet environmental flow, irrigation demands, and managing flood protection storage. A key aim is to produce “seamless” probabilistic forecasts, with high quality performance across the full range of lead times (1-30 days) and time scales (daily to monthly). This paper demonstrates that the Multi-Temporal Hydrological Residual Error (MuTHRE) model can address the challenge of “seamless” sub-seasonal forecasting. The MuTHRE model is designed to capture key features of hydrological errors, namely seasonality, dynamic biases due to hydrological non-stationarity, and extreme errors poorly represented by the common Gaussian distribution. The MuTHRE model is evaluated comprehensively over 11 catchments in the MurrayDarling Basin using multiple performance metrics, across a range of lead times, months and years, and at daily and monthly time scales. It is shown to provide “high” improvements, in terms of reliability for short lead times (up to 10 days), in dry months, and dry years. Forecast performance also improved in terms of sharpness. Importantly, improvements are consistent across multiple time scales (daily and monthly). This study highlights the benefits of modelling multiple temporal characteristics of hydrological errors, and demonstrates the power of the MuTHRE model for producing seamless sub-seasonal streamflow forecasts that can be utilized for a wide range of applications.David McInerney, Mark Thyer, Dmitri Kavetski, Richard Laugesen, Narendra Tuteja, and George Kuczer

A strategy for diagnosing and interpreting hydrological model nonstationarity

Article first published online: 24 JUN 2014This paper presents a strategy for diagnosing and interpreting hydrological nonstationarity, aiming to improve hydrological models and their predictive ability under changing hydroclimatic conditions. The strategy consists of four elements: (i) detecting potential systematic errors in the calibration data; (ii) hypothesizing a set of “nonstationary” parameterizations of existing hydrological model structures, where one or more parameters vary in time as functions of selected covariates; (iii) trialing alternative stationary model structures to assess whether parameter nonstationarity can be reduced by modifying the model structure; and (iv) selecting one or more models for prediction. The Scott Creek catchment in South Australia and the lumped hydrological model GR4J are used to illustrate the strategy. Streamflow predictions improve significantly when the GR4J parameter describing the maximum capacity of the production store is allowed to vary in time as a combined function of: (i) an annual sinusoid; (ii) the previous 365 day rainfall and potential evapotranspiration; and (iii) a linear trend. This improvement provides strong evidence of model nonstationarity. Based on a range of hydrologically oriented diagnostics such as flow-duration curves, the GR4J model structure was modified by introducing an additional calibration parameter that controls recession behavior and by making actual evapotranspiration dependent only on catchment storage. Model comparison using an information-theoretic measure (the Akaike Information Criterion) and several hydrologically oriented diagnostics shows that the GR4J modifications clearly improve predictive performance in Scott Creek catchment. Based on a comparison of 22 versions of GR4J with different representations of nonstationarity and other modifications, the model selection approach applied in the exploratory period (used for parameter estimation) correctly identifies models that perform well in a much drier independent confirmatory period.Seth Westra, Mark Thyer, Michael Leonard, Dmitri Kavetski, and Martin Lamber

State updating and calibration period selection to improve dynamic monthly streamflow forecasts for an environmental flow management application

Monthly to seasonal streamflow forecasts provide useful information for a range of water resource management and planning applications. This work focuses on improving such forecasts by considering the following two aspects: (1) state updating to force the models to match observations from the start of the forecast period, and (2) selection of a shorter calibration period that is more representative of the forecast period, compared to a longer calibration period traditionally used. The analysis is undertaken in the context of using streamflow forecasts for environmental flow water management of an open channel drainage network in southern Australia. Forecasts of monthly streamflow are obtained using a conceptual rainfall–runoff model combined with a post-processor error model for uncertainty analysis. This model set-up is applied to two catchments, one with stronger evidence of non-stationarity than the other. A range of metrics are used to assess different aspects of predictive performance, including reliability, sharpness, bias and accuracy. The results indicate that, for most scenarios and metrics, state updating improves predictive performance for both observed rainfall and forecast rainfall sources. Using the shorter calibration period also improves predictive performance, particularly for the catchment with stronger evidence of non-stationarity. The results highlight that a traditional approach of using a long calibration period can degrade predictive performance when there is evidence of non-stationarity. The techniques presented can form the basis for operational monthly streamflow forecasting systems and provide support for environmental decision-making.Matthew S. Gibbs, David McInerney, Greer Humphrey, Mark A. Thyer, Holger R. Maier, Graeme C. Dandy and Dmitri Kavetsk