Neural networks and non-parametric methods for improving real-time flood forecasting through conceptual hydrological models

International audienceTime-series analysis techniques for improving the real-time flood forecasts issued by a deterministic lumped rainfall-runoff model are presented. Such techniques are applied for forecasting the short-term future rainfall to be used as real-time input in a rainfall-runoff model and for updating the discharge predictions provided by the model. Along with traditional linear stochastic models, both stationary (ARMA) and non-stationary (ARIMA), the application of non-linear time-series models is proposed such as Artificial Neural Networks (ANNs) and the ?nearest-neighbours' method, which is a non-parametric regression methodology. For both rainfall forecasting and discharge updating, the implementation of each time-series technique is investigated and the forecasting schemes which perform best are identified. The performances of the models are then compared and the improvement in the efficiency of the discharge forecasts achievable is demonstrated when i) short-term rainfall forecasting is performed, ii) the discharge is updated and iii) both rainfall forecasting and discharge updating are performed in cascade. The proposed techniques, especially those based on ANNs, allow a remarkable improvement in the discharge forecast, compared with the use of heuristic rainfall prediction approaches or the not-updated discharge forecasts given by the deterministic rainfall-runoff model alone

A Hybrid Approach Based on Variational Mode Decomposition for Analyzing and Predicting Urban Travel Speed

Predicting travel speeds on urban road networks is a challenging subject due to its uncertainty stemming from travel demand, geometric condition, traffic signals, and other exogenous factors. This uncertainty appears as nonlinearity, nonstationarity, and volatility in traffic data, and it also creates a spatiotemporal heterogeneity of link travel speed by interacting with neighbor links. In this study, we propose a hybrid model using variational mode decomposition (VMD) to investigate and mitigate the uncertainty of urban travel speeds. The VMD allows the travel speed data to be divided into orthogonal and oscillatory sub-signals, called modes. The regular components are extracted as the low-frequency modes, and the irregular components presenting uncertainty are transformed into a combination of modes, which is more predictable than the original uncertainty. For the prediction, the VMD decomposes the travel speed data into modes, and these modes are predicted and summed to represent the predicted travel speed. The evaluation results on urban road networks show that, the proposed hybrid model outperforms the benchmark models both in the congested and in the overall conditions. The improvement in performance increases significantly over specific link-days, which generally are hard to predict. To explain the significant variance of the prediction performance according to each link and each day, the correlation analysis between the properties of modes and the performance of the model are conducted. The results on correlation analysis show that the more variance of nondaily pattern is explained through the modes, the easier it was to predict the speed. Based on the results, discussions on the interpretation on the correlation analysis and future research are presented. Document type: Articl

AVERAGE MONTHLY RAINFALL FORECAST IN ROMANIA BY USING k-NEAREST NEIGHBORS REGRESSION

The discovery of the best strategies for achieving future values forecast of a time series represents a permanent concern in time series analysis, highly motivated from a theoretical point of view, but especially from a practical point of view.In the context of the explosive growth of machine learning techniques, their usein time series forecast is a natural step to find modern alternatives to overcome existing limitations of traditional techniques. Although it is a relatively a simple method of learning, (k-nearest neighbor) regression seems to be a good competitor to traditional methods.The purpose of this paper is to describe how to use this method for forecasting time series and for achieving Monthly Average Rainfall (AMR) forecast in Romania

Understanding Dynamic Spatio-Temporal Contexts in Long Short-Term Memory for Road Traffic Speed Prediction

Reliable traffic flow prediction is crucial to creating intelligent transportation systems. Many big-data-based prediction approaches have been developed but they do not reflect complicated dynamic interactions between roads considering time and location. In this study, we propose a dynamically localised long short-term memory (LSTM) model that involves both spatial and temporal dependence between roads. To do so, we use a localised dynamic spatial weight matrix along with its dynamic variation. Moreover, the LSTM model can deal with sequential data with long dependency as well as complex non-linear features. Empirical results indicated superior prediction performances of the proposed model compared to two different baseline methods.Comment: 10pages, 2 tables, 4 figures, 2017 KDD Cu

Travel time prediction for congested freeways with a dynamic linear model

Accurate prediction of travel time is an essential feature to support Intelligent Transportation Systems (ITS). The non-linearity of traffic states, however, makes this prediction a challenging task. Here we propose to use dynamic linear models (DLMs) to approximate the non-linear traffic states. Unlike a static linear regression model, the DLMs assume that their parameters are changing across time. We design a DLM with model parameters defined at each time unit to describe the spatio-temporal characteristics of time-series traffic data. Based on our DLM and its model parameters analytically trained using historical data, we suggest an optimal linear predictor in the minimum mean square error (MMSE) sense. We compare our prediction accuracy of travel time for freeways in California (I210-E and I5-S) under highly congested traffic conditions with those of other methods: the instantaneous travel time, k-nearest neighbor, support vector regression, and artificial neural network. We show significant improvements in the accuracy, especially for short-term prediction.Comment: in IEEE Transactions on Intelligent Transportation Systems, 202

Satellite Data and Supervised Learning to Prevent Impact of Drought on Crop Production: Meteorological Drought

Reiterated and extreme weather events pose challenges for the agricultural sector. The convergence of remote sensing and supervised learning (SL) can generate solutions for the problems arising from climate change. SL methods build from a training set a function that maps a set of variables to an output. This function can be used to predict new examples. Because they are nonparametric, these methods can mine large quantities of satellite data to capture the relationship between climate variables and crops, or successfully replace autoregressive integrated moving average (ARIMA) models to forecast the weather. Agricultural indices (AIs) reflecting the soil water conditions that influence crop conditions are costly to monitor in terms of time and resources. So, under certain circumstances, meteorological indices can be used as substitutes for AIs. We discuss meteorological indexes and review SL approaches that are suitable for predicting drought based on historical satellite data. We also include some illustrative case studies. Finally, we will survey rainfall products existing at the web and some alternatives to process the data: from high-performance computing systems able to process terabyte-scale datasets to open source software enabling the use of personal computers

Forecasting Models for Integration of Large-Scale Renewable Energy Generation to Electric Power Systems

Amid growing concerns about climate change and non-renewable energy sources deple¬tion, vari¬able renewable energy sources (VRESs) are considered as a feasible substitute for conventional environment-polluting fossil fuel-based power plants. Furthermore, the transition towards clean power systems requires additional transmission capacity. Dynamic thermal line rating (DTLR) is being considered as a potential solution to enhance the current transmission line capacity and omit/postpone transmission system expansion planning, while DTLR is highly dependent on weather variations. With increasing the accommodation of VRESs and application of DTLR, fluctuations and variations thereof impose severe and unprecedented challenges on power systems operation. Therefore, short-term forecasting of large-scale VERSs and DTLR play a crucial role in the electric power system op¬eration problems. To this end, this thesis devotes on developing forecasting models for two large-scale VRESs types (i.e., wind and tidal) and DTLR. Deterministic prediction can be employed for a variety of power system operation problems solved by deterministic optimization. Also, the outcomes of deterministic prediction can be employed for conditional probabilistic prediction, which can be used for modeling uncertainty, used in power system operation problems with robust optimization, chance-constrained optimization, etc. By virtue of the importance of deterministic prediction, deterministic prediction models are developed. Prevalently, time-frequency decomposition approaches are adapted to decompose the wind power time series (TS) into several less non-stationary and non-linear components, which can be predicted more precisely. However, in addition to non-stationarity and nonlinearity, wind power TS demonstrates chaotic characteristics, which reduces the predictability of the wind power TS. In this regard, a wind power generation prediction model based on considering the chaosity of the wind power generation TS is addressed. The model consists of a novel TS decomposition approach, named multi-scale singular spectrum analysis (MSSSA), and least squares support vector machines (LSSVMs). Furthermore, deterministic tidal TS prediction model is developed. In the proposed prediction model, a variant of empirical mode decomposition (EMD), which alleviates the issues associated with EMD. To further improve the prediction accuracy, the impact of different components of wind power TS with different frequencies (scales) in the spatiotemporal modeling of the wind farm is assessed. Consequently, a multiscale spatiotemporal wind power prediction is developed, using information theory-based feature selection, wavelet decomposition, and LSSVM. Power system operation problems with robust optimization and interval optimization require prediction intervals (PIs) to model the uncertainty of renewables. The advanced PI models are mainly based on non-differentiable and non-convex cost functions, which make the use of heuristic optimization for tuning a large number of unknown parameters of the prediction models inevitable. However, heuristic optimization suffers from several issues (e.g., being trapped in local optima, irreproducibility, etc.). To this end, a new wind power PI (WPPI) model, based on a bi-level optimization structure, is put forward. In the proposed WPPI, the main unknown parameters of the prediction model are globally tuned based on optimizing a convex and differentiable cost function. In line with solving the non-differentiability and non-convexity of PI formulation, an asymmetrically adaptive quantile regression (AAQR) which benefits from a linear formulation is proposed for tidal uncertainty modeling. In the prevalent QR-based PI models, for a specified reliability level, the probabilities of the quantiles are selected symmetrically with respect the median probability. However, it is found that asymmetrical and adaptive selection of quantiles with respect to median can provide more efficient PIs. To make the formulation of AAQR linear, extreme learning machine (ELM) is adapted as the prediction engine. Prevalently, the parameters of activation functions in ELM are selected randomly; while different sets of random values might result in dissimilar prediction accuracy. To this end, a heuristic optimization is devised to tune the parameters of the activation functions. Also, to enhance the accuracy of probabilistic DTLR, consideration of latent variables in DTLR prediction is assessed. It is observed that convective cooling rate can provide informative features for DTLR prediction. Also, to address the high dimensional feature space in DTLR, a DTR prediction based on deep learning and consideration of latent variables is put forward. Numerical results of this thesis are provided based on realistic data. The simulations confirm the superiority of the proposed models in comparison to traditional benchmark models, as well as the state-of-the-art models

Robust data cleaning procedure for large scale medium voltage distribution networks feeders

Relatively little attention has been given to the short-term load forecasting problem of primary substations mainly because load forecasts were not essential to secure the operation of passive distribution networks. With the increasing uptake of intermittent generations, distribution networks are becoming active since power flows can change direction in a somewhat volatile fashion. The volatility of power flows introduces operational constraints on voltage control, system fault levels, thermal constraints, systems losses and high reverse power flows. Today, greater observability of the networks is essential to maintain a safe overall system and to maximise the utilisation of existing assets. Hence, to identify and anticipate for any forthcoming critical operational conditions, networks operators are compelled to broaden their visibility of the networks to time horizons that include not only real-time information but also hour-ahead and day-ahead forecasts. With this change in paradigm, progressively, large scales of short-term load forecasters is integrated as an essential component of distribution networks' control and planning tools. The data acquisition of large scale real-world data is prone to errors; anomalies in data sets can lead to erroneous forecasting outcomes. Hence, data cleansing is an essential first step in data-driven learning techniques. Data cleansing is a labour-intensive and time-consuming task for the following reasons: 1) to select a suitable cleansing method is not trivial 2) to generalise or automate a cleansing procedure is challenging, 3) there is a risk to introduce new errors in the data. This thesis attempts to maximise the performance of large scale forecasting models by addressing the quality of the modelling data. Thus, the objectives of this research are to identify the bad data quality causes, design an automatic data cleansing procedure suitable for large scale distribution network datasets and, to propose a rigorous framework for modelling MV distribution network feeders time series with deep learning architecture. The thesis discusses in detail the challenges in handling and modelling real-world distribution feeders time series. It also discusses a robust technique to detect outliers in the presence of level-shifts, and suitable missing values imputation techniques. All the concepts have been demonstrated on large real-world distribution network data.Open Acces