SW-ELM : A summation wavelet extreme learning machine algorithm with a priori initialization.

International audienceCombining neural networks and wavelet theory as an approximation or prediction models appears to be an effective solution in many applicative areas. However, when building such systems, one has to face parsimony problem, i.e., to look for a compromise between the complexity of the learning phase and accuracy performances. Following that, the aim of this paper is to propose a new structure of connectionist network, the Summation Wavelet Extreme Learning Machine (SW-ELM) that enables good accuracy and generalization performances, while limiting the learning time and reducing the impact of random initialization procedure. SW-ELM is based on Extreme Learning Machine (ELM) algorithm for fast batch learning, but with dual activation functions in the hidden layer nodes. This enhances dealing with non-linearity in an efficient manner. The initialization phase of wavelets (of hidden nodes) and neural network parameters (of input-hidden layer) is performed a priori, even before data are presented to the model. The whole proposition is illustrated and discussed by performing tests on three issues related to time-series application: an "input-output" approximation problem, a one-step ahead prediction problem, and a multi-steps ahead prediction problem. Performances of SW-ELM are benchmarked with ELM, Levenberg Marquardt algorithm for Single Layer Feed Forward Network (SLFN) and ELMAN network on six industrial data sets. Results show the significance of performances achieved by SW-ELM

From Statistical Physics to Algorithms in Deep Neural Systems

L'abstract è presente nell'allegato / the abstract is in the attachmen

CAN FISCAL POLICY EXPLAIN TECHNICAL INEFFICIENCY OF PRIVATISED FIRMS? A PARAMETRIC AND NONPARAMETRIC APPROACH

The massive interests of economic literature about the privatisation gave a notable impulse to the discussion about this theme in the pre and post privatisation firms performance. Basically in every case after privatisation the level of profit increases. Does this mean that privatisation is certainly able to increase efficiency? In this field a large part of the literature leave out the complex problem that public firms usually are subject to objectives and constraints that differently from private firms can affect the overall economic efficiency. Unfortunately many authors ignore the effects of taxation during the process of privatisation, but in real term there are significant tax issues that must be considered by public and private decision maker. In this paper we concentrate the attention on the efficiency measures with the purpose to identify and measure sources of successful performance that can be used in policy planning and allocation of resources. Several techniques to calculate these frontier functions have been used, some of them parametric, others non-parametric to empirically investigate the relationship between taxation on firm’s income and efficiency in the period pre and post-privatisation. In this work we use both econometric and mathematical programming approaches for measuring efficiency. The econometric tool provide maximum likelihood estimates of a stochastic production and cost functions to distinguish noise from inefficiency. Instead, the mathematical programming approaches are nonstochastic and they do not make strict assumptions on the functional form of production and the statistical properties of the data. The general results obtained from the 3 different tools (Stochastic Frontier, Data Envelopment Analysis and Neural Network) are consistent. In fact, we see that privatization enhanced efficiency in three out of four sample firms.Privatization, Fiscal policy, Data Envelopment Analysis, Stochastic Frontier, Neural Network

Seasonal rainfall prediction in Juba County, South Sudan using the feedforward neural networks

Historical rainfall data from 1997-2016 of Juba County, South Sudan were used in a Feed-Forward Neural Network (FFNN) model to make future predictions. Annual rainfall data were aggregated into three seasons MAMJ, JAS and OND and later trained for best forecasts for the period 2017-2034 using the Alyuda Forecaster XL software. Best training of the time series was attained, once the minimum error of the weight expressed as MSE or AE between the measured variable and predicted was achieved during gradient descent.  The results showed that for MAMJ and JAS months, the number forecasts were over 85% whereas this was between 60-80% for OND months. The Seasonal Kendal (SK) test on future rainfall forecasts as well as the Theil-Sen slope showed negative monotonic trends in the mean values till the end of 2034 of all three seasons with MAMJ, JAS at OND at 100, 150 and 80 mm respectively.  Rainfall forecast showed that the MAMJ months for the years 2019 to 2027 will be moderately wet except in April 2021 which will experience some severe wetness (due to intensive rainfall). Interdecadal severe drought with less than 60, 100 and 10 mm for MAMJ, JAS and OND respectively, is expected between 2028 to 2033 after almost two decades. The declining onset of MAMJ rains is expected to significantly affect the timing for land preparation and crop planting.&nbsp

Coupling satellite rainfall estimates and machine learning techniques for flow forecast: application to a large catchment in Southern Africa

Accurate river flow forecasting is an important asset for stream and reservoir management, being often translated into substantial social, economic and ecological gains. This contribution aims at coupling satellite rainfall estimates and machine learning techniques for daily flow forecast. Two lead times, of 30 and 60 days, were tested for flows at Victoria Falls, in Southern Africa. Six distinct machine learning models were compared with optimized ARMA models and benchmarked against a Fourierseries approximation. Results show that the addition of rainfall data generally enhanced the performances of machine learning models at 30 days but did not improve forecasts at 60 days. Also, it was shown that traditional ARMA models do not make use of the rainfall information. Regarding a lead time of 60 days, the machine learning models appear to bear great advantages compared to ARMA models which, for such a lead time have shown practically no forecast capabilities

An integrated approach to artificial neural network based process modelling

ANN technology exploded into the world of process modelling and control in the late 1980’s. The technology shows great promise and is seen as a technology that could provide models for most systems without the need to understand the fundamental behaviour or relationships among the process variables. Today, ANN applications have been applied successfully in a number of areas of process modelling and control, with the best-established applications being in the area of inferential measurements or soft sensors.Unfortunately, ‘the free lunch did not have much meat’. Overtime, people focused more on the true capabilities and power of ANN, the ability to model nonlinear relationships in data without having to define the form of the nonlinearity. However, there is often a tendency to merely plug in the data, turn the ANN training software on, and blindly accept the results. This is probably inevitable since, to date, there are no textbooks or scientific journal papers providing an integrated and systematic approach for ANN model development addressing pre-modelling, training and postmodelling stages. Therefore, addressing issues in those three phases of ANN model development is essential to support and to improve further applications of ANN technology in the area of process modelling and control.The model development issues in pre-modelling and training phases were addressed by reviewing current practice and existing techniques. For each issue, a novel method was proposed to improve the performance of ANN models. The new approaches were tested in a variety of benchmarking studies using artificial samples and coal property datasets from power station boilers.The research work in the post-modelling stage analysis which emphasises on taking the lid off black box model, proposes a novel technique to extract knowledge from the models and simultaneously obtain better understanding of the process. Postmodelling phase issues were addressed thoroughly including construction of prediction limit, sensitivity analysis and development of mathematical representation of the trained ANN model.Confidence intervals of the ANN models were analysed to construct the prediction boundary of the model. This analysis provides useful information related to interpolation and extrapolation of the model. It also highlighted how good the ANN models can be used for extrapolation purposes.An effort based on sensitivity analysis of hidden layers is also proposed to understand the behaviours of the ANN models. Using this technique, knowledge and information are retrieved from the developed models. A comparative study of the proposed techniques and the current practice was also presented.The last topic addressed in this thesis is knowledge extraction of ANN models using mathematical analysis of the hidden layers. The proposed analysis is applied in order to open the black box of the ANN models and is implemented to simulated and real historical plant data so that useful information from those data and better understanding of the process are obtained.All in all, efforts have been made in this thesis to minimise the use of abstract mathematical language and in some cases, simplify the language so that ANN modelling theory can be understood by a wider range of audience, especially the new practitioners in ANN based modelling and control. It is hoped that the insight provided in the dissertation will provide an integrated approach to pre-modelling, training and post-modelling stages of ANN models. This ‘new guideline’ of ANN model development is unique and beneficial, providing a systematic framework for the preparation, design, evaluation and implementation of ANN models in process modelling and control in particular and prediction / forecasting tool in general

Photovoltaic forecasting with artificil neural networks

Tese de mestrado em Engenharia da Energia e do Ambiente, apresentada à Universidade de Lisboa, através da Faculdade de Ciências, 2014São necessários esforços adicionais para promover a utilização de sistemas de produção de energia fotovoltaica conectados à rede como uma fonte fundamental de sistemas de energia elétrica, em níveis de penetrações mais elevados. Nesta tese é abordada a variabilidade da geração elétrica por sistemas fotovoltaicos e é desenvolvida com base na premissa de que o desempenho e a gestão de pequenas redes elétricas podem ser melhorados quando são utilizadas as informações de previsão de energia solar. É implementado um sistema de arquitetura de rede neuronal para o modelo auto-regressivo não-linear com variáveis exógenas (NARX) utilizando, não só, dados meteorológicos locais, mas também medições de sistemas fotovoltaicos circunjacentes. Diferentes configurações de entrada são otimizadas e comparadas para avaliar os efeitos no desempenho do modelo para previsão. A precisão das previsões revelou melhoria quando lhe são adicionadas informações de sistemas fotovoltaicos circunjacentes. Após ser selecionada a configuração de entrada da rede com o melhor desempenho, são testadas previsões com várias horas de antecedência e comparadas com o modelo da persistência, para verificar a precisão do modelo na previsão de diferentes horizontes temporais de curto prazo. O modelo NARX superou, claramente, o modelo de persistência, resultando num RMSE de 3,7% e de 4,5% aquando da antecipação das previsões de 5min e 2h30min, respetivamente.Additional efforts are required to promote the use of grid-connected photovoltaic (PV) systems as a fundamental source in electric power systems at the higher penetration levels. This thesis addresses the variability of PV electric generation and is built based on the premise that the performance and management of small electric networks can be improved when solar power forecast information is used. A neural network architecture system for the Nonlinear Autoregressive with eXogenous inputs (NARX) model is implemented using not only local meteorological data but also measurements of neighbouring PV systems. Input configurations are optimized and compared to assess the effects in the model forecasting performance. The added value of the information of the neighbouring PV systems has demonstrated to further improve the prediction accuracy. After selecting the input configuration with the best network performance, forecasts up to several hours in advance are tested to verify the model forecasting accuracy for different short-term time horizons and compared with the persistence model. The NARX model clearly outperformed the persistence model and yielded a 3.7% and a 4.5% RMSE for the anticipation of the 5min and 2h30 forecasts, respectively

Optimization of windspeed prediction using an artificial neural network compared with a genetic programming model

The precise prediction of windspeed is essential in order to improve and optimize wind power prediction. However, due to the sporadic and inherent complexity of weather parameters, the prediction of windspeed data using different patterns is difficult. Machine learning (ML) is a powerful tool to deal with uncertainty and has been widely discussed and applied in renewable energy forecasting. In this chapter, the authors present and compare an artificial neural network (ANN) and genetic programming (GP) model as a tool to predict windspeed of 15 locations in Queensland, Australia. After performing feature selection using neighborhood component analysis (NCA) from 11 different metrological parameters, seven of the most important predictor variables were chosen for 85 Queensland locations, 60 of which were used for training the model, 10 locations for model validation, and 15 locations for the model testing. For all 15 target sites, the testing performance of ANN was significantly superior to the GP model