International audienceShort-term forecasting of wind energy production up to 2-3 days ahead is recognized as a major contribution for reliable large-scale wind power integration. Increasing the value of wind generation through the improvement of prediction systems performance is recognised as one of the priorities in wind energy research needs for the coming years. This paper aims to evaluate Data Mining type of models for wind power forecasting. Models that are examined include neural networks, support vector machines, the recently proposed regression trees approach, and others. Evaluation results are presented for several real wind farms

Fugon, Lionel

Juban, Jérémie

Kariniotakis, Georges

HAL-MINES ParisTech

Data mining for wind power forecastingLionel Fugon, Je´re´mie Juban, Georges KariniotakisTo cite this version:Lionel Fugon, Je´re´mie Juban, Georges Kariniotakis. Data mining for wind power forecasting.European Wind Energy Conference & Exhibition EWEC 2008, Mar 2008, Brussels, Belgium.EWEC, 6 p., 2008. <hal-00506101>HAL Id: hal-00506101https://hal-mines-paristech.archives-ouvertes.fr/hal-00506101Submitted on 27 Jul 2010HAL is a multi-disciplinary open accessarchive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come fromteaching and research institutions in France orabroad, or from public or private research centers.L’archive ouverte pluridisciplinaire HAL, estdestine´e au de´poˆt et a` la diffusion de documentsscientifiques de niveau recherche, publie´s ou non,e´manant des e´tablissements d’enseignement et derecherche franc¸ais ou e´trangers, des laboratoirespublics ou prive´s.European Wind Energy Conference - Brussels, Belgium, April 2008Data mining for wind power forecastingLionel Fugon , Je´re´mie Juban and George KariniotakisE´cole des Mines de ParisB.P.207, F-06904 Sophia-Antipolis, Francejeremie.juban@ensmp.fr; georges.kariniotakis@ensmp.frAbstractShort-term forecasting of wind energy produc-tion up to 2-3 days ahead is recognized as amajor contribution for reliable large-scale windpower integration. Increasing the value of windgeneration through the improvement of predic-tion systems performance is recognised as oneof the priorities in wind energy research needsfor the coming years. This paper aims to evalu-ate Data Mining type of models for wind powerforecasting. Models that are examined includeneural networks, support vector machines, therecently proposed regression trees approach, andothers. Evaluation results are presented for sev-eral real wind farms.1 IntroductionWind power has been undergoing a rapid devel-opment in recent years. Several countries havereached already a high level of installed windpower capacity, such as Germany, Spain and,Denmark, while others follow with fast ratesof development. Such large-scale integration ofwind power is challenging in terms of power sys-tem management. Indeed, wind is a variable re-source that is difficult to predict. As an example,traditionally, additional reserves are allocated tomanage this uncertainty.This increases the overall cost of the producedenergy and limits the benefits of using such arenewable energy resource. A way of reducingthe uncertainty associated to wind power pro-duction is to use forecasting tools. Developmentof such tools has been ongoing for more than15 years [1]. These tools are multi-step aheadforecasting models that provide information forseveral horizons ahead. In the same way, thecontinuous improvement of computers and theconstant increase in databases capacity permit-ted the development of a new scientific investi-gation field called Data Mining. Data Mininghas been defined as ”the nontrivial extractionof implicit, previously unknown, and potentiallyuseful information from data (Fayyad, 1996)”.Several algorithmic techniques issued from DataMining have already been adapted to the windpower forecasting [2] to provide a single expectedvalue for each forecast horizon, called determin-istic, spot or point forecast.The paper initially introduces the different al-gorithms used. Then the real-world data usedto evaluate the models are presented. They arefrom French wind farms located in different ter-rain complexity and climatic conditions. Finallyan evaluation and comparison of the models per-formance for each wind farm follows. The paperends with some conclusions and remarks.2 Data Mining Models UsedData Mining encompasses different algorithmsfrom many scientific fields (statistics, artifi-cial intelligence...) for building models (super-vised methods) : Y = φ(X) + ǫ where Y isthe variable to predict or explain and X =(X1, X2, ..., Xn) represents the vector of the n-explanatory variables.ǫ is the error of the model.The aim is to approximate the function φ andminimize the error. For that purpose, we usedboth a linear and a non-linear approach.2.1 Linear ModelsTwo versions of linear regression models havebeen considered: one base version, which is usedas a simple reference model and a second onethat includes interactions. Interactions basicallyconsist in combining the input variables to cre-ate extra variables used in the same linear set-tings. This has the advantage of consideringnon-linearities while keeping a simple linear set-ting. However, this can enlarge considerably thedimension of the problem.The linear model without interaction can bepresented by the following formula:1European Wind Energy Conference - Brussels, Belgium, April 2008Y = Xβ + ǫ (1)where β = (β1, β2, ..., βn) represents the vec-tor of linear regression coefficients to estimate.If we take account of different hypothesis on ǫ(such as i.i.d.), we approximate the β-vector byleast square minimization on the learning set:n∑i=1yi − β0 − β1xi1 − β2xi2 − ...− βpxip= ‖y −Xβ‖2(2)For the linear regression with interaction,cross products between variables are added tothe input variables. Then, the same method isapplied on this augmented input set.2.2 Non-Linear ModelsIn this part, several non-linear models for windpower forecasting are considered. Such mod-els are better suited to account for the non-linearity of the wind to power conversion. First,neural networks are considered. Then, we con-sider models based on classification and regres-sion trees with a simple bagging version and amore advanced version called random forest. Fi-nally, we present a last approach based on sup-port vector machines.2.2.1 Neural NetworksA neural network is an ensemble of neurons (ornodes) connected along several levels called layerinspired by the structure of the human brain. Itis generally composed of three layers of neurons,namely an input layer corresponding to explana-tory variables, an output layer that provides aresponse and one or several hidden layers. Neu-rons of a same layer are never connected betweenthemselves.Each neuron is affected a weight and passessignals based on a specific transfer function. Themost used transfer function is the sigmoid, whichreceives as input a weighted linear combinationof the output of the neurons of the previouslayer:s(θ) =11 + exp(−θ)(3)with θ = ΣiβiXi.The learning step of the neural network isbased on a back-propagation algorithm. Theprinciple is to adjust the different neuron weightsprogressively by back-propagating the errorfrom the output layer to the input layer. Themain advantage of the neural network is its ca-pacity to model complex structures and accountfor non-linear relations between the explanatoryvariables and the output. Although, one of themain drawbacks, is that the final performance ofa neural network is very sensitive to the designof the network. The choice of the number of hid-den layers and neurons contained in each layeris very important. For instance integrating ahigh number of neurons in the network permitsto model more complex relations but can alsolead to overfitting of the data and to poor outof sample performances. In this paper, in orderto overcome this problem, a neural network ar-chitecture optimisation algorithm based on theparameter decay principle has been used (regu-larisation of the problem). A second drawback isthat it is difficult to get insight on the learnt rela-tion simply by looking at the information storedin the network. In this way, neural networks aregenerally considered as “black box” algorithms.2.2.2 Random ForestsBagging for Bootstrap Aggregating, is a methodfor generating an ensemble of models con-structed from samples bootstrap replicates [3].These replicates are obtained by sampling uni-formly with replacement from the original sam-ples. The predictors are then combined by vot-ing for classification or averaging for regression[3]. The main advantage of averages of predic-tions from several models (like bootstrap sam-pling) is that it reduces the variance and predic-tion error.The base method used in the models hereafternamed “Bagging” and “Random Forest” is clas-sification and regression trees (CARTs) [4]. Thegoal of CARTs is to divide a sample of data usingbinary rules making the child nodes less hetero-geneous than the parent nodes. Once a tree isgrown it is possible to extract information fromthe tree structure, which makes it also a tool fordata analysis. The main advantages of CARTsis that it permits to perform a regression or aclassification with high dimensional inputs andthe major disadvantage of the later is that itis unstable i.e. a small change in the trainingsample can generate large changes in the learnedpredictor (classification or regression) [3]. TheBagging algorithm has been used in the specificcase of binary tree and Random Forest is a ver-sion more sophisticated of Bagging because itadds a random input selection which consists inselecting at random, at each node, a small group2European Wind Energy Conference - Brussels, Belgium, April 2008Figure 1: Map representing the location of the three considered wind farms WF1, WF2 and WF3.of input variables to split on. That way, the treesbuilt are more independent.In the Random Forest approach, the condi-tional mean E[Y |X = x] is approximated by theaveraged prediction of K single trees, each con-structed with an i.i.d. vector θk, k=1..K, whichrepresents the tree parameters defining how thetree is grown (e.g. split points).The main drawbacks of these two algorithmsare the important computing time of the learn-ing step and trees storage but the advantagesare that an insensibility to overfitting thanks toout-of-bag error and few parameters to adjust.Moreover, Random Forest provides informationabout the frequency of variables appearance intrees, which can be used to determine the im-portance of each input variable.2.2.3 Support Vector Machines (SVM)Support vector machines (SVM) are a recentsupervised learning methods used, initially forclassification, and generalized later for regres-sion. They are based on Vapnik’s research aboutlearning theory. Support vector machines forclassification are based on two ideas. First, aprinciple of maximum margin, which is the dis-tance maximizing the separation frontier andnearest elements called support vector. Thelearning step is the optimization of this frontierand can be presented like a quadratic optimiza-tion problem. Secondly, the input dimensionspace is transformed into a higher dimensionalspace, thanks to a kernel function, where a max-imal separating hyperplane is constructed. Thegoal is to transform a complex (non-linear) lowdimension problem into a simple (linear) highdimensional problem.The SVM can be also used to predict a quan-titative variable: “the Support Vector methodcan also be applied to the case of regression,maintaining all the main features that charac-terise the maximal margin algorithm: a non-linear function is learned by a linear learningmachine in a kernel-induced feature space whilethe capacity of the system is controlled by a pa-rameter that does not depend on the dimension-ality of the space” Cristianini and Shawe-Taylor(2000).The main advantage of SVM is that the regu-larisation technique makes the model very resis-tant to overfitting. The main drawback of SVMis that the computing time required might bevery high when compared to other non-linearlearning approaches.3European Wind Energy Conference - Brussels, Belgium, April 20083 Case study descriptionThree wind farms in France, denoted as WF1,WF2, and, WF3, are considered. They are rep-resentative of various terrain and climate con-ditions. WF1 is situated on a complex terrainand WF2, WF3 on a flat terrain. Hourly powerproduction time series are considered spanninga period of 18 months from July 2004 to De-cember 2005. For the same period, numericalweather predictions (NWPs) by the ARPEGEmodel of Meteo France are used. The forecastsare provided once a day for horizons 0 to 60hours ahead, with a 3-hour resolution, i.e. 20values for each meteorological variable are pro-vided per run.The meteorological variables considered inthis study are 50 meter above ground level windspeed and gust wind direction. These meteo-rological variables were found to be the mostinformative for these case study [5].The variable to be predicted Yt is the hourlyaverage power production of each wind farm.The explanatory variable vector (Xt) containsthe predicted wind speed and wind direction bythe NWP model, the last measured wind powerand the forecast horizon. These two last vari-ables permit to improve forecasts for the firstforecast horizons. The horizons of power pre-dictions are the same as that of NWPs, whichrange from 0 to 60 hours ahead, with a 3-hour resolution. The available dataset is dividedinto a learning-set and a test-set comprising 1year and 6 months of data respectively. The1 year learning-set permits to integrate all sea-sonal variations.4 ResultsThe chosen evaluation criteria are the Normal-ized Mean Absolute Error:NMAE(k) =∑Nt=1 |ε(t+ k|t)|N(4)and the Normalized Root Mean Square Error:NRMSE(k) =√∑Nt=1 (ε(t+ k|t))2N(5)where ε is the normalized prediction error, kis the horizon and N is the number of samples inthe testing set. These forecasts are compared topersistence, which is used as base line referencemodel, and simply consists in using the latestFigure 2: Comparison of NMAE results in thecase of the wind farm WF1 situated on a com-plex terrainFigure 3: Comparison of NMAE results in thecase of the wind farm WF2 situated on a flatterrainFigure 4: Comparison of NMAE results in thecase of the wind farm WF3 situated on a flatterrain4European Wind Energy Conference - Brussels, Belgium, April 2008Figure 5: Comparison of NRMSE results in thecase of the wind farm WF1 situated on a com-plex terrainFigure 6: Comparison of NRMSE results in thecase of the wind farm WF2 situated on a flatterrainFigure 7: Comparison of NRMSE results in thecase of the wind farm WF3 situated on a flatterrainobservation as forecast for all horizons. Persis-tence is commonly used as a benchmark modelin wind power forecasting.The first common conclusion is that all modelsoutperform persistence (excepted for 0-horizonwhich is a particular case corresponding to now-casting) and the level of accuracy depends ofthe terrain. Thus, we can observe a higher errorlevel for WF1, which is situated on a complexterrain, compared to WF2 and WF3, which areon flat terrains. Then, for the three wind farms,we observe a global superiority of the non-linearmodels over the linear ones. This could be easilyexplained by the non-linear relationship betweenwind and power.Even if the non-linear models presented herepermit to improve the global performance whencompared to linear ones, it should be noticedthat the performances of the simple linear ap-proach is still reasonably good when comparedto the persistence reference model. It shouldalso be noticed that the results presented hereare comparable to results found in the literaturefor wind farms located in similar terrains.5 ConclusionsThis paper presents a comparison of the perfor-mance of various data mining algorithms appliedto short-term wind power forecasting. The inter-est of non-linear methods is illustrated for whichthe performance is equivalent to that found inthe literature for wind farms located in simi-lar terrains. The comparison has revealed thatRandom Forest outperforms the rest of the mod-els. This model, originally applied here for windpower forecasting, is interesting since it does notrequire a long architecture optimisation step,only the number of trees in the forest has tobe optimised.Moreover, a generalization of RandomForests, Quantile Regression Forests give anon-parametric way of estimating conditionalquantiles for high-dimensional predictor vari-ables [6, 5]. Thus provides an additionalinformation on the uncertainty of the predic-tions for performing efficiently functions such asreserves estimation, unit commitment, tradingin electricity markets, a.o. Such prediction (de-terministic with prediction intervals) is namedprobabilistic forecasting and are, nowadays, animportant research field.5European Wind Energy Conference - Brussels, Belgium, April 20086 AcknowledgementsThe authors would like to thank EDF and MtoFrance for providing the data for the variouscase studies. This work was performed in theframe of project ENSEOLE, funded in partby ADEME, the French Environment and En-ergy Management Agency and a special thanksto Philippe Besse, professor at the engineeringschool INSAT.References[1] G. & Brownsword R. Giebel, G.; Karinio-takis. The state-of-the-art in short-term pre-diction of wind power - from a danish per-spective, 2003.[2] S. Santoso, M. Negnevitsky, and N. Hatziar-gyriou. Data mining and analysis tech-niques in wind power system applications:abridged. Power Engineering Society Gen-eral Meeting, 2006. IEEE, pages 3 pp.–, 18-22 June 2006.[3] Leo Breiman. Bagging predictors. MachineLearning, 24(2):123140, August 1996.[4] Charles J. Stone Leo Breiman, Jerome Fried-man and R.A. Olshen. Classification and Re-gression Trees. Chapman & Hall/CRC, 1984.[5] Jeremie Juban, Lionel Fugon, and GeorgeKariniotakis. Probabilistic short-term windpower forecasting based on kernel densityestimators. In Proceedings of the EuropeanWind Energy Conference, Milan, Italy, 7-10May 2007.[6] Nicolai Meinshausen. Quantile regressionforests. Journal of Machine Learning Re-search, 7:983999, June 2006.6

Data mining for wind power forecasting

https://hal-mines-paristech.archives-ouvertes.fr/hal-00506101

Data mining for wind power forecasting

Abstract

Similar works

Full text

Available Versions

HAL-MINES ParisTech