Autoregressive time series prediction by means of fuzzy inference systems using nonparametric residual variance estimation

We propose an automatic methodology framework for short- and long-term prediction of time series by means of fuzzy inference systems. In this methodology, fuzzy techniques and statistical techniques for nonparametric residual variance estimation are combined in order to build autoregressive predictive models implemented as fuzzy inference systems. Nonparametric residual variance estimation plays a key role in driving the identification and learning procedures. Concrete criteria and procedures within the proposed methodology framework are applied to a number of time series prediction problems. The learn from examples method introduced by Wang and Mendel (W&M) is used for identification. The Levenberg–Marquardt (L–M) optimization method is then applied for tuning. The W&M method produces compact and potentially accurate inference systems when applied after a proper variable selection stage. The L–M method yields the best compromise between accuracy and interpretability of results, among a set of alternatives. Delta test based residual variance estimations are used in order to select the best subset of inputs to the fuzzy inference systems as well as the number of linguistic labels for the inputs. Experiments on a diverse set of time series prediction benchmarks are compared against least-squares support vector machines (LS-SVM), optimally pruned extreme learning machine (OP-ELM), and k-NN based autoregressors. The advantages of the proposed methodology are shown in terms of linguistic interpretability, generalization capability and computational cost. Furthermore, fuzzy models are shown to be consistently more accurate for prediction in the case of time series coming from real-world applications.Ministerio de Ciencia e Innovación TEC2008-04920Junta de Andalucía P08-TIC-03674, IAC07-I-0205:33080, IAC08-II-3347:5626

Design and implementation of machine learning techniques for modeling and managing battery energy storage systems

The fast technological evolution and industrialization that have interested the humankind since the fifties has caused a progressive and exponential increase of CO2 emissions and Earth temperature. Therefore, the research community and the political authorities have recognized the need of a deep technological revolution in both the transportation and the energy distribution systems to hinder climate changes. Thus, pure and hybrid electric powertrains, smart grids, and microgrids are key technologies for achieving the expected goals. Nevertheless, the development of the above mentioned technologies require very effective and performing Battery Energy Storage Systems (BESSs), and even more effective Battery Management Systems (BMSs). Considering the above background, this Ph.D. thesis has focused on the development of an innovative and advanced BMS that involves the use of machine learning techniques for improving the BESS effectiveness and efficiency. Great attention has been paid to the State of Charge (SoC) estimation problem, aiming at investigating solutions for achieving more accurate and reliable estimations. To this aim, the main contribution has concerned the development of accurate and flexible models of electrochemical cells. Three main modeling requirements have been pursued for ensuring accurate SoC estimations: insight on the cell physics, nonlinear approximation capability, and flexible system identification procedures. Thus, the research activity has aimed at fulfilling these requirements by developing and investigating three different modeling approaches, namely black, white, and gray box techniques. Extreme Learning Machines, Radial Basis Function Neural Networks, and Wavelet Neural Networks were considered among the black box models, but none of them were able to achieve satisfactory SoC estimation performances. The white box Equivalent Circuit Models (ECMs) have achieved better results, proving the benefit that the insight on the cell physics provides to the SoC estimation task. Nevertheless, it has appeared clear that the linearity of ECMs has reduced their effectiveness in the SoC task. Thus, the gray box Neural Networks Ensemble (NNE) and the white box Equivalent Neural Networks Circuit (ENNC) models have been developed aiming at exploiting the neural networks theory in order to achieve accurate models, ensuring at the same time very flexible system identification procedures together with nonlinear approximation capabilities. The performances of NNE and ENNC have been compelling. In particular, the white box ENNC has reached the most effective performances, achieving accurate SoC estimations, together with a simple architecture and a flexible system identification procedure. The outcome of this thesis makes it possible the development of an interesting scenario in which a suitable cloud framework provides remote assistance to several BMSs in order to adapt the managing algorithms to the aging of BESSs, even considering different and distinct applications

Peramalan Beban Listrik Jangka Pendek Menggunakan Optimally Pruned Extreme Learning Machine (OPELM) Pada Sistem Kelistrikan Jawa Timur

Peramalan beban listrik jangka pendek merupakan faktor yang sangat penting dalam perencanaan dan pengoperasian sistem tenaga listrik. Tujuan dari peramalan beban listrik adalah agar permintaan listrik dan penyediaan listrik dapat seimbang. Karakteristik beban di wilayah Jawa Timur sangat fluktuatif sehingga pada penelitian ini digunakan metode Optimally Pruned Extreme Learning Machine (OPELM) untuk meramalkan beban listrik. Kelebihan OPELM ada pada learning speed yang cepat dan pemilihan model yang tepat meskipun datanya mempunyai pola non linier. Keakuratan metode OPELM dapat diketahui dengan menggunakan metode pembanding yaitu metode ELM. Kriteria keakuratan yang digunakan adalah MAPE. Hasil dari perbandingan kriteria keakuratan menunjukkan bahwa hasil peramalan OPELM lebih baik dari ELM. Error rata-rata hasil pengujian peramalan paling minimum menunjukkan MAPE sebesar 1,3579% terjadi pada peramalan hari Jumat, sementara pada hari yang sama dengan metode ELM menghasilkan MAPE sebesar 2,2179%

Methodologies for time series prediction and missing value imputation

The amount of collected data is increasing all the time in the world. More sophisticated measuring instruments and increase in the computer processing power produce more and more data, which requires more capacity from the collection, transmission and storage. Even though computers are faster, large databases need also good and accurate methodologies for them to be useful in practice. Some techniques are not feasible to be applied to very large databases or are not able to provide the necessary accuracy. As the title proclaims, this thesis focuses on two aspects encountered with databases, time series prediction and missing value imputation. The first one is a function approximation and regression problem, but can, in some cases, be formulated also as a classification task. Accurate prediction of future values is heavily dependent not only on a good model, which is well trained and validated, but also preprocessing, input variable selection or projection and output approximation strategy selection. The importance of all these choices made in the approximation process increases when the prediction horizon is extended further into the future. The second focus area deals with missing values in a database. The missing values can be a nuisance, but can be also be a prohibiting factor in the use of certain methodologies and degrade the performance of others. Hence, missing value imputation is a very necessary part of the preprocessing of a database. This imputation has to be done carefully in order to retain the integrity of the database and not to insert any unwanted artifacts to aggravate the job of the final data analysis methodology. Furthermore, even though the accuracy is always the main requisite for a good methodology, computational time has to be considered alongside the precision. In this thesis, a large variety of different strategies for output approximation and variable processing for time series prediction are presented. There is also a detailed presentation of new methodologies and tools for solving the problem of missing values. The strategies and methodologies are compared against the state-of-the-art ones and shown to be accurate and useful in practice.Maailmassa tuotetaan koko ajan enemmän ja enemmän tietoa. Kehittyneemmät mittalaitteet, nopeammat tietokoneet sekä kasvaneet siirto- ja tallennuskapasiteetit mahdollistavat suurien tietomassojen keräämisen, siirtämisen ja varastoinnin. Vaikka tietokoneiden laskentateho kasvaa jatkuvasti, suurten tietoaineistojen käsittelyssä tarvitaan edelleen hyviä ja tarkkoja menetelmiä. Kaikki menetelmät eivät sovellu valtavien aineistojen käsittelyyn tai eivät tuota tarpeeksi tarkkoja tuloksia. Tässä työssä keskitytään kahteen tärkeään osa-alueeseen tietokantojen käsittelyssä: aikasarjaennustamiseen ja puuttuvien arvojen täydentämiseen. Ensimmäinen näistä alueista on regressio-ongelma, jossa pyritään arvioimaan aikasarjan tulevaisuutta edeltävien näytteiden pohjalta. Joissain tapauksissa regressio-ongelma voidaan muotoilla myös luokitteluongelmaksi. Tarkka aikasarjan ennustaminen on riippuvainen hyvästä ja luotettavasta ennustusmallista. Malli on opetettava oikein ja sen oikeellisuus ja tarkkuus on varmistettava. Lisäksi aikasarjan esikäsittely, syötemuuttujien valinta- tai projektiotapa sekä ennustusstrategia täytyy valita huolella ja niiden soveltuvuus mallin yhteyteen on varmistettava huolellisesti. Tehtyjen valintojen tärkeys kasvaa entisestään mitä pidemmälle tulevaisuuteen ennustetaan. Toinen tämän työn osa-alue käsittelee puuttuvien arvojen ongelmaa. Tietokannasta puuttuvat arvot voivat heikentää data-analyysimenetelmän tuottamia tuloksia tai jopa estää joidenkin menetelmien käytön, joten puuttuvien arvojen arviointi ja täydentäminen esikäsittelyn osana on suositeltavaa. Täydentäminen on kuitenkin tehtävä harkiten, sillä puutteellinen täydentäminen johtaa hyvin todennäköisesti epätarkkuuksiin lopullisessa käyttökohteessa ja ei-toivottuihin rakenteisiin tietokannan sisällä. Koska kyseessä on esikäsittely, eikä varsinainen datan hyötykäyttö, puuttuvien arvojen täydentämiseen käytetty laskenta-aika tulisi minimoida säilyttäen laskentatarkkuus. Tässä väitöskirjassa on esitelty erilaisia tapoja ennustaa pitkän ajan päähän tulevaisuuteen ja keinoja syötemuuttujien valintaan. Lisäksi uusia menetelmiä puuttuvien arvojen täydentämiseen on kehitetty ja niitä on vertailtu olemassa oleviin menetelmiin

Peramalan Beban Listrik Jangka Pendek Menggunakan Optimally Pruned Extreme Learning Machine (OPELM) pada Sistem Kelistrikan Jawa Timur

Peramalan beban listrik jangka pendek merupakan faktor yang sangat penting dalam perencanaan dan pengoperasian sistem tenaga listrik. Tujuan dari peramalan beban listrik adalah agar permintaan listrik dan penyediaan listrik dapat seimbang. Karakteristik beban di wilayah Jawa Timur sangat fluktuatif sehingga pada penelitian ini digunakan metode Optimally Pruned Extreme Learning Machine (OPELM) untuk meramalkan beban listrik. Kelebihan OPELM ada pada learning speed yang cepat dan pemilihan model yang tepat meskipun datanya mempunyai pola non linier. Keakuratan metode OPELM dapat diketahui dengan menggunakan metode pembanding yaitu metode ELM. Kriteria keakuratan yang digunakan adalah MAPE. Hasil dari perbandingan kriteria keakuratan menunjukkan bahwa hasil peramalan OPELM lebih baik dari ELM. Error rata-rata hasil pengujian peramalan paling minimum menunjukkan MAPE sebesar 1,3579% terjadi pada peramalan hari Jumat, sementara pada hari yang sama dengan metode ELM menghasilkan MAPE sebesar 2,2179%

Peramalan Beban Listrik Jangka Pendek Menggunakan Optimally Pruned Extreme Learning Machine (OPELM) pada Sistem Kelistrikan Jawa Timur

Peramalan beban listrik jangka pendek merupakan faktor yang sangat penting dalam perencanaan dan pengoperasian sistem tenaga listrik. Tujuan dari peramalan beban listrik adalah agar permintaan listrik dan penyediaan listrik dapat seimbang. Karakteristik beban di wilayah Jawa Timur sangat fluktuatif sehingga pada penelitian ini digunakan metode Optimally Pruned Extreme Learning Machine (OPELM) untuk meramalkan beban listrik. Kelebihan OPELM ada pada learning speed yang cepat dan pemilihan model yang tepat meskipun datanya mempunyai pola non linier. Keakuratan metode OPELM dapat diketahui dengan menggunakan metode pembanding yaitu metode ELM. Kriteria keakuratan yang digunakan adalah MAPE. Hasil dari perbandingan kriteria keakuratan menunjukkan bahwa hasil peramalan OPELM lebih baik dari ELM. Error rata-rata hasil pengujian peramalan paling minimum menunjukkan MAPE sebesar 1,3579% terjadi pada peramalan hari Jumat, sementara pada hari yang sama dengan metode ELM menghasilkan MAPE sebesar 2,2179%

Redes neuronales y preprocesado de variables para modelos y sensores en bioingeniería

El propósito de esta Tesis Doctoral es proponer una alternativa viable a la aproximación de modelos y procesos en el ámbito científico y, más concretamente, en aplicaciones complejas de bioingeniería, en las cuales es imposible o muy costoso encontrar una relación directa entre las señales de entrada y de salida mediante modelos matemáticos sencillos o aproximaciones estadísticas. Del mismo modo, es interesante lograr una compactación de los datos que necesita un modelo para conseguir una predicción o clasificación en un tiempo y con un coste de implementación mínimos. Un modelo puede ser simplificado en gran medida al reducir el número de entradas o realizar operaciones matemáticas sobre éstas para transformarlas en nuevas variables. En muchos problemas de regresión (aproximación de funciones), clasificación y optimización, en general se hace uso de las nuevas metodologías basadas en la inteligencia artificial. La inteligencia artificial es una rama de las ciencias de la computación que busca automatizar la capacidad de un sistema para responder a los estímulos que recibe y proponer salidas adecuadas y racionales. Esto se produce gracias a un proceso de aprendizaje, mediante el cual se presentan ciertas muestras o �ejemplos� al modelo y sus correspondientes salidas y éste aprende a proponer las salidas correspondientes a nuevos estímulos que no ha visto previamente. Esto se denomina aprendizaje supervisado. También puede darse el caso de que tal modelo asocie las entradas con características similares entre sí para obtener una clasificación de las muestras de entrada sin necesidad de un patrón de salida. Este modelo de aprendizaje se denomina no supervisado. El principal exponente de la aplicación de la inteligencia artificial para aproximación de funciones y clasificación son las redes neuronales artificiales. Se trata de modelos que han demostrado sobradamente sus ventajas en el ámbito del modelado estadístico y de la predicción frente a otros métodos clásicos. NMateo Jiménez, F. (2012). Redes neuronales y preprocesado de variables para modelos y sensores en bioingeniería [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/16702Palanci

A Feature Selection Methodology for Steganalysis

Steganography has been known and used for a very long time, as a way to exchange information in an unnoticeable manner between parties, by embedding it in another, apparently innocuous, document. Nowadays steganographic techniques are mostly used on digital content. The online newspaper Wired News, reported in one of its articles [2] on steganography that several steganographic contents have been found on web sites with very large image database such as eBay. Niels Provos [3] has somewhat refuted these ideas by analyzing and classifying two million images from eBay and one million from USENet network and not finding any steganographic content embedded in these images. This could be due to many reasons, such as very low payloads, making the steganographic images very robust and secure to steganalysis. The security of a steganographic scheme has been defined theoretically by Cachin in [1] but this definition is very seldomly usable in practice. It requires to evaluate distributions and measure the Kullback-Leibler divergence between them. In practice, steganalysis is used as a way to evaluate the security of a steganographic scheme empirically: it aims at detecting whether a medium has been tampered with – but not to detect what is in the medium or how it has been embedded. By the use of features, one can get some relevant characteristics of the considered medium, and assess, by the use of machine learning tools, usually, whether the medium is genuine or not. This is only one way to perform steganalysis, but it remains the most common....Le principe de la stéganalyse est de classer un document incriminé comme original ou comme stéganographié. Cet article propose une méthodologie pour la stéganalyse utilisant la sélection de caractéristiques, orientée vers une diminution des intervales de confiance des résultats habituellement donnés. La sélection de caractéristiques permet également d’envisager une interprétation des caractéristiques d’images sélectionnées, dans le but de comprendre le fonctionnement intrinsèque des algorithmes de stéganographie. Il est montré que l’écart type des résultats obtenus habituellement en classification peut être très important (jusqu’à 5 %) lorsque des ensembles d’entrainements comportant trop peu d’échantillons sont utilisés. Ces tests sont menés sur six algorithmes de stéganographie, utilisés avec quatre taux d’insertions différents : 5, 10, 15 et 20 %. D’autre part, les caractéristiques sélectionnées (généralement 10 à 13 fois moins nombreuses que dans l’ensemble complet) permettent effectivement de faire ressortir les faiblesses ainsi que les avantages des algorithmes utilisés

Model based forecasting for demand response strategies

The incremental deployment of decentralized renewable energy sources in the distribution grid is triggering a paradigm change for the power sector. This shift from a centralized structure with big power plants to a decentralized scenario of distributed energy resources, such as solar and wind, calls for a more active management of the distribution grid. Conventional distribution grids were passive systems, in which the power was flowing unidirectionally from upstream to downstream. Nowadays, and increasingly in the future, the penetration of distributed generation (DG), with its stochastic nature and lack of controllability, represents a major challenge for the stability of the network, especially at the distribution level. In particular, the power flow reversals produced by DG cause voltage excursions, which must be compensated. This poses an obstacle to the energy transition towards a more sustainable energy mix, which can however be mitigated by using a more active approach towards the control of the distribution networks. Demand side management (DSM) offers a possible solution to the problem, allowing to actively control the balance between generation, consumption and storage, close to the point of generation. An active energy management implies not only the capability to react promptly in case of disturbances, but also to ability to anticipate future events and take control actions accordingly. This is usually achieved through model predictive control (MPC), which requires a prediction of the future disturbances acting on the system. This thesis treat challenges of distributed DSM, with a particular focus on the case of a high penetration of PV power plants. The first subject of the thesis is the evaluation of the performance of models for forecasting and control with low computational requirements, of distributed electrical batteries. The proposed methods are compared by means of closed loop deterministic and stochastic MPC performance. The second subject of the thesis is the development of model based forecasting for PV power plants, and methods to estimate these models without the use of dedicated sensors. The third subject of the thesis concerns strategies for increasing forecasting accuracy when dealing with multiple signals linked by hierarchical relations. Hierarchical forecasting methods are introduced and a distributed algorithm for reconciling base forecasters is presented. At the same time, a new methodology for generating aggregate consistent probabilistic forecasts is proposed. This method can be applied to distributed stochastic DSM, in the presence of high penetration of rooftop installed PV systems. In this case, the forecasts' errors become mutually dependent, raising difficulties in the control problem due to the nontrivial summation of dependent random variables. The benefits of considering dependent forecasting errors over considering them as independent and uncorrelated, are investigated. The last part of the thesis concerns models for distributed energy markets, relying on hierarchical aggregators. To be effective, DSM requires a considerable amount of flexible load and storage to be controllable. This generates the need to be able to pool and coordinate several units, in order to reach a critical mass. In a real case scenario, flexible units will have different owners, who will have different and possibly conflicting interests. In order to recruit as much flexibility as possible, it is therefore importan