A comparison of four data selection methods for artificial neural networks and support vector machines

The performance of data-driven models such as Artificial Neural Networks and Support Vector Machines relies to a good extent on selecting proper data throughout the design phase. This paper addresses a comparison of four unsupervised data selection methods including random, convex hull based, entropy based and a hybrid data selection method. These methods were evaluated on eight benchmarks in classification and regression problems. For classification, Support Vector Machines were used, while for the regression problems, Multi-Layer Perceptrons were employed. Additionally, for each problem type, a non-dominated set of Radial Basis Functions Neural Networks were designed, benefiting from a Multi Objective Genetic Algorithm. The simulation results showed that the convex hull based method and the hybrid method involving convex hull and entropy, obtain better performance than the other methods, and that MOGA designed RBFNNs always perform better than the other models. (C) 2017, IFAC (International Federation of Automatic Control) Hosting by Elsevier Ltd. All rights reserved.FCT through IDMEC, under LAETA grant [UID/EMS/50022/2013]info:eu-repo/semantics/publishedVersio

Forecasting electricity demand in households using MOGA-designed artificial neural networks

The prediction of electricity demand plays an essential role in the building environment. It strongly contributes to making the building more energy-efficient, having the potential to increase both thermal and visual comfort of the occupants, while reducing energy consumption, by allowing the use of model predictive control. The present article focuses on the use of computational intelligence methods for prediction of the power consumption of a case study residential building, during a horizon of 12 hours. Two exogeneous variables (ambient temperature and day code) are used in the NARX model Two different time steps were considered in the simulations, as well as constrained and unconstrained model design. The study concluded that the smaller timestep and the constrained model design obtain the best power demand prediction performance. The results obtained compare very favourably with similar approaches in the literature Copyright (C) 2020 The Authors.UIDB/50022/2020, 01/SAICT/2018info:eu-repo/semantics/publishedVersio

An intelligent support system for automatic detection of cerebral vascular accidents from brain CT images

Objective: This paper presents a Radial Basis Functions Neural Network (RBFNN) based detection system, for automatic identification of Cerebral Vascular Accidents (CVA) through analysis of Computed Tomographic (CT) images. Methods: For the design of a neural network classifier, a Multi Objective Genetic Algorithm (MOGA) framework is used to determine the architecture of the classifier, its corresponding parameters and input features by maximizing the classification precision, while ensuring generalization. This approach considers a large number of input features, comprising first and second order pixel intensity statistics, as well as symmetry/asymmetry information with respect to the ideal mid-sagittal line. Results: Values of specificity of 98% and sensitivity of 98% were obtained, at pixel level, by an ensemble of non-dominated models generated by MOGA, in a set of 150 CT slices (1,867,602 pixels), marked by a NeuroRadiologist. This approach also compares favorably at a lesion level with three other published solutions, in terms of specificity (86% compared with 84%), degree of coincidence of marked lesions (89% compared with 77%) and classification accuracy rate (96% compared with 88%). (C) 2017 Published by Elsevier Ireland Ltd.FCTIDMECLAETA [UID/EMS/50022/2013

Non-intrusive load monitoring of household devices using a hybrid deep learning model through convex hull-based data selection

The availability of smart meters and IoT technology has opened new opportunities, ranging from monitoring electrical energy to extracting various types of information related to household occupancy, and with the frequency of usage of different appliances. Non-intrusive load monitoring (NILM) allows users to disaggregate the usage of each device in the house using the total aggregated power signals collected from a smart meter that is typically installed in the household. It enables the monitoring of domestic appliance use without the need to install individual sensors for each device, thus minimizing electrical system complexities and associated costs. This paper proposes an NILM framework based on low frequency power data using a convex hull data selection approach and hybrid deep learning architecture. It employs a sliding window of aggregated active and reactive powers sampled at 1 Hz. A randomized approximation convex hull data selection approach performs the selection of the most informative vertices of the real convex hull. The hybrid deep learning architecture is composed of two models: a classification model based on a convolutional neural network trained with a regression model based on a bidirectional long-term memory neural network. The results obtained on the test dataset demonstrate the effectiveness of the proposed approach, achieving F1 values ranging from 0.95 to 0.99 for the four devices considered and estimation accuracy values between 0.88 and 0.98. These results compare favorably with the performance of existing approaches.This research was funded by Programa Operacional Portugal 2020 and Operational Program CRESC Algarve 2020, grant numbers 39578/2018 and 72581/2020. Antonio Ruano also acknowledges the support of Fundação para a Ciência e Tecnologia, grant UID/EMS/50022/2020, through IDMEC under LAETAinfo:eu-repo/semantics/publishedVersio

Short-term forecasting photovoltaic solar power for home energy management systems

Accurate photovoltaic (PV) power forecasting is crucial to achieving massive PV integration in several areas, which is needed to successfully reduce or eliminate carbon dioxide from energy sources. This paper deals with short-term multi-step PV power forecasts used in model-based predictive control for home energy management systems. By employing radial basis function (RBFs) artificial neural networks (ANN), designed using a multi-objective genetic algorithm (MOGA) with data selected by an approximate convex-hull algorithm, it is shown that excellent forecasting results can be obtained. Two case studies are used: a special house located in the USA, and the other a typical residential house situated in the south of Portugal. In the latter case, one-step-ahead values for unscaled root mean square error (RMSE), mean relative error (MRE), normalized mean average error (NMAE), mean absolute percentage error (MAPE) and R2 of 0.16, 1.27%, 1.22%, 8% and 0.94 were obtained, respectively. These results compare very favorably with existing alternatives found in the literature.Programa Operacional Portugal 2020 and Operational Program CRESC Algarve 2020 grant 01/SAICT/2018. Antonio Ruano acknowledges the support of Fundação para a Ciência e Tecnologia, through IDMEC, under LAETA, grant UIDB/50022/2020.info:eu-repo/semantics/publishedVersio

Designing robust forecasting ensembles of Data-Driven Models with a Multi-Objective Formulation: An application to Home Energy Management Systems

This work proposes a procedure for the multi-objective design of a robust forecasting ensemble of data-driven models. Starting with a data-selection algorithm, a multi-objective genetic algorithm is then executed, performing topology and feature selection, as well as parameter estimation. From the set of non-dominated or preferential models, a smaller sub-set is chosen to form the ensemble. Prediction intervals for the ensemble are obtained using the covariance method. This procedure is illustrated in the design of four different models, required for energy management systems. Excellent results were obtained by this methodology, superseding the existing alternatives. Further research will incorporate a robustness criterion in MOGA, and will incorporate the prediction intervals in predictive control techniques.Grant number 72581/2020info:eu-repo/semantics/publishedVersio

Design of ensemble forecasting models for home energy management systems

The increasing levels of energy consumption worldwide is raising issues with respect to surpassing supply limits, causing severe effects on the environment, and the exhaustion of energy resources. Buildings are one of the most relevant sectors in terms of energy consumption; as such, efficient Home or Building Management Systems are an important topic of research. This study discusses the use of ensemble techniques in order to improve the performance of artificial neural networks models used for energy forecasting in residential houses. The case study is a residential house, located in Portugal, that is equipped with PV generation and battery storage and controlled by a Home Energy Management System (HEMS). It has been shown that the ensemble forecasting results are superior to single selected models, which were already excellent. A simple procedure was proposed for selecting the models to be used in the ensemble, together with a heuristic to determine the number of models.info:eu-repo/semantics/publishedVersio

Artificial neural network models: data selection and online adaptation

Energy consumption has been increasing steadily due to globalization and industrialization. Studies have shown that buildings have the biggest proportion in energy consumption; for example in European Union countries, energy consumption in buildings represents around 40% of the total energy consumption. Hence this PhD was intended towards managing the energy consumed by Heating, Ventilating and Air Conditioning (HVAC) systems in buildings benefiting from Model Predictive Control (MPC) technique. To achieve this goal, artificial intelligence models such as neural networks and Support Vector Machines (SVM) have been proposed because of their high potential capabilities of performing accurate nonlinear mappings between inputs and outputs in real environments which are not noise-free. In this PhD, Radial Basis Function Neural Networks (RBFNN) as a promising class of Artificial Neural Networks (ANN) were considered to model a sequence of time series processes where the RBFNN models were built using Multi Objective Genetic Algorithm (MOGA) as a design platform. Regarding the design of such models, two main challenges were tackled; data selection and model adaptation. Since RBFNNs are data driven models, the performance of such models relies, to a good extent, on selecting proper data throughout the design phase, covering the whole input-output range in which they will be employed. The convex hull algorithms can be applied as methods for data selection; however the use of conventional implementations of these methods in high dimensions, due to their high complexity, is not feasible. As the first phase of this PhD, a new randomized approximation convex hull algorithm called ApproxHull was proposed for high dimensions so that it can be used in an acceptable execution time, and with low memory requirements. Simulation results showed that applying ApproxHull as a filter data selection method (i.e., unsupervised data selection method) could improve the performance of the classification and regression models, in comparison with random data selection method. In addition, ApproxHull was employed in real applications in terms of three case studies. The first two were in association with applying predictive models for energy saving. The last case study was related to segmentation of lesion areas in brain Computed Tomography (CT) images. The evaluation results showed that applying ApproxHull in MOGA could result in models with an acceptable level of accuracy. Specifically, the results obtained from the third case study demonstrated that ApproxHull is capable of being applied on large size data sets in high dimensions. Besides the random selection method, it was also compared with an entropy based unsupervised data selection method and a hybrid method involving ApproxHull and the entropy based method. Based on the simulation results, for most cases, ApproxHull and the hybrid method achieved a better performance than the others. In the second phase of this PhD, a new convex-hull-based sliding window online adaptation method was proposed. The goal was to update the offline predictive RBFNN models used in HVAC MPC technique, where these models are applied to processes in which the data input-output range changes over time. The idea behind the proposed method is capturing a new arriving point at each time instant which reflects a new range of data by comparing the point with current convex hull presented via ApproxHull. In this situation the underlying model’s parameters are updated based on the new point and a sliding window of some past points. The simulation results showed that not only the proposed method could efficiently update the model while a good level of accuracy is kept but also it was comparable with other methods.Devido aos processos de industrialização e globalização o consumo de energia tem aumentado de forma contínua. A investigação sobre o consumo mostra que os edifícios consomem a maior fatia de energia. Por exemplo nos países da União Europeia essa fatia corresponde a cerca de 40% de toda a energia consumida. Assim, esta tese de Doutoramento tem um objetivo prático de contribuir para melhorar a gestão da energia consumida por sistemas Heating, Ventilating and Air Conditioning (HVAC) em edifícios, no âmbito de uma estratégia de controlo preditivo baseado em modelos. Neste contexto foram já propostos modelos baseados em redes neuronais artificiais e máquinas de vetores de suporte, para mencionar apenas alguns. Estas técnicas têm uma grande capacidade de modelar relações não-lineares entre entradas e saídas de sistemas, e são aplicáveis em ambientes de operação, que, como sabemos, estão sujeitos a várias formas de ruído. Nesta tese foram consideradas redes neuronais de função de base radial, uma técnica consolidada no contexto da modelação de séries temporais. Para desenhar essas redes foi utilizada uma ferramenta baseada num algoritmo genético multi-objectivo. Relativamente ao processo de desenho destes modelos, esta tese versa sobre dois aspetos menos estudados: a seleção de dados e a adaptação em linha dos modelos. Uma vez que as redes neuronais artificiais são modelos baseados em dados, a sua performance depende em boa medida da existência de dados apropriados e representativos do sistema/processo, que cubram toda a gama de valores que a representação entrada/saída do processo/sistema gera. Os algoritmos que determinam a figura geométrica que envolve todos os dados, denominados algoritmos convex hull, podem ser aplicados à tarefa de seleção de dados. Contudo a utilização das implementações convencionais destes algoritmos em problemas de grane dimensionalidade não é viável do ponto de vista prático. Numa primeira fase deste trabalho foi proposto um novo método randomizado de aproximação ao convex hull, cunhado com o nome ApproxHull, apropriado para conjuntos de dados de grande dimensão, de forma a ser viável do ponto de vista das aplicações práticas. Os resultados experimentais mostraram que a aplicação do ApproxHull como método de seleção de dados do tipo filtro, ou seja, não supervisionado, pode melhorar o desempenho de modelos em problemas de classificação e regressão, quando comparado com a seleção aleatória de dados. O ApproxHull foi também aplicado em três casos de estudo relativos a aplicações reais. Nos dois primeiros casos no contexto do desenvolvimento de modelos preditivos para sistemas na área da eficiência energética. O terceiro caso de estudo consiste no desenvolvimento de modelos de classificação para uma aplicação na área da segmentação de lesões em imagens de tomografia computorizada. Os resultados revelaram que da aplicação do método proposto resultaram modelos com uma precisão aceitável. Do ponto de vista da aplicabilidade do método, os resultados mostraram que o ApproxHull pode ser utilizado em conjuntos de dados grandes e com dados de grande dimensionalidade. Para além da comparação com a seleção aleatória de dados, o método foi também comparado com um método de seleção de dados baseado no conceito de entropia e com um método híbrido que resulta da combinação do ApproxHull com o método entrópico. Com base nos resultados experimentais apurou-se que na maioria dos casos estudados o método híbrido conseguiu melhor desempenho que os restantes. Numa segunda fase do trabalho foi proposto um novo método de adaptação em linha com base no algoritmo ApproxHull e numa janela deslizante no tempo. Uma vez que os processos e sistemas na envolvente do sistema HVAC são variantes no tempo e dinâmicos, o objetivo foi aplicar o método proposto para adaptar em linha os modelos que foram primeiramente obtidos fora de linha. A ideia base do método proposto consiste em comparar cada novo par entrada/saída com o convex hull conhecido, e determinar se o novo par tem dados situados fora da gama conhecida. Nessa situação os parâmetros dos modelos são atualizados com base nesse novo ponto e num conjunto de pontos numa determinada janela temporal deslizante. Os resultados experimentais demonstraram não só que o novo método é eficiente na atualização dos modelos e em mantê-los num bom nível de precisão, mas também que era comparável a outros métodos existentes

A convex hull-based data selection method for data driven models

The accuracy of classification and regression tasks based on data driven models, such as Neural Networks or Support Vector Machines, relies to a good extent on selecting proper data for designing these models, covering the whole input range in which they will be employed. The convex hull algorithm can be applied as a method for data selection; however the use of conventional implementations of this method in high dimensions, due to its high complexity, is not feasible. In this paper, we propose a randomized approximation convex hull algorithm which can be used for high dimensions in an acceptable execution time, and with low memory requirements. Simulation results show that data selection by the proposed algorithm (coined as ApproxHull) can improve the performance of classification and regression models, in comparison with random data selection. (C) 2016 Elsevier B.V. All rights reserved

Intelligent support system for CVA diagnosis by cerebral computerized tomography

The Cerebral Vascular Accident (CVA) is one of the major causes of death in USA and developed countries, immediately following cardiac diseases and tumors. The increasing number of CVA’s and the requirement of short time diagnosis to minimize morbidity and mortality encourages the development of computer aided diagnosis systems. Early stages of CVA are often undetected by human eye observation of Computer Tomographic (CT) images, thus incorporation of intelligent based techniques on such systems is expected to highly improve their performance. This thesis presents a Radial Basis Functions Neural Network (RBFNN) based diagnosis system for automatic identification of CVA through analysis of CT images. The research hereby reported included construction of a database composed of annotated CT images, supported by a web-based tool for Neuroradiologist registration of his/her normal or abnormal interpretation of each CT image; in case of an abnormal identification the medical doctor was indicted by the software application to designate the lesion type and to identify the abnormal region on each CT’s slice image. Once provided the annotated database each CT image processing considered a pre-processing stage for artefact removal and tilted images’ realignment followed by a feature extraction stage. A large number of features was considered, comprising first and second order pixel intensity statistics as well as symmetry/asymmetry information with respect to the ideal mid-sagittal line of each image. The policy conducted during the intelligent-driven image processing system development included the design of a neural network classifier. The architecture was determined by a Multi Objective Genetic Algorithm (MOGA) where the classifier structure, parameters and image features (input features) were chosen based on the use of different (often conflicting) objectives, ensuring maximization of the classification precision and a good generalization of its performance for unseen data Several scenarios of choosing proper MOGA’s data sets were conducted. The best result was obtained from the scenario where all boundary data points of an enlarged dataset were included in the MOGA training set. Confronted with the NeuroRadiologist annotations, specificity values of 98.01% and sensitivity values of 98.22% were obtained by the computer aided system, at pixel level. These values were achieved when an ensemble of non-dominated models generated by MOGA in the best scenario, was applied to a set of 150 CT slices (1,867,602 pixels). Present results show that the MOGA designed RBFNN classifier achieved better classification results than Support Vector Machines (SVM), despite the huge difference in complexity of the two classifiers. The proposed approach compares also favorably with other similar published solutions, both at lesion level specificity and at the degree of coincidence of marked lesions