Development of Neurofuzzy Architectures for Electricity Price Forecasting

In 20th century, many countries have liberalized their electricity market. This power markets liberalization has directed generation companies as well as wholesale buyers to undertake a greater intense risk exposure compared to the old centralized framework. In this framework, electricity price prediction has become crucial for any market player in their decision‐making process as well as strategic planning. In this study, a prototype asymmetric‐based neuro‐fuzzy network (AGFINN) architecture has been implemented for short‐term electricity prices forecasting for ISO New England market. AGFINN framework has been designed through two different defuzzification schemes. Fuzzy clustering has been explored as an initial step for defining the fuzzy rules while an asymmetric Gaussian membership function has been utilized in the fuzzification part of the model. Results related to the minimum and maximum electricity prices for ISO New England, emphasize the superiority of the proposed model over well‐established learning‐based models

Machine Learning Techniques to Mitigate Nonlinear Phase Noise in Moderate Baud Rate Optical Communication Systems

Nonlinear phase noise (NLPN) is the most common impairment that degrades the performance of radio-over-fiber networks. The effect of NLPN in the constellation diagram consists of a shape distortion of symbols that increases the symbol error rate due to symbol overlapping when using a conventional demodulation grid. Symbol shape characterization was obtained experimentally at a moderate baud rate (250 MBd) for constellations impaired by phase noise due to a mismatch between the optical carrier and the transmitted radio frequency signal. Machine learning algorithms have become a powerful tool to perform monitoring and to identify and mitigate distortions introduced in both the electrical and optical domains. Clustering-based demodulation assisted with Voronoi contours enables the definition of non-Gaussian boundaries to provide flexible demodulation of 16-QAM and 4+12 PSK modulation formats. Phase-offset and in-phase and quadrature imbalance may be detected on the received constellation and compensated by applying thresholding boundaries obtained from impairment characterization through statistical analysis. Experimental results show increased tolerance to the optical signal-to-noise ratio (OSNR) obtained from clustering methods based on k-means and fuzzy c-means Gustafson-Kessel algorithms. Improvements of 3.2 dB for 16-QAM, and 1.4 dB for 4+12 PSK in the OSNR scale as a function of the bit error rate are obtained without requiring additional compensation algorithms

Machine Learning Techniques to Mitigate Nonlinear Phase Noise in Moderate Baud Rate Optical Communication Systems

Nonlinear phase noise (NLPN) is the most common impairment that degrades the performance of radio-over-fiber networks. The effect of NLPN in the constellation diagram consists of a shape distortion of symbols that increases the symbol error rate due to symbol overlapping when using a conventional demodulation grid. Symbol shape characterization was obtained experimentally at a moderate baud rate (250 MBd) for constellations impaired by phase noise due to a mismatch between the optical carrier and the transmitted radio frequency signal. Machine learning algorithms have become a powerful tool to perform monitoring and to identify and mitigate distortions introduced in both the electrical and optical domains. Clustering-based demodulation assisted with Voronoi contours enables the definition of non-Gaussian boundaries to provide flexible demodulation of 16-QAM and 4+12 PSK modulation formats. Phase-offset and in-phase and quadrature imbalance may be detected on the received constellation and compensated by applying thresholding boundaries obtained from impairment characterization through statistical analysis. Experimental results show increased tolerance to the optical signal-to-noise ratio (OSNR) obtained from clustering methods based on k-means and fuzzy c-means Gustafson-Kessel algorithms. Improvements of 3.2 dB for 16-QAM, and 1.4 dB for 4+12 PSK in the OSNR scale as a function of the bit error rate are obtained without requiring additional compensation algorithms

Chapter Machine Learning Techniques to Mitigate Nonlinear Phase Noise in Moderate Baud Rate Optical Communication Systems

Nonlinear phase noise (NLPN) is the most common impairment that degrades the performance of radio-over-fiber networks. The effect of NLPN in the constellation diagram consists of a shape distortion of symbols that increases the symbol error rate due to symbol overlapping when using a conventional demodulation grid. Symbol shape characterization was obtained experimentally at a moderate baud rate (250 MBd) for constellations impaired by phase noise due to a mismatch between the optical carrier and the transmitted radio frequency signal. Machine learning algorithms have become a powerful tool to perform monitoring and to identify and mitigate distortions introduced in both the electrical and optical domains. Clustering-based demodulation assisted with Voronoi contours enables the definition of non-Gaussian boundaries to provide flexible demodulation of 16-QAM and 4+12 PSK modulation formats. Phase-offset and in-phase and quadrature imbalance may be detected on the received constellation and compensated by applying thresholding boundaries obtained from impairment characterization through statistical analysis. Experimental results show increased tolerance to the optical signal-to-noise ratio (OSNR) obtained from clustering methods based on k-means and fuzzy c-means Gustafson-Kessel algorithms. Improvements of 3.2 dB for 16-QAM, and 1.4 dB for 4+12 PSK in the OSNR scale as a function of the bit error rate are obtained without requiring additional compensation algorithms

A new approach to dealing with missing values in data-driven fuzzy modeling

Real word data sets often contain many missing elements. Most algorithms that automatically develop a rule-based model are not well suited to deal with incomplete data. The usual technique is to disregard the missing values or substitute them by a best guess estimate, which can bias the results. In this paper we propose a new method for estimating the parameters of a Takagi-Sugeno fuzzy model in the presence of incomplete data. We also propose an inference mechanism that can deal with the incomplete data. The presented method has the added advantage that it does not require imputation or iterative guess-estimate of the missing values. This methodology is applied to fuzzy modeling of a classification and regression problem. The performance of the obtained models are comparable with the results obtained when using a complete data set

Observer-biased bearing condition monitoring: from fault detection to multi-fault classification

Bearings are simultaneously a fundamental component and one of the principal causes of failure in rotary machinery. The work focuses on the employment of fuzzy clustering for bearing condition monitoring, i.e., fault detection and classification. The output of a clustering algorithm is a data partition (a set of clusters) which is merely a hypothesis on the structure of the data. This hypothesis requires validation by domain experts. In general, clustering algorithms allow a limited usage of domain knowledge on the cluster formation process. In this study, a novel method allowing for interactive clustering in bearing fault diagnosis is proposed. The method resorts to shrinkage to generalize an otherwise unbiased clustering algorithm into a biased one. In this way, the method provides a natural and intuitive way to control the cluster formation process, allowing for the employment of domain knowledge to guiding it. The domain expert can select a desirable level of granularity ranging from fault detection to classification of a variable number of faults and can select a specific region of the feature space for detailed analysis. Moreover, experimental results under realistic conditions show that the adopted algorithm outperforms the corresponding unbiased algorithm (fuzzy c-means) which is being widely used in this type of problems. (C) 2016 Elsevier Ltd. All rights reserved.Grant number: 145602

Neuro-Fuzzy Based Intelligent Approaches to Nonlinear System Identification and Forecasting

Nearly three decades back nonlinear system identification consisted of several ad-hoc approaches, which were restricted to a very limited class of systems. However, with the advent of the various soft computing methodologies like neural networks and the fuzzy logic combined with optimization techniques, a wider class of systems can be handled at present. Complex systems may be of diverse characteristics and nature. These systems may be linear or nonlinear, continuous or discrete, time varying or time invariant, static or dynamic, short term or long term, central or distributed, predictable or unpredictable, ill or well defined. Neurofuzzy hybrid modelling approaches have been developed as an ideal technique for utilising linguistic values and numerical data. This Thesis is focused on the development of advanced neurofuzzy modelling architectures and their application to real case studies. Three potential requirements have been identified as desirable characteristics for such design: A model needs to have minimum number of rules; a model needs to be generic acting either as Multi-Input-Single-Output (MISO) or Multi-Input-Multi-Output (MIMO) identification model; a model needs to have a versatile nonlinear membership function. Initially, a MIMO Adaptive Fuzzy Logic System (AFLS) model which incorporates a prototype defuzzification scheme, while utilising an efficient, compared to the Takagi–Sugeno–Kang (TSK) based systems, fuzzification layer has been developed for the detection of meat spoilage using Fourier transform infrared (FTIR) spectroscopy. The identification strategy involved not only the classification of beef fillet samples in their respective quality class (i.e. fresh, semi-fresh and spoiled), but also the simultaneous prediction of their associated microbiological population directly from FTIR spectra. In the case of AFLS, the number of memberships for each input variable was directly associated to the number of rules, hence, the “curse of dimensionality” problem was significantly reduced. Results confirmed the advantage of the proposed scheme against Adaptive Neurofuzzy Inference System (ANFIS), Multilayer Perceptron (MLP) and Partial Least Squares (PLS) techniques used in the same case study. In the case of MISO systems, the TSK based structure, has been utilized in many neurofuzzy systems, like ANFIS. At the next stage of research, an Adaptive Fuzzy Inference Neural Network (AFINN) has been developed for the monitoring the spoilage of minced beef utilising multispectral imaging information. This model, which follows the TSK structure, incorporates a clustering pre-processing stage for the definition of fuzzy rules, while its final fuzzy rule base is determined by competitive learning. In this specific case study, AFINN model was also able to predict for the first time in the literature, the beef’s temperature directly from imaging information. Results again proved the superiority of the adopted model. By extending the line of research and adopting specific design concepts from the previous case studies, the Asymmetric Gaussian Fuzzy Inference Neural Network (AGFINN) architecture has been developed. This architecture has been designed based on the above design principles. A clustering preprocessing scheme has been applied to minimise the number of fuzzy rules. AGFINN incorporates features from the AFLS concept, by having the same number of rules as well as fuzzy memberships. In spite of the extensive use of the standard symmetric Gaussian membership functions, AGFINN utilizes an asymmetric function acting as input linguistic node. Since the asymmetric Gaussian membership function’s variability and flexibility are higher than the traditional one, it can partition the input space more effectively. AGFINN can be built either as an MISO or as an MIMO system. In the MISO case, a TSK defuzzification scheme has been implemented, while two different learning algorithms have been implemented. AGFINN has been tested on real datasets related to electricity price forecasting for the ISO New England Power Distribution System. Its performance was compared against a number of alternative models, including ANFIS, AFLS, MLP and Wavelet Neural Network (WNN), and proved to be superior. The concept of asymmetric functions proved to be a valid hypothesis and certainly it can find application to other architectures, such as in Fuzzy Wavelet Neural Network models, by designing a suitable flexible wavelet membership function. AGFINN’s MIMO characteristics also make the proposed architecture suitable for a larger range of applications/problems

Unsupervised Heart-rate Estimation in Wearables With Liquid States and A Probabilistic Readout

Heart-rate estimation is a fundamental feature of modern wearable devices. In this paper we propose a machine intelligent approach for heart-rate estimation from electrocardiogram (ECG) data collected using wearable devices. The novelty of our approach lies in (1) encoding spatio-temporal properties of ECG signals directly into spike train and using this to excite recurrently connected spiking neurons in a Liquid State Machine computation model; (2) a novel learning algorithm; and (3) an intelligently designed unsupervised readout based on Fuzzy c-Means clustering of spike responses from a subset of neurons (Liquid states), selected using particle swarm optimization. Our approach differs from existing works by learning directly from ECG signals (allowing personalization), without requiring costly data annotations. Additionally, our approach can be easily implemented on state-of-the-art spiking-based neuromorphic systems, offering high accuracy, yet significantly low energy footprint, leading to an extended battery life of wearable devices. We validated our approach with CARLsim, a GPU accelerated spiking neural network simulator modeling Izhikevich spiking neurons with Spike Timing Dependent Plasticity (STDP) and homeostatic scaling. A range of subjects are considered from in-house clinical trials and public ECG databases. Results show high accuracy and low energy footprint in heart-rate estimation across subjects with and without cardiac irregularities, signifying the strong potential of this approach to be integrated in future wearable devices.Comment: 51 pages, 12 figures, 6 tables, 95 references. Under submission at Elsevier Neural Network

Dynamic non-linear system modelling using wavelet-based soft computing techniques

The enormous number of complex systems results in the necessity of high-level and cost-efficient modelling structures for the operators and system designers. Model-based approaches offer a very challenging way to integrate a priori knowledge into the procedure. Soft computing based models in particular, can successfully be applied in cases of highly nonlinear problems. A further reason for dealing with so called soft computational model based techniques is that in real-world cases, many times only partial, uncertain and/or inaccurate data is available. Wavelet-Based soft computing techniques are considered, as one of the latest trends in system identification/modelling. This thesis provides a comprehensive synopsis of the main wavelet-based approaches to model the non-linear dynamical systems in real world problems in conjunction with possible twists and novelties aiming for more accurate and less complex modelling structure. Initially, an on-line structure and parameter design has been considered in an adaptive Neuro- Fuzzy (NF) scheme. The problem of redundant membership functions and consequently fuzzy rules is circumvented by applying an adaptive structure. The growth of a special type of Fungus (Monascus ruber van Tieghem) is examined against several other approaches for further justification of the proposed methodology. By extending the line of research, two Morlet Wavelet Neural Network (WNN) structures have been introduced. Increasing the accuracy and decreasing the computational cost are both the primary targets of proposed novelties. Modifying the synoptic weights by replacing them with Linear Combination Weights (LCW) and also imposing a Hybrid Learning Algorithm (HLA) comprising of Gradient Descent (GD) and Recursive Least Square (RLS), are the tools utilised for the above challenges. These two models differ from the point of view of structure while they share the same HLA scheme. The second approach contains an additional Multiplication layer, plus its hidden layer contains several sub-WNNs for each input dimension. The practical superiority of these extensions is demonstrated by simulation and experimental results on real non-linear dynamic system; Listeria Monocytogenes survival curves in Ultra-High Temperature (UHT) whole milk, and consolidated with comprehensive comparison with other suggested schemes. At the next stage, the extended clustering-based fuzzy version of the proposed WNN schemes, is presented as the ultimate structure in this thesis. The proposed Fuzzy Wavelet Neural network (FWNN) benefitted from Gaussian Mixture Models (GMMs) clustering feature, updated by a modified Expectation-Maximization (EM) algorithm. One of the main aims of this thesis is to illustrate how the GMM-EM scheme could be used not only for detecting useful knowledge from the data by building accurate regression, but also for the identification of complex systems. The structure of FWNN is based on the basis of fuzzy rules including wavelet functions in the consequent parts of rules. In order to improve the function approximation accuracy and general capability of the FWNN system, an efficient hybrid learning approach is used to adjust the parameters of dilation, translation, weights, and membership. Extended Kalman Filter (EKF) is employed for wavelet parameters adjustment together with Weighted Least Square (WLS) which is dedicated for the Linear Combination Weights fine-tuning. The results of a real-world application of Short Time Load Forecasting (STLF) further re-enforced the plausibility of the above technique

Application of soft computing models with input vectors of snow cover area in addition to hydro-climatic data to predict the sediment loads

The accurate estimate of sediment load is important for management of the river ecosystem, designing of water infrastructures, and planning of reservoir operations. The direct measurement of sediment is the most credible method to estimate the sediments. However, this requires a lot of time and resources. Because of these two constraints, most often, it is not possible to continuously measure the daily sediments for most of the gauging sites. Nowadays, data-based sediment prediction models are famous for bridging the data gaps in the estimation of sediment loads. In data-driven sediment predictions models, the selection of input vectors is critical in determining the best structure of models for the accurate estimation of sediment yields. In this study, time series inputs of snow cover area, basin effective rainfall, mean basin average temperature, and mean basin evapotranspiration in addition to the flows were assessed for the prediction of sediment loads. The input vectors were assessed with artificial neural network (ANN), adaptive neuro-fuzzy logic inference system with grid partition (ANFIS-GP), adaptive neuro-fuzzy logic inference system with subtractive clustering (ANFIS-SC), adaptive neuro-fuzzy logic inference system with fuzzy c-means clustering (ANFIS-FCM), multiple adaptive regression splines (MARS), and sediment rating curve (SRC) models for the Gilgit River, the tributary of the Indus River in Pakistan. The comparison of different input vectors showed improvements in the prediction of sediments by using the snow cover area in addition to flows, effective rainfall, temperature, and evapotranspiration. Overall, the ANN model performed better than all other models. However, as regards sediment load peak time series, the sediment loads predicted using the ANN, ANFIS-FCM, and MARS models were found to be closer to the measured sediment loads. The ANFIS-FCM performed better in the estimation of peak sediment yields with a relative accuracy of 81.31% in comparison to the ANN and MARS models with 80.17% and 80.16% of relative accuracies, respectively. The developed multiple linear regression equation of all models show an R$^{2}$ value of 0.85 and 0.74 during the training and testing period, respectively