Study on Optimized Elman Neural Network Classification Algorithm Based on PLS and CA

High-dimensional large sample data sets, between feature variables and between samples, may cause some correlative or repetitive factors, occupy lots of storage space, and consume much computing time. Using the Elman neural network to deal with them, too many inputs will influence the operating efficiency and recognition accuracy; too many simultaneous training samples, as well as being not able to get precise neural network model, also restrict the recognition accuracy. Aiming at these series of problems, we introduce the partial least squares (PLS) and cluster analysis (CA) into Elman neural network algorithm, by the PLS for dimension reduction which can eliminate the correlative and repetitive factors of the features. Using CA eliminates the correlative and repetitive factors of the sample. If some subclass becomes small sample, with high-dimensional feature and fewer numbers, PLS shows a unique advantage. Each subclass is regarded as one training sample to train the different precise neural network models. Then simulation samples are discriminated and classified into different subclasses, using the corresponding neural network to recognize it. An optimized Elman neural network classification algorithm based on PLS and CA (PLS-CA-Elman algorithm) is established. The new algorithm aims at improving the operating efficiency and recognition accuracy. By the case analysis, the new algorithm has unique superiority, worthy of further promotion

Dynamic non-linear system modelling using wavelet-based soft computing techniques

The enormous number of complex systems results in the necessity of high-level and cost-efficient modelling structures for the operators and system designers. Model-based approaches offer a very challenging way to integrate a priori knowledge into the procedure. Soft computing based models in particular, can successfully be applied in cases of highly nonlinear problems. A further reason for dealing with so called soft computational model based techniques is that in real-world cases, many times only partial, uncertain and/or inaccurate data is available. Wavelet-Based soft computing techniques are considered, as one of the latest trends in system identification/modelling. This thesis provides a comprehensive synopsis of the main wavelet-based approaches to model the non-linear dynamical systems in real world problems in conjunction with possible twists and novelties aiming for more accurate and less complex modelling structure. Initially, an on-line structure and parameter design has been considered in an adaptive Neuro- Fuzzy (NF) scheme. The problem of redundant membership functions and consequently fuzzy rules is circumvented by applying an adaptive structure. The growth of a special type of Fungus (Monascus ruber van Tieghem) is examined against several other approaches for further justification of the proposed methodology. By extending the line of research, two Morlet Wavelet Neural Network (WNN) structures have been introduced. Increasing the accuracy and decreasing the computational cost are both the primary targets of proposed novelties. Modifying the synoptic weights by replacing them with Linear Combination Weights (LCW) and also imposing a Hybrid Learning Algorithm (HLA) comprising of Gradient Descent (GD) and Recursive Least Square (RLS), are the tools utilised for the above challenges. These two models differ from the point of view of structure while they share the same HLA scheme. The second approach contains an additional Multiplication layer, plus its hidden layer contains several sub-WNNs for each input dimension. The practical superiority of these extensions is demonstrated by simulation and experimental results on real non-linear dynamic system; Listeria Monocytogenes survival curves in Ultra-High Temperature (UHT) whole milk, and consolidated with comprehensive comparison with other suggested schemes. At the next stage, the extended clustering-based fuzzy version of the proposed WNN schemes, is presented as the ultimate structure in this thesis. The proposed Fuzzy Wavelet Neural network (FWNN) benefitted from Gaussian Mixture Models (GMMs) clustering feature, updated by a modified Expectation-Maximization (EM) algorithm. One of the main aims of this thesis is to illustrate how the GMM-EM scheme could be used not only for detecting useful knowledge from the data by building accurate regression, but also for the identification of complex systems. The structure of FWNN is based on the basis of fuzzy rules including wavelet functions in the consequent parts of rules. In order to improve the function approximation accuracy and general capability of the FWNN system, an efficient hybrid learning approach is used to adjust the parameters of dilation, translation, weights, and membership. Extended Kalman Filter (EKF) is employed for wavelet parameters adjustment together with Weighted Least Square (WLS) which is dedicated for the Linear Combination Weights fine-tuning. The results of a real-world application of Short Time Load Forecasting (STLF) further re-enforced the plausibility of the above technique

Artificial intelligence in wind speed forecasting: a review

Wind energy production has had accelerated growth in recent years, reaching an annual increase of 17% in 2021. Wind speed plays a crucial role in the stability required for power grid operation. However, wind intermittency makes accurate forecasting a complicated process. Implementing new technologies has allowed the development of hybrid models and techniques, improving wind speed forecasting accuracy. Additionally, statistical and artificial intelligence methods, especially artificial neural networks, have been applied to enhance the results. However, there is a concern about identifying the main factors influencing the forecasting process and providing a basis for estimation with artificial neural network models. This paper reviews and classifies the forecasting models used in recent years according to the input model type, the pre-processing and post-processing technique, the artificial neural network model, the prediction horizon, the steps ahead number, and the evaluation metric. The research results indicate that artificial neural network (ANN)-based models can provide accurate wind forecasting and essential information about the specific location of potential wind use for a power plant by understanding the future wind speed values

Sleep Stage Classification: A Deep Learning Approach

Sleep occupies significant part of human life. The diagnoses of sleep related disorders are of great importance. To record specific physical and electrical activities of the brain and body, a multi-parameter test, called polysomnography (PSG), is normally used. The visual process of sleep stage classification is time consuming, subjective and costly. To improve the accuracy and efficiency of the sleep stage classification, automatic classification algorithms were developed. In this research work, we focused on pre-processing (filtering boundaries and de-noising algorithms) and classification steps of automatic sleep stage classification. The main motivation for this work was to develop a pre-processing and classification framework to clean the input EEG signal without manipulating the original data thus enhancing the learning stage of deep learning classifiers. For pre-processing EEG signals, a lossless adaptive artefact removal method was proposed. Rather than other works that used artificial noise, we used real EEG data contaminated with EOG and EMG for evaluating the proposed method. The proposed adaptive algorithm led to a significant enhancement in the overall classification accuracy. In the classification area, we evaluated the performance of the most common sleep stage classifiers using a comprehensive set of features extracted from PSG signals. Considering the challenges and limitations of conventional methods, we proposed two deep learning-based methods for classification of sleep stages based on Stacked Sparse AutoEncoder (SSAE) and Convolutional Neural Network (CNN). The proposed methods performed more efficiently by eliminating the need for conventional feature selection and feature extraction steps respectively. Moreover, although our systems were trained with lower number of samples compared to the similar studies, they were able to achieve state of art accuracy and higher overall sensitivity

Koneoppimiskehys petrokemianteollisuuden sovelluksille

Machine learning has many potentially useful applications in process industry, for example in process monitoring and control. Continuously accumulating process data and the recent development in software and hardware that enable more advanced machine learning, are fulfilling the prerequisites of developing and deploying process automation integrated machine learning applications which improve existing functionalities or even implement artificial intelligence. In this master's thesis, a framework is designed and implemented on a proof-of-concept level, to enable easy acquisition of process data to be used with modern machine learning libraries, and to also enable scalable online deployment of the trained models. The literature part of the thesis concentrates on studying the current state and approaches for digital advisory systems for process operators, as a potential application to be developed on the machine learning framework. The literature study shows that the approaches for process operators' decision support tools have shifted from rule-based and knowledge-based methods to machine learning. However, no standard methods can be concluded, and most of the use cases are quite application-specific. In the developed machine learning framework, both commercial software and open source components with permissive licenses are used. Data is acquired over OPC UA and then processed in Python, which is currently almost the de facto standard language in data analytics. Microservice architecture with containerization is used in the online deployment, and in a qualitative evaluation, it proved to be a versatile and functional solution.Koneoppimisella voidaan osoittaa olevan useita hyödyllisiä käyttökohteita prosessiteollisuudessa, esimerkiksi prosessinohjaukseen liittyvissä sovelluksissa. Jatkuvasti kerääntyvä prosessidata ja toisaalta koneoppimiseen soveltuvien ohjelmistojen sekä myös laitteistojen viimeaikainen kehitys johtavat tilanteeseen, jossa prosessiautomaatioon liitettyjen koneoppimissovellusten avulla on mahdollista parantaa nykyisiä toiminnallisuuksia tai jopa toteuttaa tekoälysovelluksia. Tässä diplomityössä suunniteltiin ja toteutettiin prototyypin tasolla koneoppimiskehys, jonka avulla on helppo käyttää prosessidataa yhdessä nykyaikaisten koneoppimiskirjastojen kanssa. Kehys mahdollistaa myös koneopittujen mallien skaalautuvan käyttöönoton. Diplomityön kirjallisuusosa keskittyy prosessioperaattoreille tarkoitettujen digitaalisten avustajajärjestelmien nykytilaan ja toteutustapoihin, avustajajärjestelmän tai sen päätöstukijärjestelmän ollessa yksi mahdollinen koneoppimiskehyksen päälle rakennettava ohjelma. Kirjallisuustutkimuksen mukaan prosessioperaattorin päätöstukijärjestelmien taustalla olevat menetelmät ovat yhä useammin koneoppimiseen perustuvia, aiempien sääntö- ja tietämyskantoihin perustuvien menetelmien sijasta. Selkeitä yhdenmukaisia lähestymistapoja ei kuitenkaan ole helposti pääteltävissä kirjallisuuden perusteella. Lisäksi useimmat tapausesimerkit ovat sovellettavissa vain kyseisissä erikoistapauksissa. Kehitetyssä koneoppimiskehyksessä on käytetty sekä kaupallisia että avoimen lähdekoodin komponentteja. Prosessidata haetaan OPC UA -protokollan avulla, ja sitä on mahdollista käsitellä Python-kielellä, josta on muodostunut lähes de facto -standardi data-analytiikassa. Kehyksen käyttöönottokomponentit perustuvat mikropalveluarkkitehtuuriin ja konttiteknologiaan, jotka osoittautuivat laadullisessa testauksessa monipuoliseksi ja toimivaksi toteutustavaksi

Neuro-Fuzzy Based Intelligent Approaches to Nonlinear System Identification and Forecasting

Nearly three decades back nonlinear system identification consisted of several ad-hoc approaches, which were restricted to a very limited class of systems. However, with the advent of the various soft computing methodologies like neural networks and the fuzzy logic combined with optimization techniques, a wider class of systems can be handled at present. Complex systems may be of diverse characteristics and nature. These systems may be linear or nonlinear, continuous or discrete, time varying or time invariant, static or dynamic, short term or long term, central or distributed, predictable or unpredictable, ill or well defined. Neurofuzzy hybrid modelling approaches have been developed as an ideal technique for utilising linguistic values and numerical data. This Thesis is focused on the development of advanced neurofuzzy modelling architectures and their application to real case studies. Three potential requirements have been identified as desirable characteristics for such design: A model needs to have minimum number of rules; a model needs to be generic acting either as Multi-Input-Single-Output (MISO) or Multi-Input-Multi-Output (MIMO) identification model; a model needs to have a versatile nonlinear membership function. Initially, a MIMO Adaptive Fuzzy Logic System (AFLS) model which incorporates a prototype defuzzification scheme, while utilising an efficient, compared to the Takagi–Sugeno–Kang (TSK) based systems, fuzzification layer has been developed for the detection of meat spoilage using Fourier transform infrared (FTIR) spectroscopy. The identification strategy involved not only the classification of beef fillet samples in their respective quality class (i.e. fresh, semi-fresh and spoiled), but also the simultaneous prediction of their associated microbiological population directly from FTIR spectra. In the case of AFLS, the number of memberships for each input variable was directly associated to the number of rules, hence, the “curse of dimensionality” problem was significantly reduced. Results confirmed the advantage of the proposed scheme against Adaptive Neurofuzzy Inference System (ANFIS), Multilayer Perceptron (MLP) and Partial Least Squares (PLS) techniques used in the same case study. In the case of MISO systems, the TSK based structure, has been utilized in many neurofuzzy systems, like ANFIS. At the next stage of research, an Adaptive Fuzzy Inference Neural Network (AFINN) has been developed for the monitoring the spoilage of minced beef utilising multispectral imaging information. This model, which follows the TSK structure, incorporates a clustering pre-processing stage for the definition of fuzzy rules, while its final fuzzy rule base is determined by competitive learning. In this specific case study, AFINN model was also able to predict for the first time in the literature, the beef’s temperature directly from imaging information. Results again proved the superiority of the adopted model. By extending the line of research and adopting specific design concepts from the previous case studies, the Asymmetric Gaussian Fuzzy Inference Neural Network (AGFINN) architecture has been developed. This architecture has been designed based on the above design principles. A clustering preprocessing scheme has been applied to minimise the number of fuzzy rules. AGFINN incorporates features from the AFLS concept, by having the same number of rules as well as fuzzy memberships. In spite of the extensive use of the standard symmetric Gaussian membership functions, AGFINN utilizes an asymmetric function acting as input linguistic node. Since the asymmetric Gaussian membership function’s variability and flexibility are higher than the traditional one, it can partition the input space more effectively. AGFINN can be built either as an MISO or as an MIMO system. In the MISO case, a TSK defuzzification scheme has been implemented, while two different learning algorithms have been implemented. AGFINN has been tested on real datasets related to electricity price forecasting for the ISO New England Power Distribution System. Its performance was compared against a number of alternative models, including ANFIS, AFLS, MLP and Wavelet Neural Network (WNN), and proved to be superior. The concept of asymmetric functions proved to be a valid hypothesis and certainly it can find application to other architectures, such as in Fuzzy Wavelet Neural Network models, by designing a suitable flexible wavelet membership function. AGFINN’s MIMO characteristics also make the proposed architecture suitable for a larger range of applications/problems

State-of-the-Art Using Bibliometric Analysis of Wind-Speed and -Power Forecasting Methods Applied in Power Systems

The integration of wind energy into power systems has intensified as a result of the urgency for global energy transition. This requires more accurate forecasting techniques that can capture the variability of the wind resource to achieve better operative performance of power systems. This paper presents an exhaustive review of the state-of-the-art of wind-speed and -power forecasting models for wind turbines located in different segments of power systems, i.e., in large wind farms, distributed generation, microgrids, and micro-wind turbines installed in residences and buildings. This review covers forecasting models based on statistical and physical, artificial intelligence, and hybrid methods, with deterministic or probabilistic approaches. The literature review is carried out through a bibliometric analysis using VOSviewer and Pajek software. A discussion of the results is carried out, taking as the main approach the forecast time horizon of the models to identify their applications. The trends indicate a predominance of hybrid forecast models for the analysis of power systems, especially for those with high penetration of wind power. Finally, it is determined that most of the papers analyzed belong to the very short-term horizon, which indicates that the interest of researchers is in this time horizon

The role of artificial intelligence-driven soft sensors in advanced sustainable process industries: a critical review

With the predicted depletion of natural resources and alarming environmental issues, sustainable development has become a popular as well as a much-needed concept in modern process industries. Hence, manufacturers are quite keen on adopting novel process monitoring techniques to enhance product quality and process efficiency while minimizing possible adverse environmental impacts. Hardware sensors are employed in process industries to aid process monitoring and control, but they are associated with many limitations such as disturbances to the process flow, measurement delays, frequent need for maintenance, and high capital costs. As a result, soft sensors have become an attractive alternative for predicting quality-related parameters that are ‘hard-to-measure’ using hardware sensors. Due to their promising features over hardware counterparts, they have been employed across different process industries. This article attempts to explore the state-of-the-art artificial intelligence (Al)-driven soft sensors designed for process industries and their role in achieving the goal of sustainable development. First, a general introduction is given to soft sensors, their applications in different process industries, and their significance in achieving sustainable development goals. AI-based soft sensing algorithms are then introduced. Next, a discussion on how AI-driven soft sensors contribute toward different sustainable manufacturing strategies of process industries is provided. This is followed by a critical review of the most recent state-of-the-art AI-based soft sensors reported in the literature. Here, the use of powerful AI-based algorithms for addressing the limitations of traditional algorithms, that restrict the soft sensor performance is discussed. Finally, the challenges and limitations associated with the current soft sensor design, application, and maintenance aspects are discussed with possible future directions for designing more intelligent and smart soft sensing technologies to cater the future industrial needs

Multianalyte Quantifications by Means of Integration of Artificial Neural Networks, Genetic Algorithms and Chemometrics for Time-Resolved Analytical Data

During the last decade the application of sensors for the detection and determination of various substances has gained an increasing popularity not only in the field of analytical chemistry but also in our daily life. Most sensor systems such as exhaust gas sensors for automobiles are based on single sensors, which are as selective as possible for the analyte of interest. The problems of interfering cross-reactive analytes and the lack of specific sensors for many analytes have ended up in the development of so-called sensor-arrays. Thereby, several analytes can be simultaneously quantified by the multivariate data analysis of the signal patterns of several cross-reactive sensors. Yet, this approach is also limited since the number of sensors in the array has to exceed the number of cross-reacting analytes. In this work, a new approach is presented, which allows multi-analyte quantifications on the basis of single-sensor systems. Thereby, differences of interaction kinetics of the analytes and sensor are exploited using time-resolved measurements and time-resolved data analyses. This time-resolved evaluation of sensor signals together with suitable sensor materials combines the sensory principle with the chromatographic principle of separating analytes in space or time. The main objectives of this work can be subsumed into two focuses concerning the measurement principle and the data analysis. The first focus is the introduction of time-resolved measurements in the field of chemical sensing. In this work the time-resolved measurements are based on the microporous polymer Makrolon as sensitive sensor coating, which allows a kinetic separation of the analytes during the sorption and desorption on the basis of the size of analytes. Multi-analyte determinations using single sensors are successfully performed for three different setups and for many multicomponent mixtures of the low alcohols and the refrigerants R22 and R134a. The second focus concerns the multivariate data analysis of the data. It is demonstrated that a highest possible scanning rate of the time-resolved sensor responses is desirable resulting in a high number of variables. It is shown that wide-spread data analysis methods cannot cope with the amount of variables and with the nonlinear relationship between the sensor responses and the concentrations of the analytes. Thus, three different algorithms are innovated and optimized in this study to find a calibration with the highest possible generalization ability. These algorithms perform a simultaneous calibration and variable selection exploiting a data set limited in size to a maximum extend. One algorithm is based on many parallel runs of genetic algorithms combined with neural networks, one algorithm bases on many parallel runs of growing neural networks and the third algorithm uses several runs of the growing neural networks in a loop. All three algorithms show by far better calibrations than all common methods of multivariate calibration and than simple non-optimized neural networks for all data sets investigated. Additionally, the variable selection of these algorithms allows an insight into the relationship between the time-resolved sensor responses and the concentrations of the analytes. The variable selections also suggest optimizations in terms of shorter measurements for several data sets. All three algorithms successfully solve the problems of too many variables for too few samples and the problems caused by the nonlinearities present in the data with practically no input needed by the analyst. Together, both main focuses of this work impressively demonstrate how the combination of an advanced measurement principle and of an intelligent data analysis can improve the results of measurements at reduced hardware costs. Thereby the principle of single-sensor setups or few-sensor setups is not only limited to a size-selective recognition but can be extended to many analyte discriminating principles such as temperature-resolved measurements leaving room for many further investigations.Während des letzten Jahrzehnts haben Sensoren zur Detektion und Bestimmung von verschiedenen Substanzen nicht nur auf dem Gebiet der analytischen Chemie sondern auch im täglichen Leben rasend Verbreitung gefunden. Die meisten Sensorsysteme, wie zum Beispiel Abgasdetektoren für Automobile beruhen auf einzelnen Sensoren, welche möglichst spezifisch für den interessanten Analyten sind. Probleme auf Grund störender kreuzreaktiver Analyte und auf Grund eines Mangels an spezifischen Sensoren für viele Analyte führten zur Entwicklung so genannter Sensor-Arrays. Dabei können mehrere Analyte gleichzeitig quantifiziert werden, indem die Signalmuster von mehreren kreuzreaktiven Sensoren ausgewertet werden. Dieser Ansatz ist jedoch auch limitiert, da die Anzahl der Sensoren im Array größer als die Anzahl der kreuzreaktiven Analyte sein muss. In dieser Arbeit wird ein neuer Ansatz präsentiert, welcher es erlaubt, Multi-Analyt Quantifizierungen mit einem Einsensor-System durchzuführen. Hierbei werden Unterschiede der Wechselwirkungskinetiken zwischen den Analyten und dem Sensor mit Hilfe von zeitaufgelösten Messungen und zeitaufgelösten Datenauswertungen ausgenutzt. Zusammen mit geeigneten Sensormaterialien kombiniert die zeitaufgelöste Auswertung das Prinzip der Sensoren mit dem Prinzip der Chromatographie, welche Analyte räumlich oder zeitlich trennt. Die wichtigsten Zielsetzungen dieser Arbeit können unter den zwei Hauptgesichtspunkten Messprinzip und die Datenauswertung gestellt werden. Der erste Hauptgesichtspunkt ist die Einführung der zeitaufgelösten Messungen in die Sensorik. In dieser Arbeit basieren die zeitaufgelösten Messungen auf dem mikroporösen Polymer Makrolon als sensitive Sensorbeschichtung, welches eine kinetische Trennung der Analyte während der Sorption und der Desorption auf Grund der Analytgröße erlaubt. Es werden mit drei verschiedenen Einsensor-Aufbauten und vielen Mischungen der niederen Alkohole und der Kühlmittel R22 und R134a erfolgreich Mehrkomponentenanalysen erfolgreich durchgeführt. Der zweite Hauptgesichtspunkt betrifft die multivariate Datenauswertung. Es wird gezeigt, dass eine höchstmögliche Scanrate der zeitaufgelösten Sensorantworten wünschenswert ist, was zu einer hohen Anzahl an Variablen führt. Es wird demonstriert, dass weit verbreitete Datenauswertungsmethoden nicht mit der großen Anzahl an Variablen und mit dem nichtlinearen Zusammenhang zwischen den Sensorsignalen und den Analytkonzentrationen zurechtkommen. Deshalb werden in dieser Arbeit drei verschiedene Algorithmen entwickelt und optimiert, um eine Kalibration mit der höchstmöglichen Generalisierung zu finden. Diese Algorithmen führen eine gleichzeitige Kalibrierung und Variablenselektion durch, wobei sie einen Datensatz, welcher in der Größe limitiert ist, bestmöglich ausnutzen. Ein Algorithmus basiert auf vielen parallelen Läufen von genetischen Algorithmen kombiniert mit neuronalen Netzen. Der zweite Algorithmus beruht auf vielen parallelen Läufen von wachsenden neuronalen Netzen, während der dritte Algorithmus mehrere wachsende neuronale Netze in einer Schleife benutzt. Alle drei Algorithmen zeigen eine bei weitem bessere Kalibration als gewöhnliche Methoden der multivariaten Kalibration und als einfache nicht optimierte neuronale Netze für alle Datensätze, welche untersucht wurden. Zusätzlich erlaubt die Variablenselektion einen Einblick in den Zusammenhang zwischen den zeitaufgelösten Sensorantworten und den Konzentrationen der verschiedenen Analyte. Außerdem schlägt die Variablenselektion Optimierungen bezüglich kürzerer Messungen für mehrere Datensätze vor. Alle drei Algorithmen meistern erfolgreich das Problem von zu vielen Variablen für zu wenige Proben und die Probleme, welche von den in den Daten vorhandenen Nichtlinearitäten verursacht werden. Dabei sind praktisch keine Eingaben des Benutzers nötig. Zusammen liefern beide Hauptaspekte dieser Arbeit eine beeindruckende Demonstration, wie die Kombination eines fortschrittlichen Messprinzips mit einer intelligenten Datenauswertung die Ergebnisse von Messungen bei reduzierten Kosten für die Hardware verbessern kann. Dabei ist das Prinzip der Einsensor-Aufbauten beziehungsweise der Aufbauten mit wenigen Sensoren nicht auf ein größenselektives Erkennungsprinzip limitiert, sondern kann auf viele Prinzipien der Unterscheidung von Analyten wie zum Beispiel temperaturaufgelöste Messungen erweitert werden, was weiteren Untersuchungen ein nahezu endloses Feld eröffnet