Metaheuristic design of feedforward neural networks: a review of two decades of research

Over the past two decades, the feedforward neural network (FNN) optimization has been a key interest among the researchers and practitioners of multiple disciplines. The FNN optimization is often viewed from the various perspectives: the optimization of weights, network architecture, activation nodes, learning parameters, learning environment, etc. Researchers adopted such different viewpoints mainly to improve the FNN's generalization ability. The gradient-descent algorithm such as backpropagation has been widely applied to optimize the FNNs. Its success is evident from the FNN's application to numerous real-world problems. However, due to the limitations of the gradient-based optimization methods, the metaheuristic algorithms including the evolutionary algorithms, swarm intelligence, etc., are still being widely explored by the researchers aiming to obtain generalized FNN for a given problem. This article attempts to summarize a broad spectrum of FNN optimization methodologies including conventional and metaheuristic approaches. This article also tries to connect various research directions emerged out of the FNN optimization practices, such as evolving neural network (NN), cooperative coevolution NN, complex-valued NN, deep learning, extreme learning machine, quantum NN, etc. Additionally, it provides interesting research challenges for future research to cope-up with the present information processing era

Methodology for utilising prior knowledge in constructing data-based process monitoring systems with an application to a dearomatisation process

Global competition is forcing the process industry to optimise the production processes. One key factor in optimisation is effective process state monitoring and fault detection. Another motivator to improve process monitoring systems are the substantial losses of revenue resulting from abnormal process conditions. It has been estimated that the petrochemical industry in the US alone loses 20 billion dollars per year because of unoptimal handling of abnormal process situations. Traditionally, the monitoring systems have been based on first principle models, constructed by specialists with process specific expertise. In contrast, the use of data-based modelling methods require less expertise and offers the possibilities to build and update the monitoring models in a short period of time, thus allowing more efficient development of monitoring systems. The aims of this thesis are to augment data-driven modelling with existing process knowledge, to combine different data-based modelling methods, and to utilise calculated variables in modelling in order to improve the accuracy of fault detection and identification (FDI) and to provide all necessary diagnostic information for fault tolerant control. The suggested improvements are included in a methodology for setting up FDI systems. The methodology has been tested by building FDI systems for detecting faults in two online quality analysers in a simulated and in a real industrial dearomatisation process at the Naantali oil refinery (Neste Oil Oyj). In developing an FDI system, background information about the user requirements for the monitoring system is first acquired. The information is then analysed and suitable modelling methods are selected according to the guidelines given in the methodology. Second, the process data are prepared for the modelling methods and augmented with appropriate calculated variables. Next, the input variable sets are determined with the introduced method and the models are constructed. After the estimation accuracy of the models is validated, the values of the fault detection parameters are determined. Finally, the fault detection performance of the system is tested. The system was evaluated during a period of one month at the Naantali refinery in 2007. The monitoring system was able to detect all the introduced analyser faults and to provide the information needed for a fault tolerant control system, thus validating the methodology. The effects of a number of suggested improvements in data-based modelling are analysed by means of a comparison study

Integration Techniques of Fault Detection and Isolation Using Interval Observers

An interval observer has been illustrated to be a suitable approach to detect and isolate faults affecting complex dynamical industrial systems. Concerning fault detection, interval observation is an appropriate passive robust strategy to generate an adaptive threshold to be used in residual evaluation when model uncertainty is located in parameters (interval model). In such approach, the observer gain is a key parameter since it determines the time evolution of the residual sensitivity to a fault and the minimum detectable fault. This thesis illustrates that the whole fault detection process is ruled by the dynamics of the fault residual sensitivity functions and by the time evolution of the adaptive threshold related to the interval observer. Besides, it must be taken into account that these two observer fault detection properties depend on the used observer gain. As a consequence, the observer gain becomes a tuning parameter which allows enhancing the observer fault detection performance while avoiding some drawbacks related to the analytical models, as the wrapping effect. In this thesis, the effect of the observer gain on fault detection and how this parameter can avoid some observer drawbacks (i.e. wrapping effect) are deeply analyzed. One of the results of this analysis is the determination of the minimum detectable fault function related to a given fault type. This function allows introducing a fault classification according to the fault detectability time evolution: permanently (strongly) detected, non-permanently (weakly) detected or just non-detected. In this fault detection part of this thesis, two examples have been used to illustrate the derived results: a mineral grinding-classification process and an industrial servo actuator. Concerning the interface between fault detection and fault isolation, this thesis shows that both modules can not be considered separately since the fault detection process has an important influence on the fault isolation result. This influence is not only due to the time evolution of the fault signals generated by the fault detection module but also to the fact that the fault residual sensitivity functions determines the faults which are affecting a given fault signal and the dynamics of this fault signal for each fault. This thesis illustrates this point suggesting that the interface between fault detection and fault isolation must consider a set of fault signals properties: binary property, sign property, fault residual sensitivity property, occurrence order property and occurrence time instant property. Moreover, as a result of the influence of the observer gain on the fault detection stage and on the fault residual sensitivity functions, this thesis demonstrates that the observer gain has also a key role in the fault isolation module which might allow enhancing its performance when this parameter is tuned properly (i.e. fault distinguishability may be increased). As a last point, this thesis analyzes the timed discrete-event nature of the fault signals generated by the fault detection module. As a consequence, it suggests using timed discrete-event models to model the fault isolation module. This thesis illustrates that this kind of models allow enhancing the fault isolation result. Moreover, as the monitored system is modelled using an interval observer, this thesis shows as this qualitative fault isolation model can be built up on the grounds of this system analytical model. Finally, the proposed fault isolation method is applied to detect and isolate faults of the Barcelona’s urban sewer system limnimeters. Keywords: Fault Detection, Fault Diagnosis, Robustness, Observers, Intervals, Discrete-event Systems.En la presente tesis se demuestra que el uso de observadores intervalares para detectar y aislar fallos en sistemas dinámicos complejos constituye una estrategia apropiada. En la etapa de detección del fallo, dicha estrategia permite determinar el umbral adaptativo usado en la evaluación del residuo (robustez pasiva). Dicha metodología, responde a la consideración de modelos con parámetros inciertos (modelos intervalares). En dicho enfoque, la ganancia del observador es un parámetro clave que permite determinar la evolución temporal de la sensibilidad del residuo a un fallo y el mínimo fallo detectable para un tipo de fallo determinado. Esta tesis establece que todo el proceso de detección de fallos viene determinado por la dinámica de las funciones sensibilidad del residuo a los diferentes fallos considerados y por la evolución temporal del umbral adaptativo asociado al observador intervalar. Además, se debe tener en cuenta que estas dos propiedades del observador respecto la detección de fallos dependen de la ganancia del observador. En consecuencia, la ganancia del observador se convierte en el parámetro de diseño que permite mejorar las prestaciones de dicho modelo respecto la detección de fallos mientras que permite evitar algunos defectos asociados al uso de modelos intervalares, como el efecto wrapping. Uno de los resultados obtenidos es la determinación de la función fallo mínimo detectable para un tipo de fallo dado. Esta función permite introducir una clasificación de los fallos en función de la evolución temporal de su detectabilidad: fallos permanentemente detectados, fallos no permanentemente detectados y fallos no detectados. En la primera parte de la tesis centrada en la detección de fallos se utilizan dos ejemplos para ilustrar los resultados obtenidos: un proceso de trituración y separación de minerales y un servoactuador industrial. Respecto a la interfaz entre la etapa de detección de fallos y el proceso de aislamiento, esta tesis muestra que ambos módulos no pueden considerarse separadamente dado que el proceso de detección tiene una importante influencia en el resultado de la etapa de aislamiento. Esta influencia no es debida sólo a la evolución temporal de las señales de fallo generados por el módulo de detección sino también porque las funciones sensibilidad del residuo a los diferentes posibles fallos determinan los fallos que afectan a un determinado señal de fallo y la dinámica de éste para cada uno de los fallos. Esta tesis ilustra este punto sugiriendo que el interfaz entre detección y aislamiento del fallo debe considerar un conjunto de propiedades de dichos señales: propiedad binaria, propiedad del signo, propiedad de la sensibilidad del residuo a un fallo dado, propiedad del orden de aparición de las señales causados por los fallos y la propiedad del tiempo de aparición de estos. Además, como resultado de la influencia de la ganancia del observador en la etapa de detección y en las funciones sensibilidad asociadas a los residuos, esta tesis ilustra que la ganancia del observador tiene también un papel crucial en el módulo de aislamiento, el cual podría permitir mejorar el comportamiento de dicho módulo diseñando éste parámetro del observador de forma adecuada (Ej. Incrementar la distinción de los fallos para su mejor aislamiento). Como último punto, esta tesis analiza la naturaleza temporal de eventos discretos asociada a las señales de fallo generados por el módulo de detección. A consecuencia, se sugiere usar modelos de eventos discretos temporales para modelizar el módulo de aislamiento del fallo. Esta tesis muestra que este tipo de modelos permite mejorar el resultado de aislamiento del fallo. Además, dado que el sistema monitorizado es modelado usando un observador intervalar, esta tesis muestra como este modelo cualitativo de aislamiento puede ser construido usando dicho modelo analítico del sistema. Finalmente, el método propuesto de aislamiento del fallo es aplicado para detectar y aislar fallos en los limnimetros del sistema de alcantarillado de Barcelona. Palabras clave: Detección de Fallos, Diagnosis de Fallos, Robusteza, Observadores, Intervalos, Sistemas de Eventos Discretos

Projecto e implementação de supervisor inteligente em controladores lógicos programáveis

Dissertação para obtenção do Grau de Mestre em Engenharia Electrotécnica e de ComputadoresEsta Tese enquadra-se na área de controlo e automação e tem como principal objectivo o desenvolvimento de um sistema de supervisão e controlo de processos dinâmicos com tolerância a falhas. É apresentada uma abordagem para a sua implementação em Controladores Lógicos Programáveis (PLC). Os processos considerados apresentam características não lineares e são representativos de sistemas de maior dimensão e complexidade, existentes, por exemplo, em meios industriais, em aeronáutica ou em equipamentos médicos. O sistema de supervisão proposto inclui também um sistema de detecção e diagnóstico de falhas (FDD), e enquadra-se na vertente dos métodos activos de controlo tolerante a falhas. O sistema de FDD e o supervisor devem ser capazes de, em conjunto, detectar, identificar e, sempre que possível, accionar mecanismos de correcção/reconfiguração do sistema, quando ocorrem falhas no processo ou nos seus componentes. A abordagem considera, essencialmente, um mecanismo baseado em lógica difusa para gerir a informação recebida do processo, do módulo de diagnóstico e do supervisor humano. Neste trabalho é considerado o processo laboratorial existente no Laboratório de Automação do DEE (Processo “Dois Tanques”). A implementação destas metodologias no processo será efectuada com recurso a Controladores Lógicos Programáveis do modelo M340 da Schneider Electric e deverá permitir a comunicação com controladores remotos implementados noutros PLC’s similares. A implementação proposta para os módulos de supervisão e FDD faz uso de linguagens de programação descritas na norma 61311-3 da IEC (International Electrotechnical Commission) – texto estruturado, diagrama em escada e diagrama de blocos funcionais

On the Combination of Game-Theoretic Learning and Multi Model Adaptive Filters

This paper casts coordination of a team of robots within the framework of game theoretic learning algorithms. In particular a novel variant of fictitious play is proposed, by considering multi-model adaptive filters as a method to estimate other players’ strategies. The proposed algorithm can be used as a coordination mechanism between players when they should take decisions under uncertainty. Each player chooses an action after taking into account the actions of the other players and also the uncertainty. Uncertainty can occur either in terms of noisy observations or various types of other players. In addition, in contrast to other game-theoretic and heuristic algorithms for distributed optimisation, it is not necessary to find the optimal parameters a priori. Various parameter values can be used initially as inputs to different models. Therefore, the resulting decisions will be aggregate results of all the parameter values. Simulations are used to test the performance of the proposed methodology against other game-theoretic learning algorithms.</p

Computational Optimizations for Machine Learning

The present book contains the 10 articles finally accepted for publication in the Special Issue “Computational Optimizations for Machine Learning” of the MDPI journal Mathematics, which cover a wide range of topics connected to the theory and applications of machine learning, neural networks and artificial intelligence. These topics include, among others, various types of machine learning classes, such as supervised, unsupervised and reinforcement learning, deep neural networks, convolutional neural networks, GANs, decision trees, linear regression, SVM, K-means clustering, Q-learning, temporal difference, deep adversarial networks and more. It is hoped that the book will be interesting and useful to those developing mathematical algorithms and applications in the domain of artificial intelligence and machine learning as well as for those having the appropriate mathematical background and willing to become familiar with recent advances of machine learning computational optimization mathematics, which has nowadays permeated into almost all sectors of human life and activity

Advances in Computational Intelligence Applications in the Mining Industry

This book captures advancements in the applications of computational intelligence (artificial intelligence, machine learning, etc.) to problems in the mineral and mining industries. The papers present the state of the art in four broad categories: mine operations, mine planning, mine safety, and advances in the sciences, primarily in image processing applications. Authors in the book include both researchers and industry practitioners

Journal of Telecommunications and Information Technology, 2011, nr 4

kwartalni