Adaptive threshold PCA for fault detection and isolation

Fault diagnosis is an important issue in industrial processes to avoid economic losses, process damage, and to guarantee safe working conditions for the operators. For high scale industrial processes the data-driven based methods are the best solution for process monitoring and fault diagnosis. Thus, in this paper, the principal component analysis is shown to detect and isolate faults. Also, a dynamic threshold is implemented to avoid false alarms because incipient faults are difficult to be detected. As a case of study, the Tennessee Eastman (TE) process is used to apply this strategy because the interaction among five units with internal control loops makes difficult to have an approached model. As results are shown the detection times, for cases where were analyzed incipient faults, the time required for fault detection must be improved, in this work, an adaptive threshold was used to reduce the false alarms but it also increases the detection times. It was concluded that the Q chart gave a better result for fault detection; the isolation times were similar to the detection ones. Two incipient faults could not be detected, the fault detection rate was similar to the shown in literature, but the detection times were better in 35% of the cases, unfortunately for four faults the detection times were bigger than the reported in other papers. It is proposed to help this method with independent component analysis due it is not guaranteed to have a Gaussian distribution in the samples

SensorSCAN: Self-Supervised Learning and Deep Clustering for Fault Diagnosis in Chemical Processes

Modern industrial facilities generate large volumes of raw sensor data during the production process. This data is used to monitor and control the processes and can be analyzed to detect and predict process abnormalities. Typically, the data has to be annotated by experts in order to be used in predictive modeling. However, manual annotation of large amounts of data can be difficult in industrial settings. In this paper, we propose SensorSCAN, a novel method for unsupervised fault detection and diagnosis, designed for industrial chemical process monitoring. We demonstrate our model's performance on two publicly available datasets of the Tennessee Eastman Process with various faults. The results show that our method significantly outperforms existing approaches (+0.2-0.3 TPR for a fixed FPR) and effectively detects most of the process faults without expert annotation. Moreover, we show that the model fine-tuned on a small fraction of labeled data nearly reaches the performance of a SOTA model trained on the full dataset. We also demonstrate that our method is suitable for real-world applications where the number of faults is not known in advance. The code is available at https://github.com/AIRI-Institute/sensorscan

A multi-category decision support framework for the Tennessee Eastman problem

The paper investigates the feasibility of developing a classification framework, based on support vector machines, with the correct properties to act as a decision support system for an industrial process plant, such as the Tennessee Eastman process. The system would provide support to the technicians who monitor plants by signalling the occurrence of abnormal plant measurements marking the onset of a fault condition. To be practical such a system must meet strict standards, in terms of low detection latency, a very low rate of false positive detection and high classification accuracy. Experiments were conducted on examples generated by a simulation of the Tennessee Eastman process and these were preprocessed and classified using a support vector machine. Experiments also considered the efficacy of preprocessing observations using Fisher Discriminant Analysis and a strategy for combining the decisions from a bank of classifiers to improve accuracy when dealing with multiple fault categories

마르코프 랜덤 필드 학습 및 추론과 그래프 라쏘를 활용한 공정 이상 감지 및 진단 방법론

학위논문 (박사)-- 서울대학교 대학원 : 공과대학 화학생물공학부, 2019. 2. 이원보.Fault detection and diagnosis (FDD) is an essential part of safe plant operation. Fault detection refers to the process of detecting the occurrence of a fault quickly and accurately, and representative methods include the use of principal component analysis (PCA), and autoencoders (AE). Fault diagnosis is the process of isolating the root cause node of the fault, then determining the fault propagation path to identify the characteristic of the fault. Among the various methods, data-driven methods are the most widely-used, due to their applicability and good performance compared to analytical and knowledge-based methods. Although many studies have been conducted regarding FDD, no methodology for conducting every step of FDD exists, where the fault is effectively detected and diagnosed. Moreover, existing methods have limited applicability and show limited performance. Previous fault detection methods show loss of variable characteristics in dimensionality reduction methods and have large computational loads, leading to poor performance for complex faults. Likewise, preceding fault diagnosis methods show inaccurate fault isolation results, and biased fault propagation path analysis as a consequence of implementing knowledge-based characteristics for construction of digraphs of process variable relationships. Thus a comprehensive methodology for FDD which shows good performance for complex faults and variable relationships, is required. In this study, an efficient and effective comprehensive FDD methodology based on Markov random fields (MRF) modelling is proposed. MRFs provide an effective means for modelling complex variable relationships, and allows efficient computation of marginal probability of the process variables, leading to good performance regarding FDD. First, a fault detection framework for process variables, integrating the MRF modelling and structure learning with iterative graphical lasso is proposed. Graphical lasso is an algorithm for learning the structure of MRFs, and is applicable to large variable sets since it approximates the MRF structure by assuming the relationships between variables to be Gaussian. By iteratively applying the graphical lasso to monitored variables, the variable set is subdivided into smaller groups, and consequently the computational cost of MRF inference is mitigated allowing efficient fault detection. After variable groups are obtained through iterative graphical lasso, they are subject to the MRF monitoring framework that is proposed in this work. The framework obtains the monitoring statistics by calculating the probability density of the variable groups through kernel density estimation, and the monitoring limits are obtained separately for each group by using a false alarm rate of 5%. Second, a fault isolation and propagation path analysis methodology is proposed, where the conditional marginal probability of each variable is computed via inference, then is used to calculate the conditional contribution of individual variables during the occurrence of a fault. Using the kernel belief propagation (KBP) algorithm, which is an algorithm for learning and inferencing MRFs comprising continuous variables, the parameters of MRF are trained using normal process data, then the individual conditional contribution of each variable is calculated for every sample of the fault process data. By analyzing the magnitude and reaction speed of the conditional contribution of individual variables, the root fault node can be isolated and the fault propagation path can be determined effectively. Finally, the proposed methodology is verified by applying it to the well-known Tennessee Eastman process (TEP) model. Since the TEP has been used as a benchmark process over the past years for verifying various FDD methods, it serves the purpose of performance comparison. Also, since it consists of multiple units and has complex variable relationships such as recycle loops, it is suitable for verifying the performance of the proposed methodology. Application results show that the proposed methodology performs better compared to state-of-the-art FDD algorithms, in terms of both fault detection and diagnosis. Fault detection results showed that all 28 faults designed inside the TEP model were detected with a fault detection accuracy of over 95%, which is higher than any other previously proposed fault detection method. Also, the method showed good fault isolation and propagation path analysis results, where the root-cause node for every fault was detected correctly, and the characteristics of the initiated faults were identified through fault propagation path analysis.공정 이상의 감지 및 진단 시스템은 안전한 공정 운영에 필수적인 부분이다. 이상 감지는 이상이 발생했을 경우 즉각적으로 이를 정확하게 감지하는 프로세스를 의미하며, 대표적인 방법으로는 주성분 분석 및 오토인코더를 활용한 감지 방법론이 있다. 이상 진단은 결함의 근본 원인이 되는 노드를 격리하고, 이상의 전파 경로를 탐지하여 이상의 특성을 식별하는 프로세스이다. 공정 이상의 감지 및 진단 방법론에는 모델 분석 방법론, 지식 기반 방법론 등의 다양한 방법론이 있지만, 공정에 대한 적용 가능성과 성능 측면에서 가장 유용하다고 알려져 있는 데이터 기반 방법론이 널리 활용되고 있다. 공정 이상의 감지 및 진단에 대한 데이터 기반 방법론은 다방면으로 연구되어 왔지만, 이상 감지 및 진단을 모두 효과적으로 수행할 수 있는 방법론은 소수에 불과하며, 존재하고 있는 방법론들 역시 두 분야 모두에서 좋은 성능을 보여주고 있는 경우는 없다. 이는 기존 방법론들의 적용 가능성이 제한되어 있으며 공정에 적용시 제한된 성능을 보여주기 때문이다. 이상 감지의 경우, 대용량의 데이터를 처리할 때 발생하는 과부하로 인한 감지 능력의 저하, 차원 축소 방법론들을 사용할 시 이에 따른 변수 특성 반영의 부정확성, 그리고 축소된 차원에서의 계산으로 인하여 복합적인 형태의 이상을 감지해 내지 못하는 문제 등이 있다. 이상 진단의 경우 이상의 원인이 되는 노드의 격리 및 이상 전파 경로에 대한 분석이 부정확한 경우가 많은데, 이는 차원 축소로 인하여 공정 변수의 특성이 소실되는 성질이 있고, 방향성 그래프를 활용할 시 공정에 대한 선행 지식을 적용함으로써 편향된 이상 진단 결과가 나타나는 경우들이 발생하기 때문이다. 기존 방법론들에 대한 이러한 한계점들을 고려해 봤을때, 변수 각각의 특성이 소실되지 않도록하여 효과적으로 이상에 대한 감지와 진단을 모두 수행해 낼 수 있으면서도, 계산상의 효율성을 갖춘, 이상 감지 및 진단에 대한 통합된 방법론의 개발이 시급하다고 할 수 있다. 본 연구에서는 마르코프 랜덤 필드 모델링과 그래프 라쏘를 기반으로하여, 이상에 대한 감지 및 진단을 모두 수행해 낼 수 있는 통합적인 공정 모니터링 방법론을 제안한다. 마르코프 랜덤 필드는 비선형적이고 비정규적인 변수 관계를 효과적으로 모델링할 수 있게 해주고, 이상 발생 상황에서의 모니터링 통계값 계산시에 각 변수의 특성을 반영하여 확률 계산을 해 낼 수 있기 때문에 효과적인 이상 감지 및 진단 수단이 된다. 기본적으로 마르코프 랜덤 필드는 확률값 계산시의 부하가 크지만, 본 연구에서는 그래프 라쏘 방법론을 추가적으로 함께 활용하여 계산 상의 부하를 줄이고 효율적으로 이상 감지 및 진단을 해낼 수 있도록 하였다. 본 연구에서 제안된 내용들은 다음과 같다. 첫째, 공정 변수를 마르코프 랜덤 필드 형태로 모델링하고, 그래프 라쏘를 활용해 마르코프 랜덤 필드의 구조를 얻을 수 있는 방법론을 제시하였다. 그래프 라쏘는 마르코프 랜덤 필드의 구조를 파악하기 위한 방법론인데, 변수 간의 관계를 가우스 함수의 형태로 가정하기 때문에 다변수 시스템에서도 효율적으로 그래프 구조를 파악할 수 있도록 해준다. 본 연구에서는 반복적 그래프 라쏘를 제안하여 모든 공정 변수들이 상관관계가 높은 변수 집단으로 묶일 수 있도록 하였다. 이를 활용하면 전체 공정 변수 집단을 다수의 소집단으로 분류하고 각각에 대한 그래프 구조를 파악할 수 있게 되는데, 크게 두 가지의 효과를 얻을 수 있다. 우선적으로 마르코프 랜덤 필드 확률 계산의 대상이 되는 변수의 개수를 줄여줌으로써 계산 부하를 줄이고 효율적인 이상 감지가 이루어질 수 있도록 한다. 또한 상관관계가 높은 집단끼리 묶여서 모델링 된 그래프를 활용하여 이상의 진단 과정에서 공정 변수 간의 관계 파악 및 전파 경로 분석을 용이하도록 해준다. 두 번째로, 마르코프 랜덤 필드의 확률 추론을 기반으로 하여 효과적으로 이상 감지가 이루어질 수 있도록 하는 방법론을 제안하였다. 반복적 그래프 라쏘를 통해 얻어진 다수의 변수 소집단에 대하여 각각 확률 추론을 적용하여 이상 감지를 진행하게 되는데, 제안된 방법론에서는 커널 밀도 추정 방법론을 활용하였다. 정상 데이터를 활용하여 각 변수들에 대한 커널 밀도의 대역폭을 학습하고, 이상 데이터가 발생할 시 이를 활용한 커널 밀도 추정법을 사용하여 이상감시 통계치를 계산하게 된다. 이때 허위 진단율을 5%로 가정하여 각각의 소집단에 대한 공정 감지 기준선을 설정하였고, 이상감시 통계치가 공정 감시 기준선보다 낮게 될 경우 이상이 감지된다. 세 번째로, 이상 발생 시 원인이 되는 변수의 격리 및 이상 전파 경로 분석을 효과적으로 수행할 수 있는 방법론을 제시하였다. 제시된 방법론에서는 마르코프 랜덤 필드의 확률 추론 과정을 활용하여 이상 발생 시 각 변수의 조건부 한계 확률을 계산하고, 이를 활용해 새롭게 정의된 조건부 기여도 값을 계산하여, 이상에 대한 각 변수의 기여도를 파악할 수 있도록 한다. 이 과정에서는 커널 신뢰도 전파 방법론이 사용되는데, 이는 연속 변수를 가지는 마르코프 랜덤 필드에 대하여 확률 추론을 수행할 수 있도록 하는 방법론이다. 커널 신뢰도 전파법을 사용하면 정상 상태의 공정 데이터를 활용하여 마르코프 랜덤 필드를 구성하는 파라미터 값들을 학습하고, 이상 발생시 이상 데이터에 대하여 각 변수의 조건부 기여도 값을 계산할 수 있게 된다. 이 때 계산된 조건부 기여도 값의 크기와, 이상 발생 이후 각 변수의 조건부 기여도 값의 변화 반응 속도를 종합적으로 판단하여, 이상의 원인 변수에 대한 격리와 이상 전파 경로 분석을 효과적으로 수행할 수 있게 된다. 본 연구에서는 제안된 이상 감지 및 진단 방법론의 성능을 검증하기 위하여 테네시 이스트만 공정 모델에 이를 적용하고 결과를 분석하였다. 테네시 이스트만 공정은 수년간 공정 감시 방법론을 검증하기 위한 벤치마크 공정으로 널리 사용되어 왔기 때문에, 제시된 방법론을 이에 적용해 봄으로써 다른 공정 감시 방법론들과의 성능을 비교해 볼 수 있었다. 또한 다수의 단위 공정을 포함하고 있고, 순환적인 변수 관계 역시 포함하고 있기 때문에 제시된 방법론의 성능을 시험해 보기에 적합했다. 테네시 이스트만 공정 내부에는 28개 종류의 이상이 프로그램 상에 내장되어 있는데, 제시된 공정 감지 방법론을 적용한 결과 모든 이상에 대하여 96% 이상의 높은 이상 감지율을 나타내었다. 이는 기존에 제시된 공정 감시 방법론들에 비하여 월등히 높은 수치였다. 또한 이상 진단 성능을 분석해 본 결과, 모든 이상에 대하여 원인이 되는 노드를 효과적으로 파악할 수 있었고, 이상 전파 경로 역시 정확하게 탐지하여 기존 방법론들과는 차별화된 성능을 나타내었다. 제시된 방법론을 테네시 이스트만 공정에 적용해 봄으로써, 본 연구 내용이 공정 이상의 감지 및 진단에 대한 통합적인 방법론 중에서 가장 우수한 성능을 나타내는 것을 확인할 수 있었다.Contents Abstract i Contents iv List of Tables vii List of Figures ix 1 Introduction 1 1.1 Research Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Research Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 1.3 Outline of the Thesis . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2 Markov Random Fields Modelling, Graphical Lasso, and Optimal Structure Learning 10 2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.2 Markov Random Fields . . . . . . . . . . . . . . . . . . . . . . . . . 12 2.3 Graphical Lasso . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 2.4 MRF Modelling & Structure Learning . . . . . . . . . . . . . . . . . 19 2.4.1 MRF modelling in process systems . . . . . . . . . . . . . . 19 2.4.2 Structure learning using iterative graphical lasso . . . . . . . 20 2.5 Application of Iterative Graphical Lasso on the TEP . . . . . . . . . . 24 3 Efficient Process Monitoring via the Integrated Use of Markov Random Fields Learning and the Graphical Lasso 31 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 3.2 MRF Monitoring Integrated with Graphical Lasso . . . . . . . . . . . 35 3.2.1 Step 1: Iterative graphical lasso . . . . . . . . . . . . . . . . 36 3.2.2 Step 2: MRF monitoring . . . . . . . . . . . . . . . . . . . . 36 3.3 Implementation of Glasso-MRF monitoring to the Tennessee Eastman process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 3.3.1 Tennessee Eastman process . . . . . . . . . . . . . . . . . . 41 3.3.2 Glasso-MRF monitoring on TEP . . . . . . . . . . . . . . . . 48 3.3.3 Fault detection accuracy comparison with other monitoring techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 3.3.4 Fault detection speed & fault propagation . . . . . . . . . . . 95 4 Process Fault Diagnosis via Markov Random Fields Learning and Inference 101 4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 4.2 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 4.2.1 Probabilistic graphical models & Markov random fields . . . 106 4.2.2 Kernel belief propagation . . . . . . . . . . . . . . . . . . . . 107 4.3 Fault Diagnosis via MRF Modeling . . . . . . . . . . . . . . . . . . 113 4.3.1 MRF structure learning via graphical lasso . . . . . . . . . . 116 4.3.2 Kernel belief propagation - bandwidth selection . . . . . . . . 116 4.3.3 Conditional contribution evaluation . . . . . . . . . . . . . . 117 4.4 Application Results & Discussion . . . . . . . . . . . . . . . . . . . 118 4.4.1 Two tank process . . . . . . . . . . . . . . . . . . . . . . . . 119 4.4.2 Tennessee Eastman process . . . . . . . . . . . . . . . . . . 137 5 Concluding Remarks 152 Bibliography 157 Nomenclature 169 Abstract (In Korean) 170Docto

Identifying and Detecting Attacks in Industrial Control Systems

The integrity of industrial control systems (ICS) found in utilities, oil and natural gas pipelines, manufacturing plants and transportation is critical to national wellbeing and security. Such systems depend on hundreds of field devices to manage and monitor a physical process. Previously, these devices were specific to ICS but they are now being replaced by general purpose computing technologies and, increasingly, these are being augmented with Internet of Things (IoT) nodes. Whilst there are benefits to this approach in terms of cost and flexibility, it has attracted a wider community of adversaries. These include those with significant domain knowledge, such as those responsible for attacks on Iran’s Nuclear Facilities, a Steel Mill in Germany, and Ukraine’s power grid; however, non specialist attackers are becoming increasingly interested in the physical damage it is possible to cause. At the same time, the approach increases the number and range of vulnerabilities to which ICS are subject; regrettably, conventional techniques for analysing such a large attack space are inadequate, a cause of major national concern. In this thesis we introduce a generalisable approach based on evolutionary multiobjective algorithms to assist in identifying vulnerabilities in complex heterogeneous ICS systems. This is both challenging and an area that is currently lacking research. Our approach has been to review the security of currently deployed ICS systems, and then to make use of an internationally recognised ICS simulation testbed for experiments, assuming that the attacking community largely lack specific ICS knowledge. Using the simulator, we identified vulnerabilities in individual components and then made use of these to generate attacks. A defence against these attacks in the form of novel intrusion detection systems were developed, based on a range of machine learning models. Finally, this was further subject to attacks created using the evolutionary multiobjective algorithms, demonstrating, for the first time, the feasibility of creating sophisticated attacks against a well-protected adversary using automated mechanisms

Fault diagnosis in industrial process by using LSTM and an elastic net

[EN] Fault diagnosis is important for industrial processes because it permits to determine the necessity of emergency stops in a process and/or to propose a maintenance plan. Two strategies for fault diagnosis are compared in this work. On the one hand, the data are preprocessed using the independent components analysis for dimension reduction, then the wavelet transform is used in order to highlight the faulty signals, with this information an artificial neural network was fed. On the other hand, the second strategy, the main contribution of this work, is the implementation of a long short term memory. This memory is fed with the most representative variables selected by an elastic net to use both, the L1 and L2 norms. These strategies are applied in the Tennessee Eastman process, a benchmark widely used for fault diagnosis. The fault isolation had better results than those reported in the literature.[ES] El diagnóstico de fallas es importante en los procesos industriales, ya que permite determinar si es necesario detener el proceso en operación y/o proponer un plan de mantenimiento. En el presente trabajo se comparan dos estrategias para diagnosticar fallas. La primera realiza un preprocesamiento de datos usando el análisis de componentes independientes para reducir la dimensión de los datos, posteriormente, se emplea la transformada wavelet para resaltar las señales de falla, con esta información se alimenta una red neuronal artificial. Por su parte, la segunda estrategia, principal contribución de este trabajo, usa una memoria de corto y largo plazo. Esta memoria es alimentada por las variables más significativas seleccionadas mediante una red elástica para usar tanto la norma $L_1$ como la $L_2$. Como ejemplo de aplicación se utilizó el proceso químico Tennessee Eastman, un proceso ampliamente usado en el diagnóstico de fallas. El aislamiento de fallas mostró mejores resultados con respecto a los reportados en la literatura.Márquez-Vera, MA.; López-Ortega, O.; Ramos-Velasco, LE.; Ortega-Mendoza, RM.; Fernández-Neri, BJ.; Zúñiga-Peña, NS. (2021). Diagnóstico de fallas mediante una LSTM y una red elástica. Revista Iberoamericana de Automática e Informática industrial. 18(2):164-175. https://doi.org/10.4995/riai.2020.13611OJS164175182Adewole, A., Tzoneva, R., Behardien, S., 2016. Distribution network fault section identification and fault location using wavelet entropy and neural networks. Applied Soft Computing 46, 296-306. https://doi.org/10.1016/j.asoc.2016.05.013Alkaya, A., Eker, I., 2011. Variance sensitive adaptive threshold-based PCA method for fault detection with experimental application. ISA Transactions 50, 287-302. https://doi.org/10.1016/j.isatra.2010.12.004Barakat, S., Eteiba, M., Wahba, W., 2014. Fault location in underground cables using anfis nets and discrete wavelet transform. Journal of Electrical Systems and Information Technology 1, 198-211. https://doi.org/10.1016/j.jesit.2014.12.003Bathelt, A., Ricker, N., Jelali, M., 2015. Revision of the Tennessee Eastman process model. IFAC Papers-Online 48 (8), 309-314. https://doi.org/10.1016/j.ifacol.2015.08.199Boldt, F., Rauber, T., Varejao, F., October 2014. Evaluation of the extreme learning machine for automatic fault diagnosis of the Tennessee Eastman chemical process. In: IEEE (Ed.), Annual Conference of the IEEE Industrial Electronics Society. Vol. 40. Dallas, Texas, pp. 2551-2557. https://doi.org/10.1109/IECON.2014.7048865Chen, H., Tino, P., Yao, X., 2014. Cognitive fault diagnosis in Tennessee Eastman process using learning in the model space. Computers and Chemical Engineering 67, 33-42. https://doi.org/10.1016/j.compchemeng.2014.03.015Rodrigues, J., Filho, P., PeixotoJr., E., Kumar, A., deAlbuquerque, V., 2019. Classification of EEG signals to detect alcoholism using machine learning techniques. Pattern Recognition Letters 125, 140-149. https://doi.org/10.1016/j.patrec.2019.04.019Dixit, A., Majumdar, S., 2013. Comparative analysis of coiflet and daubechies wavelets using global threshold for image denoising. Intenational Journal of Advances in Engineering & Technology 6 (5), 2247-2252.Downs, J., Vogel, E., 1993. A plant-wide industrial process control problem. Computers and Chemical Engineering 17 (3), 245-255. https://doi.org/10.1016/0098-1354(93)80018-IFischer, T., Krauss, C., 2018. Deep learning with long short-term memory networks for financial market predictions. European Journal of Operational Research 270, 654-669. https://doi.org/10.1016/j.ejor.2017.11.054Gao, X., Hou, J., 2016. An improved SVM integrated GS-PCA fault diagnosis approach of Tennessee Eastman process. Neurocomputing 174, 906-911. https://doi.org/10.1016/j.neucom.2015.10.018Geng, Z., Li, Z., Han, Y., 2018. A new deep belief network based on RBM with glial chains. Information Sciences 463, 294-306. https://doi.org/10.1016/j.ins.2018.06.043Goodfellow, I., Bengio, Y., Courville, A., 2016. Deep Learning. MIT Press, United States of America, http://www.deeplearningbook.ogr.Han, L., Li, C., Guo, S., Su, X., 2015. Feature extraction method of bearing AE signal based on improved Fast-ICA and wavelet packet energy. Mechanical Systems and Signal Processing 62-63, 91-99. https://doi.org/10.1016/j.ymssp.2015.03.009Hastie, T., Tibshirani, R., Friedman, J., 2009. The elements of statistical learning: data mining, inference and prediction. Springer, New York. https://doi.org/10.1007/978-0-387-84858-7Hoang, D., Kang, H., 2019. A survey on deep learning based bearing fault diagnosis. Neurocomputing 335, 327-335. ttps://doi.org/10.1016/j.neucom.2018.06.078Hochreiter, S., Schmidhuber, J., 1997. Long short term memory. Neural Computation 9 (8), 1735-1780. ttps://doi.org/10.1162/neco.1997.9.8.1735Hyvärinen, A., Oja, E., 2000. Independent component analysis: Algorithms and applications. Neural Networks 13, 411-430. ttps://doi.org/10.1016/S0893-6080(00)00026-5Jing, C., Gao, X., Zhu, X., Lang, S., July 2014. Fault classificaction on Tennessee Eastman process: PCA and SVM. In: IEEE (Ed.), Intenational Conference on Mecatronics and Control. Jinzhou, China, pp. 2194-2197. https://doi.org/10.1109/ICMC.2014.7231958Jung, C., Kim, K., Lee, J., Klockl, B., 2007. Wavelet and neuro-fuzzy based fault location for combined transmission systems. Energy Systems 29, 445-454. https://doi.org/10.1016/j.ijepes.2006.11.003Kandula, V. K., 2011. Fault detection in process control plants using principal component analysis. Master's thesis, Louisiana State University, Department of Electrical Engineering.Karpenko, M., Sepehri, N., Octubre 2001. A neural network based fault detection and identification scheme for pneumatic process control valves. In: IEEE (Ed.), International Conference on Systems, Man and Cybernetics. Tucson, USA, pp. 93-98. https://doi.org/10.1109/ICSMC.2001.969794Khakipour, M., Safavi, A., Setoodeh, P., 2017. Bearing fault diagnosis with morphological gradient wavelet. Journal of the Franklin Institute 354, 2465-2476. https://doi.org/10.1016/j.jfranklin.2016.11.013Kuang, T., Yang, Z., Yao, Y., 2015. Multivariate fault isolation via variable selection in discriminant analysis. Journal of Process Control 35, 30-40. https://doi.org/10.1016/j.isatra.2017.06.014Kumar, R., Bansal, H., 2019. Hardware in the loop implementation of wavelet based strategy in shuntactive powerfilter to mitigate power quality issues. Electric Power Systems Research 169, 92-104. https://doi.org/10.1016/j.epsr.2019.01.001Lau, C., Ghosh, K., Hussain, M., Hassan, C. C., 2013. Fault disgnosis of Tennessee Eastman process with multi-scale PCA and ANFIS. Chemom. Intell. Lab. Syst. 120, 1-14. https://doi.org/10.1016/j.chemolab.2012.10.005Lee, J., Yoo, C., Lee, I., 2004. Statistical process monitoring with independent component analysis. Journal of Process Control 14 (5), 467-485. https://doi.org/10.1016/j.jprocont.2003.09.004Lei, J., Liu, C., Jiang, D., 2019. Fault diagnosis of wind turbine based on long short-term memory networks. Renewable Energy 133, 422-432. https://doi.org/10.1016/j.renene.2018.10.031Li, W., Monti, A., Ponci, F., 2014. Fault detection and classification in medium voltage DC shipboard power systems with wavelets and artificial neural networks. IEEE Transactions on Instrumentation and Measurement 63 (11), 2651-2665. https://doi.org/10.1109/TIM.2014.2313035Liang, P., Deng, C.,Wu, J., Yang, Z., Zhu, J., Zhang, Z., 2019. Compound fault diagnosis of gearboxes via multi-label convolutional neural network and wavelet transform. Computers in Industry 113, 103132. https://doi.org/10.1016/j.compind.2019.103132Lin, J., Zhang, A., 2005. Fault feature separation using wavelet-ICA filter. NDT&E International 38, 421-427. https://doi.org/10.1016/j.ndteint.2004.11.005Linker, R., Gutman, P., Seginer, I., 2002. Observer-based robust failure detection and isolation in greenhouses. Control Engineering Practice 10 (5), 519- 531. https://doi.org/10.1016/S0967-0661(02)00002-3Lou, W., Loparo, K., 2004. Bearing fault diagnosis based on wavelet transform and fuzzy inference. Mechanical Systems and Signal Processing 18, 1077-1095. https://doi.org/10.1016/S0888-3270(03)00077-3Lv, F.,Wen, C., Bao, Z., Liu, M., 2016. Fault diagnosis based on deep learning. In: AACC (Ed.), American Control Conference. Boston, USA, pp. 6851-6856. https://doi.org/10.1109/ACC.2016.7526751Lv, F., Wen, C., Liu, M., Bao, Z., 2017. Weighted time series fault diagnosis based on a staked sparce autoencoder. Journal of Chemometrics 31, 16 pages. https://doi.org/10.1002/cem.2912Lv, F., Fan, X., Wen, C., Bao, Z., 2018. Stacked sparse auto encoder network based multimode process monitoring. In: IEEE (Ed.), International Conference on Control Automation & Information Science. Hangzhou, China, pp. 227-232. https://doi.org/10.1109/ICCAIS.2018.8570618Maglaveras, N., Stamkopoulos, T., Diamantaras, K., Pappas, C., Strintzis, M., 1998. ECG pattern recognition and classification using non-linear transfor mations and neural networks: A review. International Journal of Medical Informatics 52, 191-208. https://doi.org/10.1016/S1386-5056(98)00138-5Methnani, S., Lafont, F., Gautier, J., Damak, T., Toumi, A., 2013. Actuator and sensor fault detection, isolation and identification in nonlinear dynamical systems, with applications to a waste water treatment plant. Journal of Computer Engineering and Informatics 1 (4), 112-125. https://doi.org/10.1080/21642583.2014.888525Muñoz-Cobo, J., Mendizábal, R., Miquel, A., Berna, C., Escrivá, A., 2017. Use of the principles of maximum entropy and maximum relative entropy for the determination of uncertain parameter distributions in engineering applications. Entropy 19, 486, 37 pages. https://doi.org/10.3390/e19090486Nguyen, B., Quyen, A., Nguyen, P., Ton, T., July 2017. Wavelet-based neural network for recognition of faults at nhabe power substation of the vietnam power system. In: IEEE (Ed.), International Conference on System Science and Engineering. Ho Chi Minh City, Vietnam, pp. 165-168. https://doi.org/10.1109/ICSSE.2017.8030858Ojeda-González, A., Mendes-Jr., O., Oliveira-Domingues, M., Menconi, V., 2014. Daubechies wavelet coeffcients: a tool to study interplanetary magnetic field fluctuations. Geof'ısica Internacional 53 (2), 101-115. https://doi.org/10.1016/S0016-7169(14)71494-1Oliveira, J., Pontes, K., Santori, I., Embirucu, M., 2017. Fault detection and diagnosis in dynamic systems using weightless neural networks. Expert Systems With Applications 84, 200-219. https://doi.org/10.1016/j.eswa.2017.05.020Patan, K., 2008. Artificial neural networks for the modelling and fault diagnosis of technical process. Lecture Notes in Control and Information Sciences. Springer, India.Rafiee, J., Rafiee, M., Tse, P., 2010. Application of mother wavelet functions for automatic gear and bearing fault diagnosis. Expert Systems with Applications 37, 4568-4579. https://doi.org/10.1016/j.eswa.2009.12.051Ramos-Velasco, L., Ramos-Fernández, J., Islar-Gómez, O., Espejel-Rivera, M., García-Lamont, J., Márquez-Vera, M., 2013. Identificación y control wavenet de un motor de ca. Revista Iberoamericana de Automática e Informática Industrial 10, 269-278. https://doi.org/10.1016/j.riai.2013.05.002Rato, T., Reis, M., 2013. Defining the structure of DPCA models and its impact on process monitoring and prediction ctivities. Chemometrics and Intelligent Laboratory Systems 125, 74-86. https://doi.org/10.1016/j.chemolab.2013.03.009Rockinger, M., Jondeau, E., 2002. Entropy densities with an application to autoregressive conditional skewness and kurtosis. Journal of Econometrics 106, 119-142. https://doi.org/10.1016/S0304-4076(01)00092-6Salahschoor, K., Kiasi, F., July 2008. On-line process monitoring based on wavelet-ICA methodology. In: IFAC (Ed.), Proceedings of the 17th World Congress. Seul- Korea, pp. 6-11. https://doi.org/10.3182/20080706-5-KR-1001.01253Salahshoor, K., Khoshro, M., Kordestani, M., 2011. Fault detection and diagnosis of an industrial steam turbine using a distributed configuration of adaptive neuro-fuzzy inference systems. Simulation Modelling Practice and Theory 19, 1280-1293. https://doi.org/10.1016/j.simpat.2011.01.005Sharif, I., Khare, S., 2014. Comparative analysis of Haar and Daubechies wavelet for hyper spectral image classification. In: Commission, I. T. (Ed.), VIII Symposium of The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Science. Hyderabad-India, pp. 937-941. https://doi.org/10.5194/isprsarchives-XL-8-937-2014Smirnov, E., Timoshenko, D., Adrianov, S., 2014. Comparison of regularization methods for imagenet classification with deep convolutional neural networks. AASRI Procedia 6, 89-94. https://doi.org/10.1016/j.aasri.2014.05.013Sobhani-Tehrani, E., Khorasani, K., 2009. Fault diagnosis of nonlinear systems using a hybrid approach. Fault detetion and diagnosis. Springer, Berlin, Ch. 2, pp. 22-49. https://doi.org/10.1007/978-0-387-92907-1_2Tayarani-Bathaie, S., Vanini, Z., Khorasani, K., 2014. Dynamic neural networkbased fault diagnosis of gas turbine engines. Neurocomputing 125, 153-165. https://doi.org/10.1016/j.neucom.2012.06.050Zvokelj, M., Zupan, S., Prebil, I., 2016. EEMD-based multiscale ICA method for slewing bearing fault detection and diagnosis. Journal of Sound and Vibration 26, 394-423. https://doi.org/10.1016/j.jsv.2016.01.046Wang, X., Qin, Y., Wang, Y., Xiang, S., Chen, H., 2019. ReLTanh: An activation function with vanishing gradient resistance for SAE-based DNNs and its application to rotating machinery fault diagnosis. Neurocomputing 363, 88-98. https://doi.org/10.1016/j.neucom.2019.07.017Wu, F., Tong, F., Yang, Z., 2016. EMGdi signal enhancement based on ICA decomposition and wavelet transform. Applied Soft Computing 43, 561-571. https://doi.org/10.1016/j.asoc.2016.03.002Wu, J., Hsu, C., Wu, G., 2009. Fault gear identification and classification using discrete wavelet transform and adaptive neuro-fuzzy inference. Expert Systems with Applications 36, 6244-6255. https://doi.org/10.1016/j.eswa.2008.07.023Wu, Q., Law, R., Wu, S., 2011. Fault diagnosis of car assembly line based on fuzzy wavelet kernel support vector classifier machine and modified genetic algorithm. Expert Systems with Applications 38, 9096-9104. https://doi.org/10.1016/j.eswa.2010.12.109Wu, H., Zhao., Jinsong, 2018. Deep convolutional neural network model based chemical process fault diagnosis. Computers and Chemical Engineering 115, 185-197. https://doi.org/10.1016/j.compchemeng.2018.04.009Xiao, C., Chen, N., Hu, C., Wang, K., Gong, J., Chen, Z., 2019. Short and midterm sea surface temperature prediction using time-series satellite data and LSTM-AdaBoost combination approach. Remote Sensing of Environment 233, 111358. https://doi.org/10.1016/j.rse.2019.111358Xie, D., Bai, L., December 2015. A hierarchical deep neural network for fault diagnosis on Tennessee-Eastman process. In: IEEE (Ed.), International Conference on Machine Learning and Applications. Vol. 14. Miami, USA, pp. 745-748. https://doi.org/10.1109/ICMLA.2015.208Yan, R., Gao, R., Chen, X., 2014. Wavelets for fault diagnosis of rotary machines: A review with applications. Signal Processing 351, 4555-4569. https://doi.org/10.1016/j.sigpro.2013.04.015Yan, Z., Yao, Y., 2015. Variable selection method for fault isolation using least absolute shrinkage and selection operator (LASSO). Chemometrics and Intelligent Laboratory Systems 146, 136-146. https://doi.org/10.1016/j.chemolab.2015.05.019Yao, G., Lei, T., Zhong, J., 2019. A review of convolutional-neural-networkbased action recognition. Pattern Recognition Letters 118, 14-22. https://doi.org/10.1016/j.patrec.2018.05.018Yin, S., Ding, S., Haghani, A., Hao, H., Zhang, P., 2012. A comparison study of basic data-driven fault diagnosis and process monitoring methods on the benchmark Tennessee Eastman process. Journal of Process Control 22, 1567-1581. https://doi.org/10.1016/j.jprocont.2012.06.009Zhang, Q., Yang, L., Chen, Z., Li, P., 2018. A survey on deep learning for big data. Information Fusion 42, 146-157. https://doi.org/10.1016/j.inffus.2017.10.006Zhang, X., Polycarpou, M., Parisini, T., 2002. A robust detection and isolation scheme for abrupt and incipient faults in nonlinear systems. IEEE Transactions on Automatic Control 47 (4), 576-593. https://doi.org/10.1109/9.995036Zhang, Y., Zhang, L., Zhang, H., 2012. Fault detection for industrial processes. Mathematical Problems in Engineering 2012, 18 pages. https://doi.org/10.1155/2012/757828Zhang, Z., Zhao, J., 2017. A deep belief network based fault diagnosis model for complex chemical process. Computers and Chemical Engineering 107, 395-407. https://doi.org/10.1016/j.compchemeng.2017.02.041Zhao, H., 2018. Neural component analysis for fault detection. Chemometrics and Intelligent Laboratory Systems 176, 11-21. https://doi.org/10.1016/j.chemolab.2018.02.001Zhao, R., Yan, R., Chen, Z., Mao, K., Wang, P., Gao, R., 2019. Deep learning and its applications to machine health monitoring. Mechanical Systems and Signal Processing 115, 213-237. https://doi.org/10.1016/j.ymssp.2018.05.050Zheng, J., Huang, W., Wang, Z., Liang, J., 2019. Mutual information-based sparse multiblock dissimilarity method for incipient fault detection and diagnosis in plant-wide process. Journal of Process Control 83, 63-76. https://doi.org/10.1016/j.jprocont.2019.09.00

Data driven methods for updating fault detection and diagnosis system in chemical processes

Modern industrial processes are becoming more complex, and consequently monitoring them has become a challenging task. Fault Detection and Diagnosis (FDD) as a key element of process monitoring, needs to be investigated because of its essential role in decision making processes. Among available FDD methods, data driven approaches are currently receiving increasing attention because of their relative simplicity in implementation. Regardless of FDD types, one of the main traits of reliable FDD systems is their ability of being updated while new conditions that were not considered at their initial training appear in the process. These new conditions would emerge either gradually or abruptly, but they have the same level of importance as in both cases they lead to FDD poor performance. For addressing updating tasks, some methods have been proposed, but mainly not in research area of chemical engineering. They could be categorized to those that are dedicated to managing Concept Drift (CD) (that appear gradually), and those that deal with novel classes (that appear abruptly). The available methods, mainly, in addition to the lack of clear strategies for updating, suffer from performance weaknesses and inefficient required time of training, as reported. Accordingly, this thesis is mainly dedicated to data driven FDD updating in chemical processes. The proposed schemes for handling novel classes of faults are based on unsupervised methods, while for coping with CD both supervised and unsupervised updating frameworks have been investigated. Furthermore, for enhancing the functionality of FDD systems, some major methods of data processing, including imputation of missing values, feature selection, and feature extension have been investigated. The suggested algorithms and frameworks for FDD updating have been evaluated through different benchmarks and scenarios. As a part of the results, the suggested algorithms for supervised handling CD surpass the performance of the traditional incremental learning in regard to MGM score (defined dimensionless score based on weighted F1 score and training time) even up to 50% improvement. This improvement is achieved by proposed algorithms that detect and forget redundant information as well as properly adjusting the data window for timely updating and retraining the fault detection system. Moreover, the proposed unsupervised FDD updating framework for dealing with novel faults in static and dynamic process conditions achieves up to 90% in terms of the NPP score (defined dimensionless score based on number of the correct predicted class of samples). This result relies on an innovative framework that is able to assign samples either to new classes or to available classes by exploiting one class classification techniques and clustering approaches.Los procesos industriales modernos son cada vez más complejos y, en consecuencia, su control se ha convertido en una tarea desafiante. La detección y el diagnóstico de fallos (FDD), como un elemento clave de la supervisión del proceso, deben ser investigados debido a su papel esencial en los procesos de toma de decisiones. Entre los métodos disponibles de FDD, los enfoques basados en datos están recibiendo una atención creciente debido a su relativa simplicidad en la implementación. Independientemente de los tipos de FDD, una de las principales características de los sistemas FDD confiables es su capacidad de actualización, mientras que las nuevas condiciones que no fueron consideradas en su entrenamiento inicial, ahora aparecen en el proceso. Estas nuevas condiciones pueden surgir de forma gradual o abrupta, pero tienen el mismo nivel de importancia ya que en ambos casos conducen al bajo rendimiento de FDD. Para abordar las tareas de actualización, se han propuesto algunos métodos, pero no mayoritariamente en el área de investigación de la ingeniería química. Podrían ser categorizados en los que están dedicados a manejar Concept Drift (CD) (que aparecen gradualmente), y a los que tratan con clases nuevas (que aparecen abruptamente). Los métodos disponibles, además de la falta de estrategias claras para la actualización, sufren debilidades en su funcionamiento y de un tiempo de capacitación ineficiente, como se ha referenciado. En consecuencia, esta tesis está dedicada principalmente a la actualización de FDD impulsada por datos en procesos químicos. Los esquemas propuestos para manejar nuevas clases de fallos se basan en métodos no supervisados, mientras que para hacer frente a la CD se han investigado los marcos de actualización supervisados y no supervisados. Además, para mejorar la funcionalidad de los sistemas FDD, se han investigado algunos de los principales métodos de procesamiento de datos, incluida la imputación de valores perdidos, la selección de características y la extensión de características. Los algoritmos y marcos sugeridos para la actualización de FDD han sido evaluados a través de diferentes puntos de referencia y escenarios. Como parte de los resultados, los algoritmos sugeridos para el CD de manejo supervisado superan el rendimiento del aprendizaje incremental tradicional con respecto al puntaje MGM (puntuación adimensional definida basada en el puntaje F1 ponderado y el tiempo de entrenamiento) hasta en un 50% de mejora. Esta mejora se logra mediante los algoritmos propuestos que detectan y olvidan la información redundante, así como ajustan correctamente la ventana de datos para la actualización oportuna y el reciclaje del sistema de detección de fallas. Además, el marco de actualización FDD no supervisado propuesto para tratar fallas nuevas en condiciones de proceso estáticas y dinámicas logra hasta 90% en términos de la puntuación de NPP (puntuación adimensional definida basada en el número de la clase de muestras correcta predicha). Este resultado se basa en un marco innovador que puede asignar muestras a clases nuevas o a clases disponibles explotando una clase de técnicas de clasificación y enfoques de agrupamientoPostprint (published version

A Machine Learning-based Distributed System for Fault Diagnosis with Scalable Detection Quality in Industrial IoT

In this paper, a methodology based on machine learning for fault detection in continuous processes is presented. It aims to monitor fully distributed scenarios, such as the Tennessee Eastman Process, selected as the use case of this work, where sensors are distributed throughout an industrial plant. A hybrid feature selection approach based on filters and wrappers, called Hybrid Fisher Wrapper method, is proposed to select the most representative sensors to get the highest detection quality for fault identification. The proposed methodology provides a complete design space of solutions differing in the sensing effort, the processing complexity, and the obtained detection quality. It constitutes an alternative to the typical scheme in Industry 4.0, where multiple distributed sensor systems collect and send data to a centralised cloud. Differently, the proposed technique follows a distributed approach, in which processing can be done eventually close to the sensors where data is generated, i.e., at the edge of the Internet of Things. This approach overcomes the bandwidth, privacy, and latency limitations that centralised approaches may suffer. The experimental results show that the proposed methodology provides Tennessee Eastman Process fault detection solutions with state-of-the-art detection quality figures. In terms of latency, solutions obtained outperform in 37.5 times the implementation with the highest detection quality, using 1.99 times fewer features, on average. Also, the scalability of the framework provides a design space where the optimal implementation can be chosen according to the application needs