Incipient Fault Detection, Diagnosis, and Prognosis using Canonical Variate Dissimilarity Analysis

Industrial process monitoring deals with three main activities, namely, fault detection, fault diagnosis, and fault prognosis. Respectively, these activities seek to answer three questions: ‘Has a fault occurred?’, ‘Where did it occur and how large?’, and ‘How will it progress in the future?’ As opposed to abrupt faults, incipient faults are those that slowly develop in time, leading ultimately to process failure or an emergency situation. A recently developed multivariate statistical tool for early detection of incipient faults under varying operating conditions is the Canonical Variate Dissimilarity Analysis (CVDA). In CVDA, a dissimilarity-based statistical index was derived to improve the detection sensitivity upon the traditional canonical variate analysis (CVA) indices. This study aims to extend the CVDA detection framework towards diagnosis and prognosis of process conditions. For diagnosis, contribution maps are used to convey the magnitude and location of the incipient fault effects, as well as their evolution in time. For prognosis, CVA state-space prediction and Kalman filtering during faulty conditions are proposed in this work. By covering the three main process monitoring activities in one framework, our work can serve as a baseline strategy for future application to large process industries

A Review of Kernel Methods for Feature Extraction in Nonlinear Process Monitoring

Kernel methods are a class of learning machines for the fast recognition of nonlinear patterns in any data set. In this paper, the applications of kernel methods for feature extraction in industrial process monitoring are systematically reviewed. First, we describe the reasons for using kernel methods and contextualize them among other machine learning tools. Second, by reviewing a total of 230 papers, this work has identified 12 major issues surrounding the use of kernel methods for nonlinear feature extraction. Each issue was discussed as to why they are important and how they were addressed through the years by many researchers. We also present a breakdown of the commonly used kernel functions, parameter selection routes, and case studies. Lastly, this review provides an outlook into the future of kernel-based process monitoring, which can hopefully instigate more advanced yet practical solutions in the process industries

An Improved Particle Filtering-based Approach for Health Prediction and Prognosis of Nonlinear Systems

Health monitoring of nonlinear systems is broadly concerned with the system health tracking and its prediction to future time horizons. Estimation and prediction schemes constitute as principle components of any health monitoring technique. Particle filter (PF) represents a powerful tool for performing state and parameter estimation as well as prediction of nonlinear dynamical systems. Estimation of the system parameters along with the states can yield an up-to-date and reliable model that can be used for long-term prediction problems through utilization of particle filters. This feature enables one to deal with uncertainty issues in the resulting prediction step as the time horizon is extended. Towards this end, this paper presents an improved method to achieve uncertainty management for long-term prediction of nonlinear systems by using particle filters. In our proposed approach, an observation forecasting scheme is developed to extend the system observation profiles (as time-series) to future time horizon. Particles are then propagated to future time instants according to a resampling algorithm instead of considering constant weights for the particles propagation in the prediction step. The uncertainty in the long-term prediction of the system states and parameters are managed by utilizing dynamic linear models for development of an observation forecasting scheme. This task is addressed through an outer adjustment loop for adaptively changing the sliding observation injection window based on the Mahalanobis distance criterion. Our proposed approach is then applied to predicting the health condition as well as the remaining useful life (RUL) of a gas turbine engine that is affected by degradations in the system health parameters. Extensive simulation and case studies are conducted to demonstrate and illustrate the capabilities and performance characteristics of our proposed and developed schemes

Data driven methods for updating fault detection and diagnosis system in chemical processes

Modern industrial processes are becoming more complex, and consequently monitoring them has become a challenging task. Fault Detection and Diagnosis (FDD) as a key element of process monitoring, needs to be investigated because of its essential role in decision making processes. Among available FDD methods, data driven approaches are currently receiving increasing attention because of their relative simplicity in implementation. Regardless of FDD types, one of the main traits of reliable FDD systems is their ability of being updated while new conditions that were not considered at their initial training appear in the process. These new conditions would emerge either gradually or abruptly, but they have the same level of importance as in both cases they lead to FDD poor performance. For addressing updating tasks, some methods have been proposed, but mainly not in research area of chemical engineering. They could be categorized to those that are dedicated to managing Concept Drift (CD) (that appear gradually), and those that deal with novel classes (that appear abruptly). The available methods, mainly, in addition to the lack of clear strategies for updating, suffer from performance weaknesses and inefficient required time of training, as reported. Accordingly, this thesis is mainly dedicated to data driven FDD updating in chemical processes. The proposed schemes for handling novel classes of faults are based on unsupervised methods, while for coping with CD both supervised and unsupervised updating frameworks have been investigated. Furthermore, for enhancing the functionality of FDD systems, some major methods of data processing, including imputation of missing values, feature selection, and feature extension have been investigated. The suggested algorithms and frameworks for FDD updating have been evaluated through different benchmarks and scenarios. As a part of the results, the suggested algorithms for supervised handling CD surpass the performance of the traditional incremental learning in regard to MGM score (defined dimensionless score based on weighted F1 score and training time) even up to 50% improvement. This improvement is achieved by proposed algorithms that detect and forget redundant information as well as properly adjusting the data window for timely updating and retraining the fault detection system. Moreover, the proposed unsupervised FDD updating framework for dealing with novel faults in static and dynamic process conditions achieves up to 90% in terms of the NPP score (defined dimensionless score based on number of the correct predicted class of samples). This result relies on an innovative framework that is able to assign samples either to new classes or to available classes by exploiting one class classification techniques and clustering approaches.Los procesos industriales modernos son cada vez más complejos y, en consecuencia, su control se ha convertido en una tarea desafiante. La detección y el diagnóstico de fallos (FDD), como un elemento clave de la supervisión del proceso, deben ser investigados debido a su papel esencial en los procesos de toma de decisiones. Entre los métodos disponibles de FDD, los enfoques basados en datos están recibiendo una atención creciente debido a su relativa simplicidad en la implementación. Independientemente de los tipos de FDD, una de las principales características de los sistemas FDD confiables es su capacidad de actualización, mientras que las nuevas condiciones que no fueron consideradas en su entrenamiento inicial, ahora aparecen en el proceso. Estas nuevas condiciones pueden surgir de forma gradual o abrupta, pero tienen el mismo nivel de importancia ya que en ambos casos conducen al bajo rendimiento de FDD. Para abordar las tareas de actualización, se han propuesto algunos métodos, pero no mayoritariamente en el área de investigación de la ingeniería química. Podrían ser categorizados en los que están dedicados a manejar Concept Drift (CD) (que aparecen gradualmente), y a los que tratan con clases nuevas (que aparecen abruptamente). Los métodos disponibles, además de la falta de estrategias claras para la actualización, sufren debilidades en su funcionamiento y de un tiempo de capacitación ineficiente, como se ha referenciado. En consecuencia, esta tesis está dedicada principalmente a la actualización de FDD impulsada por datos en procesos químicos. Los esquemas propuestos para manejar nuevas clases de fallos se basan en métodos no supervisados, mientras que para hacer frente a la CD se han investigado los marcos de actualización supervisados y no supervisados. Además, para mejorar la funcionalidad de los sistemas FDD, se han investigado algunos de los principales métodos de procesamiento de datos, incluida la imputación de valores perdidos, la selección de características y la extensión de características. Los algoritmos y marcos sugeridos para la actualización de FDD han sido evaluados a través de diferentes puntos de referencia y escenarios. Como parte de los resultados, los algoritmos sugeridos para el CD de manejo supervisado superan el rendimiento del aprendizaje incremental tradicional con respecto al puntaje MGM (puntuación adimensional definida basada en el puntaje F1 ponderado y el tiempo de entrenamiento) hasta en un 50% de mejora. Esta mejora se logra mediante los algoritmos propuestos que detectan y olvidan la información redundante, así como ajustan correctamente la ventana de datos para la actualización oportuna y el reciclaje del sistema de detección de fallas. Además, el marco de actualización FDD no supervisado propuesto para tratar fallas nuevas en condiciones de proceso estáticas y dinámicas logra hasta 90% en términos de la puntuación de NPP (puntuación adimensional definida basada en el número de la clase de muestras correcta predicha). Este resultado se basa en un marco innovador que puede asignar muestras a clases nuevas o a clases disponibles explotando una clase de técnicas de clasificación y enfoques de agrupamientoPostprint (published version

Health Monitoring of Nonlinear Systems with Application to Gas Turbine Engines

Health monitoring and prognosis of nonlinear systems is mainly concerned with system health tracking and its evolution prediction to future time horizons. Estimation and prediction schemes constitute as principal components of any health monitoring framework. In this thesis, the main focus is on development of novel health monitoring techniques for nonlinear dynamical systems by utilizing model-based and hybrid prognosis and health monitoring approaches. First, given the fact that particle filters (PF) are known as a powerful tool for performing state and parameter estimation of nonlinear dynamical systems, a novel dual estimation methodology is developed for both time-varying parameters and states of a nonlinear stochastic system based on the prediction error (PE) concept and the particle filtering scheme. Estimation of system parameters along with the states generate an updated model that can be used for a long-term prediction problem. Next, an improved particle filtering-based methodology is developed to address the prediction step within the developed health monitoring framework. In this method, an observation forecasting scheme is developed to extend the system observation profiles (as time-series) to future time horizons. Particles are then propagated to future time instants according to a resampling algorithm in the prediction step. The uncertainty in the long-term prediction of the system states and parameters are managed by utilizing dynamic linear models (DLM) for development of an observation forecasting scheme. A novel hybrid architecture is then proposed to develop prognosis and health monitoring methodologies for nonlinear systems by integration of model-based and computationally intelligent-based techniques. Our proposed hybrid health monitoring methodology is constructed based on a framework that is not dependent on the structure of the neural network model utilized in the implementation of the observation forecasting scheme. Moreover, changing the neural network model structure in this framework does not significantly affect the prediction accuracy of the entire health prediction algorithm. Finally, a method for formulation of health monitoring problems of dynamical systems through a two-time scale decomposition is introduced. For this methodology the system dynamical equations as well as the affected damage model, are investigated in the two-time scale system health estimation and prediction steps. A two-time scale filtering approach is developed based on the ensemble Kalman filtering (EnKF) methodology by taking advantage of the model reduction concept. The performance of the proposed two-time scale ensemble Kalman filters is shown to be more accurate and less computationally intensive as compared to the well-known particle filtering approach for this class of nonlinear systems. All of our developed methods have been applied for health monitoring and prognosis of a gas turbine engine when it is affected by various degradation damages. Extensive comparative studies are also conducted to validate and demonstrate the advantages and capabilities of our proposed frameworks and methodologies

Advanced Fault Diagnosis and Health Monitoring Techniques for Complex Engineering Systems

Over the last few decades, the field of fault diagnostics and structural health management has been experiencing rapid developments. The reliability, availability, and safety of engineering systems can be significantly improved by implementing multifaceted strategies of in situ diagnostics and prognostics. With the development of intelligence algorithms, smart sensors, and advanced data collection and modeling techniques, this challenging research area has been receiving ever-increasing attention in both fundamental research and engineering applications. This has been strongly supported by the extensive applications ranging from aerospace, automotive, transport, manufacturing, and processing industries to defense and infrastructure industries

매개분포근사를 통한 공정시스템 공학에서의 확률기계학습 접근법

학위논문(박사) -- 서울대학교대학원 : 공과대학 화학생물공학부, 2021.8. 이종민.With the rapid development of measurement technology, higher quality and vast amounts of process data become available. Nevertheless, process data are ‘scarce’ in many cases as they are sampled only at certain operating conditions while the dimensionality of the system is large. Furthermore, the process data are inherently stochastic due to the internal characteristics of the system or the measurement noises. For this reason, uncertainty is inevitable in process systems, and estimating it becomes a crucial part of engineering tasks as the prediction errors can lead to misguided decisions and cause severe casualties or economic losses. A popular approach to this is applying probabilistic inference techniques that can model the uncertainty in terms of probability. However, most of the existing probabilistic inference techniques are based on recursive sampling, which makes it difficult to use them for industrial applications that require processing a high-dimensional and massive amount of data. To address such an issue, this thesis proposes probabilistic machine learning approaches based on parametric distribution approximation, which can model the uncertainty of the system and circumvent the computational complexity as well. The proposed approach is applied for three major process engineering tasks: process monitoring, system modeling, and process design. First, a process monitoring framework is proposed that utilizes a probabilistic classifier for fault classification. To enhance the accuracy of the classifier and reduce the computational cost for its training, a feature extraction method called probabilistic manifold learning is developed and applied to the process data ahead of the fault classification. We demonstrate that this manifold approximation process not only reduces the dimensionality of the data but also casts the data into a clustered structure, making the classifier have a low dependency on the type and dimension of the data. By exploiting this property, non-metric information (e.g., fault labels) of the data is effectively incorporated and the diagnosis performance is drastically improved. Second, a probabilistic modeling approach based on Bayesian neural networks is proposed. The parameters of deep neural networks are transformed into Gaussian distributions and trained using variational inference. The redundancy of the parameter is autonomously inferred during the model training, and insignificant parameters are eliminated a posteriori. Through a verification study, we demonstrate that the proposed approach can not only produce high-fidelity models that describe the stochastic behaviors of the system but also produce the optimal model structure. Finally, a novel process design framework is proposed based on reinforcement learning. Unlike the conventional optimization methods that recursively evaluate the objective function to find an optimal value, the proposed method approximates the objective function surface by parametric probabilistic distributions. This allows learning the continuous action policy without introducing any cumbersome discretization process. Moreover, the probabilistic policy gives means for effective control of the exploration and exploitation rates according to the certainty information. We demonstrate that the proposed framework can learn process design heuristics during the solution process and use them to solve similar design problems.계측기술의 발달로 양질의, 그리고 방대한 양의 공정 데이터의 취득이 가능해졌다. 그러나 많은 경우 시스템 차원의 크기에 비해서 일부 운전조건의 공정 데이터만이 취득되기 때문에, 공정 데이터는 ‘희소’하게 된다. 뿐만 아니라, 공정 데이터는 시스템 거동 자체와 더불어 계측에서 발생하는 노이즈로 인한 본질적인 확률적 거동을 보인다. 따라서 시스템의 예측모델은 예측 값에 대한 불확실성을 정량적으로 기술하는 것이 요구되며, 이를 통해 오진을 예방하고 잠재적 인명 피해와 경제적 손실을 방지할 수 있다. 이에 대한 보편적인 접근법은 확률추정기법을 사용하여 이러한 불확실성을 정량화 하는 것이나, 현존하는 추정기법들은 재귀적 샘플링에 의존하는 특성상 고차원이면서도 다량인 공정데이터에 적용하기 어렵다는 근본적인 한계를 가진다. 본 학위논문에서는 매개분포근사에 기반한 확률기계학습을 적용하여 시스템에 내재된 불확실성을 모델링하면서도 동시에 계산 효율적인 접근 방법을 제안하였다. 먼저, 공정의 모니터링에 있어 가우시안 혼합 모델 (Gaussian mixture model)을 분류자로 사용하는 확률적 결함 분류 프레임워크가 제안되었다. 이때 분류자의 학습에서의 계산 복잡도를 줄이기 위하여 데이터를 저차원으로 투영시키는데, 이를 위한 확률적 다양체 학습 (probabilistic manifold learn-ing) 방법이 제안되었다. 제안하는 방법은 데이터의 다양체 (manifold)를 근사하여 데이터 포인트 사이의 쌍별 우도 (pairwise likelihood)를 보존하는 투영법이 사용된다. 이를 통하여 데이터의 종류와 차원에 의존도가 낮은 진단 결과를 얻음과 동시에 데이터 레이블과 같은 비거리적 (non-metric) 정보를 효율적으로 사용하여 결함 진단 능력을 향상시킬 수 있음을 보였다. 둘째로, 베이지안 심층 신경망(Bayesian deep neural networks)을 사용한 공정의 확률적 모델링 방법론이 제시되었다. 신경망의 각 매개변수는 가우스 분포로 치환되며, 변분추론 (variational inference)을 통하여 계산 효율적인 훈련이 진행된다. 훈련이 끝난 후 파라미터의 유효성을 측정하여 불필요한 매개변수를 소거하는 사후 모델 압축 방법이 사용되었다. 반도체 공정에 대한 사례 연구는 제안하는 방법이 공정의 복잡한 거동을 효과적으로 모델링 할 뿐만 아니라 모델의 최적 구조를 도출할 수 있음을 보여준다. 마지막으로, 분포형 심층 신경망을 사용한 강화학습을 기반으로 한 확률적 공정 설계 프레임워크가 제안되었다. 최적치를 찾기 위해 재귀적으로 목적 함수 값을 평가하는 기존의 최적화 방법론과 달리, 목적 함수 곡면 (objective function surface)을 매개화 된 확률분포로 근사하는 접근법이 제시되었다. 이를 기반으로 이산화 (discretization)를 사용하지 않고 연속적 행동 정책을 학습하며, 확실성 (certainty)에 기반한 탐색 (exploration) 및 활용 (exploi-tation) 비율의 제어가 효율적으로 이루어진다. 사례 연구 결과는 공정의 설계에 대한 경험지식 (heuristic)을 학습하고 유사한 설계 문제의 해를 구하는 데 이용할 수 있음을 보여준다.Chapter 1 Introduction 1 1.1. Motivation 1 1.2. Outline of the thesis 5 Chapter 2 Backgrounds and preliminaries 9 2.1. Bayesian inference 9 2.2. Monte Carlo 10 2.3. Kullback-Leibler divergence 11 2.4. Variational inference 12 2.5. Riemannian manifold 13 2.6. Finite extended-pseudo-metric space 16 2.7. Reinforcement learning 16 2.8. Directed graph 19 Chapter 3 Process monitoring and fault classification with probabilistic manifold learning 20 3.1. Introduction 20 3.2. Methods 25 3.2.1. Uniform manifold approximation 27 3.2.2. Clusterization 28 3.2.3. Projection 31 3.2.4. Mapping of unknown data query 32 3.2.5. Inference 33 3.3. Verification study 38 3.3.1. Dataset description 38 3.3.2. Experimental setup 40 3.3.3. Process monitoring 43 3.3.4. Projection characteristics 47 3.3.5. Fault diagnosis 50 3.3.6. Computational Aspects 56 Chapter 4 Process system modeling with Bayesian neural networks 59 4.1. Introduction 59 4.2. Methods 63 4.2.1. Long Short-Term Memory (LSTM) 63 4.2.2. Bayesian LSTM (BLSTM) 66 4.3. Verification study 68 4.3.1. System description 68 4.3.2. Estimation of the plasma variables 71 4.3.3. Dataset description 72 4.3.4. Experimental setup 72 4.3.5. Weight regularization during training 78 4.3.6. Modeling complex behaviors of the system 80 4.3.7. Uncertainty quantification and model compression 85 Chapter 5 Process design based on reinforcement learning with distributional actor-critic networks 89 5.1. Introduction 89 5.2. Methods 93 5.2.1. Flowsheet hashing 93 5.2.2. Behavioral cloning 99 5.2.3. Neural Monte Carlo tree search (N-MCTS) 100 5.2.4. Distributional actor-critic networks (DACN) 105 5.2.5. Action masking 110 5.3. Verification study 110 5.3.1. System description 110 5.3.2. Experimental setup 111 5.3.3. Result and discussions 115 Chapter 6 Concluding remarks 120 6.1. Summary of the contributions 120 6.2. Future works 122 Appendix 125 A.1. Proof of Lemma 1 125 A.2. Performance indices for dimension reduction 127 A.3. Model equations for process units 130 Bibliography 132 초 록 149박

Fault Diagnosis and Failure Prognostics of Lithium-ion Battery based on Least Squares Support Vector Machine and Memory Particle Filter Framework

123456A novel data driven approach is developed for fault diagnosis and remaining useful life (RUL) prognostics for lithium-ion batteries using Least Square Support Vector Machine (LS-SVM) and Memory-Particle Filter (M-PF). Unlike traditional data-driven models for capacity fault diagnosis and failure prognosis, which require multidimensional physical characteristics, the proposed algorithm uses only two variables: Energy Efficiency (EE), and Work Temperature. The aim of this novel framework is to improve the accuracy of incipient and abrupt faults diagnosis and failure prognosis. First, the LSSVM is used to generate residual signal based on capacity fade trends of the Li-ion batteries. Second, adaptive threshold model is developed based on several factors including input, output model error, disturbance, and drift parameter. The adaptive threshold is used to tackle the shortcoming of a fixed threshold. Third, the M-PF is proposed as the new method for failure prognostic to determine Remaining Useful Life (RUL). The M-PF is based on the assumption of the availability of real-time observation and historical data, where the historical failure data can be used instead of the physical failure model within the particle filter. The feasibility of the framework is validated using Li-ion battery prognostic data obtained from the National Aeronautic and Space Administration (NASA) Ames Prognostic Center of Excellence (PCoE). The experimental results show the following: (1) fewer data dimensions for the input data are required compared to traditional empirical models; (2) the proposed diagnostic approach provides an effective way of diagnosing Li-ion battery fault; (3) the proposed prognostic approach can predict the RUL of Li-ion batteries with small error, and has high prediction accuracy; and, (4) the proposed prognostic approach shows that historical failure data can be used instead of a physical failure model in the particle filter

Vibration Monitoring: Gearbox identification and faults detection

L'abstract è presente nell'allegato / the abstract is in the attachmen