A Review of Kernel Methods for Feature Extraction in Nonlinear Process Monitoring

Kernel methods are a class of learning machines for the fast recognition of nonlinear patterns in any data set. In this paper, the applications of kernel methods for feature extraction in industrial process monitoring are systematically reviewed. First, we describe the reasons for using kernel methods and contextualize them among other machine learning tools. Second, by reviewing a total of 230 papers, this work has identified 12 major issues surrounding the use of kernel methods for nonlinear feature extraction. Each issue was discussed as to why they are important and how they were addressed through the years by many researchers. We also present a breakdown of the commonly used kernel functions, parameter selection routes, and case studies. Lastly, this review provides an outlook into the future of kernel-based process monitoring, which can hopefully instigate more advanced yet practical solutions in the process industries

Improved Dynamic Latent Variable Modeling for Process Monitoring, Fault Diagnosis and Anomaly Detection

Due to the rapid advancement of modern industrial processes, a considerable number of measured variables enhance the complexity of systems, progressively leading to the development of multivariate statistical analysis (MSA) methods to exploit valuable information from the collected data for predictive modeling, fault detection and diagnosis, such as partial least squares (PLS), canonical correlation analysis (CCA) and their extensions. However, these methods suffer from some issues, involving the irrelevant information extracted by PLS, and CCA’s inability to exploit quality information. Latent variable regression (LVR) was designed to address these issues, but it has not been fully and systematically studied. A concurrent kernel LVR (CKLVR) with a regularization term is designed for collinear and nonlinear data to construct a full decomposition of the original nonlinear data space, and to provide comprehensive information of the systems. Further, dynamics are inevitable in practical industrial processes, and thus a dynamic auto-regressive LVR (DALVR) is also proposed based on regularized LVR to capture dynamic variations in both process and quality data. The comprehensive monitoring framework and fault diagnosis and causal analysis scheme based on DALVR are developed. Their superiority can be demonstrated with case studies, involving the Tennessee Eastman process, Dow’s refining process and three-phase flow facility process. In addition to MSA approaches, autoencoder (AE) technology is extensively used in complicated processes to handle the expanding dimensionality caused by the increasing complexity of industrial applications. Apart from modeling and fault diagnosis, anomaly detection draws great attention as well to maintain the performance, avoid economic losses, and ensure safety during the industrial processes. In view of advantages in dimensionality reduction and feature retention, autoencoder (AE) technology is widely applied for anomaly detection monitoring. Considering both high dimensionality and dynamic relations between elements in the hidden layer, an improved autoencoder with dynamic hidden layer (DHL-AE) is proposed and applied for anomaly detection monitoring. Two case studies including Tennessee Eastman process and Wind data are used to show the effectiveness of the proposed algorithm

Application of dynamic partial least squares to complex processes

PhD ThesisMultivariate statistical modelling and monitoring is an active area of research and development in both academia and industry. This is due to the economic and safety benefits that can be attained from the implementation of process modelling and monitoring schemes. Most industrial processes in the chemistry-using sector exhibit complex characteristics including process dynamics, non-linearity and changes in operational behaviour which are compounded by the occurrence of non-conforming data points. To date, modelling and monitoring methodologies have focussed on processes exhibiting one of the aforementioned characteristics. This Thesis considers the development and application of multivariate statistical methods for the modelling and monitoring of the whole process as well as individual unit operations with a particular focus on the complex dynamic nonlinear behaviour of continuous processes. Following a review of Partial Least Squares (PLS), which is applicable for the analysis of problems that exhibit high dimensionality and correlated/collinear variables, it was observed that it is inappropriate for the analysis of data from complex dynamic processes. To address this issue, a multivariate statistical method Robust Adaptive PLS (RAPLS) was proposed, which has the ability to distinguish between non-conforming data, i.e. statistical outliers and a process fault. Through the analysis of data from a mathematical simulation of a time varying and non-stationary process, it is observed that RAPLS shows superior monitoring performance compared to conventional PLS. The model has the ability to adapt to changes in process operating conditions without losing its ability to detect process faults and statistical outliers. A dynamic extension, RADPLS, using an autoregressive with exogenous inputs (ARX) representation was developed to model and monitor the complex dynamic and nonlinear behaviour of an Ammonia Synthesis Fixed-bed Reactor. The resultant model, which is resistant to outliers, shows significant improvement over other dynamic PLS based representations. The proposed method shows some limitations in terms of the detection of the fault for its full duration but it significantly reduces the false alarm rate. The RAPLS algorithm is further extended to a dynamic multi-block algorithm, RAMBDPLS, through the conjunction of a finite impulse response (FIR) representation and multiblock PLS. It was applied to the benchmark Tennessee Eastman Process to illustrate its applicability for the monitoring of the whole process and individual unit operations and to demonstrate the concept of fault propagation in a dynamic and nonlinear continuous system. The resulting model detects the faults and reduces the false alarm rate compared to conventional PLS.Ministry of Higher Education and King Abdulaziz University, Saudi Arabi

Plantwide simulation and monitoring of offshore oil and gas production facility

Monitoring is one of the major concerns in offshore oil and gas production platform since the access to the offshore facilities is difficult. Also, it is quite challenging to extract oil and gas safely in such a harsh environment, and any abnormalities may lead to a catastrophic event. The process data, including all possible faulty scenarios, is required to build an appropriate monitoring system. Since the plant wide process data is not available in the literature, a dynamic model and simulation of an offshore oil and gas production platform is developed by using Aspen HYSYS. Modeling and simulations are handy tools for designing and predicting the accurate behavior of a production plant. The model was built based on the gas processing plant at the North Sea platform reported in Voldsund et al. (2013). Several common faults from different fault categories were simulated in the dynamic system, and their impacts on the overall hydrocarbon production were analyzed. The simulated data are then used to build a monitoring system for each of the faulty states. A new monitoring method has been proposed by combining Principal Component Analysis (PCA) and Dynamic PCA (DPCA) with Artificial Neural Network (ANN). The application of ANN to process systems is quite difficult as it involves a very large number of input neurons to model the system. Training of such large scale network is time-consuming and provides poor accuracy with a high error rate. In PCA-ANN and DPCA-ANN monitoring system, PCA and DPCA are used to reduce the dimension of the training data set and extract the main features of measured variables. Subsequently ANN uses this lower-dimensional score vectors to build a training model and classify the abnormalities. It is found that the proposed approach reduces the time to train ANN and successfully diagnose, detects and classifies the faults with a high accuracy rate

Data integration for the monitoring of batch processes in the pharmaceutical industry

Advances in sensor technology has resulted in large amounts of data being available electronically. However, to utilise the potential of the data, there is a need to transform the data into knowledge to realise an enhanced understanding of the process. This thesis investigates a number of multivariate statistical projection techniques for the monitoring of batch fermentation and pharmaceutical processes. In the first part of the thesis, the traditional performance monitoring tools based on the approaches of Nomikos and MacGregor (1994) and Wold et al. (1998) are introduced. Additionally, the application of data scaling as a data pre-treatment step for batch processes is examined and it is observed that it has a significant impact on monitoring performance. Based on the advantages and limitations of these techniques, an alternative methodology is proposed and applied to a simulated penicillin fermentation process. The approach is compared with existing techniques using two metrics, false alarm rate and out-ofcontrol average run length. A further manufacturing challenge facing the pharmaceutical industry is to understand the differences in the performance of a product which is manufactured at two or more sites. A retrospective multi-site monitoring model is developed utilising a pooled sample variancecovariance methodology of the two sites. The results of this approach are compared with a number of techniques that have been previously reported in the literature for the integration of data from two or more sources. The latter part of the thesis focuses on data integration using multi-block analysis. Several blocks of data can be analysed simultaneously to allow the inter- and intra- block relationships to be extracted. The methodology of multi-block Principal Component Analysis (MBPCA) is initially reviewed. To enhance the sensitivity of the algorithm, wavelet analysis is incorporated within the MBPCA framework. The fundamental advantage of wavelet analysis is its ability to process a signal at different scales so that both the global features and the localised details of a signal can be studied simultaneously. Both existing and the modified approach are applied to data generated from an experiment conducted in a batch mini-plant and that was monitored by both physical sensors and on-line UV-Visible spectrometer. The performance of the integrated approaches is benchmarked against the individual process and spectral monitoring models as well as examining their fault detection ability on two additional batches with pre-designed process deviations.EThOS - Electronic Theses Online ServiceEngineering and Physical Sciences Research Council (EPSRC) : Overseas Research Students Award Scheme (ORSAS) : Centre for Process Analytics and Control Technology (CPACT)GBUnited Kingdo

Data integration for the monitoring of batch processes in the pharmeceutical industry

PhD ThesisAdvances in sensor technology has resulted in large amounts of data being available electronically. However, to utilise the potential of the data, there is a need to transform the data into knowledge to realise an enhanced understanding of the process. This thesis investigates a number of multivariate statistical projection techniques for the monitoring of batch fermentation and pharmaceutical processes. In the first part of the thesis, the traditional performance monitoring tools based on the approaches of Nomikos and MacGregor (1994) and Wold et al. (1998) are introduced. Additionally, the application of data scaling as a data pre-treatment step for batch processes is examined and it is observed that it has a significant impact on monitoring performance. Based on the advantages and limitations of these techniques, an alternative methodology is proposed and applied to a simulated penicillin fermentation process. The approach is compared with existing techniques using two metrics, false alarm rate and out-ofcontrol average run length. A further manufacturing challenge facing the pharmaceutical industry is to understand the differences in the performance of a product which is manufactured at two or more sites. A retrospective multi-site monitoring model is developed utilising a pooled sample variancecovariance methodology of the two sites. The results of this approach are compared with a number of techniques that have been previously reported in the literature for the integration of data from two or more sources. The latter part of the thesis focuses on data integration using multi-block analysis. Several blocks of data can be analysed simultaneously to allow the inter- and intra- block relationships to be extracted. The methodology of multi-block Principal Component Analysis (MBPCA) is initially reviewed. To enhance the sensitivity of the algorithm, wavelet analysis is incorporated within the MBPCA framework. The fundamental advantage of wavelet analysis is its ability to process a signal at different scales so that both the global features and the localised details of a signal can be studied simultaneously. Both existing and the modified approach are applied to data generated from an experiment conducted in a batch mini-plant and that was monitored by both physical sensors and on-line UV-Visible spectrometer. The performance of the integrated approaches is benchmarked against the individual process and spectral monitoring models as well as examining their fault detection ability on two additional batches with pre-designed process deviations.Engineering and Physical Sciences Research Council (EPSRC: The Overseas Research Students Award Scheme (ORSAS): The Centre for Process Analytics and Control Technology (CPACT)

Datenbasierter Entwurf von Fehlerdiagnosesystemen

Due to the increasing demands on system performance, production quality as well as economic operation, modern technical systems become more complicated and the automation degrees are significantly growing. To ensure the safety and reliability of such complicated processes, an effective fault diagnosis system is of prime importance in process industry nowadays. Although the model-based fault diagnosis theory has been well established, it is still difficult to establish mathematical model by means of the first principles for large-scale process. On the other hand, a large amount of historical data from regular sensor measurements, event-logs and records are often available in such industrial processes. Motivated by this observation, it is of great interest to design fault diagnosis schemes only based on the available process data. Hence, development of efficient data-driven fault diagnosis schemes for different operating conditions is the primary objective of this thesis. This thesis is firstly dedicated to the modifications on the standard multivariate statistical process monitoring approaches. The modified approaches are considerably simple, and most importantly, avoid the drawbacks of the standard techniques. As a result, the proposed approaches are able to provide enhanced fault diagnosis performance on the applications under stationary operating conditions. The further study of this thesis focuses on developing reliable fault diagnosis schemes for dynamic processes under industrial operating conditions. Instead of identifying the entire process model, primary fault diagnosis can be efficiently realized by the identification of key components. Advanced design schemes like multiple residuals generator and state observer are also investigated to ensure high fault sensitivity performance. For the large-scale processes involving changes, e.g. in operating regimes for in the manipulated variables, the recursive and adaptive techniques are studied to cope with such uncertainty issues. A novel data-driven adaptive scheme is proposed, whose stability and convergence rate are analytically proven. Compared to the standard techniques, this approach does not involve complicated on-line computation and produces consistent estimate of the unknown parameters. To illustrate the effectiveness of the derived data-driven approaches, three industrial benchmark processes, i.e. the Tennessee Eastman chemical plant, the fed-batch fermentation penicillin process and the continuously stirred tank heater, are finally considered in this thesis.Durch steigende Anforderungen an Systemverhalten, Produktqualität sowie ökonomischen Betrieb werden moderne technische Systeme stets komplizierter und ihr Grad an Automatisierung steigt wesentlich. Um die Sicherheit und Zuverlässigkeit solcher komplizierter Prozesse zu gewährleisten sind effektive Fehlerdiagnosesysteme heutzutage von großer industrieller Bedeutung. Auch wenn die Theorie zur modellbasierten Fehlerdiagnose wohl etabliert ist so ist es, gerade für komplexe Prozesse, doch schwierig das hierfür benötigte mathematische Modell basierend auf physikalischen Grundprinzipien herzuleiten. Auf der anderen Seite sind meist viele historische Daten von regulären Sensormessungen sowie Verlaufsprotokolle und -aufzeichnungen von solchen Industrieprozessen vorhanden. Motiviert hierdurch ist es von großem Interesse in der Lage zu sein ein Fehlerdiagnosesystem basierend auf eben diesen vorhandenen Daten zu entwerfen. Daher liegt der Fokus dieser Arbeit auf der Entwicklung effizienter datenbasierter Fehlerdiagnose-Schemata für verschiedene Einsatzzwecke. Diese Arbeit ist hauptsächlich den Modifikationen der Standardansätze der multivariaten statistischen Prozessüberwachung gewidmet. Die modifizierten Ansätze sind deutlich einfacher als die Standardverfahren und umgehen dazu noch deren Nachteile. Als ein Ergebnis sind die vorgeschlagenen Ansätze in der Lage eine höhere Güte der Fehlerdiagnose bei Anwendungen mit stationären Betriebsbedingungen zu gewährleisten. Die weiteren Untersuchungen dieser Arbeit befassen sich mit der Entwicklung von zuverlässigen Fehlerdiagnoseschemata für dynamische Prozesse unter industriellen Betriebsbedingungen. Statt das gesamte Prozessmodell identifizieren zu müssen kann hierbei die Erkennung der Hauptfehler meist effizient realisiert werden indem nur Schlüsselkomponenten identifiziert werden. Fortgeschrittene Entwurfsschemata wie zum Beispiel multiple Residuengeneratoren und Zustandsbeobachter werden ebenso untersucht um eine hohe Fehlersensitivitäts-Güte sicherzustellen. Für Großprozesse die Änderungen, in zum Beispiel ihren Betriebspunkten oder den manipulierten Variablen, unterworfen sind werden rekursive und adaptive Techniken untersucht um Unsicherheiten begegnen zu können. Hierzu wird ein neues datenbasiertes, adaptives Schema vorgeschlagen dessen Stabilität und Konvergenzrate analytisch bewiesen werden. Verglichen mit Standardtechniken beinhaltet dieser Ansatz keine komplizierten Onlineberechnungen und erzeugt eine konsistente Schätzung der unbekannten Parameter. Um die Effektivität der hergeleiteten datenbasierten Ansätze zu zeigen werden diese am Ende der Arbeit an drei verschiedenen industriellen Beispielprozessen, dem Tennessee Eastman Chemieprozess, dem Penizilin Batch-Edukt Fermentations-Prozess und dem Rührkesselreaktor simulativ erprobt

Improved Slow Feature Analysis for Process Monitoring

Unsupervised multivariate statistical analysis models are valuable tools for process monitoring and fault diagnosis. Among them, slow feature analysis (SFA) is widely studied and used due to its explicit statistical properties, which aims to extract invariant features of temporally varying signals. This inclusion of dynamics in the model is important when working with process data where new samples are highly correlated to previous ones. However, the existing variations of SFA models cannot exploit increasingly tremendous data volume in modern industries, since they require the data to be fed in as a whole in the training stage. Further, sparsity is also desirable to provide interpretable models and prevent model overfitting. To address the aforementioned issues, a novel algorithm for inducing sparsity in SFA is first introduced, which is referred to as manifold sparse SFA (MSSFA). The non-smooth sparse SFA objective function is optimized using proximal gradient descent and the SFA constraint is fulfilled using manifold optimization. An associated fault detection and diagnosis framework is developed that retains the unsupervised nature of SFA. When compared to SFA, sparse SFA (SSFA), and sparse principal component analysis (SPCA), MSSFA shows superior performance in computational complexity, interpretability, fault detection, and fault diagnosis on the Tennessee Eastman process (TEP) and three-phase flow facility (TPFF) data sets. Furthermore, its sparsity is much improved over SFA and SSFA. Further, to exploit the increasing number of collected samples efficiently, a covariance free incremental SFA (IncSFA) is adapted in this work, which handles massive data efficiently and has a linear feature updating complexity with respect to data dimensionality. The IncSFA based process monitoring scheme is also proposed for anomaly detection. Further, a new incremental MSSFA (IncMSSFA) algorithm is also introduced that is able to use the same monitoring scheme. These two algorithms are compared against recursive SFA (RSFA) which can also process data incrementally. The efficiency of IncSFA-based monitoring is demonstrated with the TEP and TPFF data sets. The inclusion of sparsity in the IncMSSFA method provides superior monitoring performance at the cost of a quadratic complexity in terms of data dimensionality. This complexity is still an improvement over the cubic complexity of RSFA

FAULT DIAGNOSIS TOOLS IN MULTIVARIATE STATISTICAL PROCESS AND QUALITY CONTROL

[EN] An accurate fault diagnosis of both, faults sensors and real process faults have become more and more important for process monitoring (minimize downtime, increase safety of plant operation and reduce the manufacturing cost). Quick and correct fault diagnosis is required in order to put back on track our processes or products before safety or quality can be compromised. In the study and comparison of the fault diagnosis methodologies, this thesis distinguishes between two different scenarios, methods for multivariate statistical quality control (MSQC) and methods for latent-based multivariate statistical process control: (Lb-MSPC). In the first part of the thesis the state of the art on fault diagnosis and identification (FDI) is introduced. The second part of the thesis is devoted to the fault diagnosis in multivariate statistical quality control (MSQC). The rationale of the most extended methods for fault diagnosis in supervised scenarios, the requirements for their implementation, their strong points and their drawbacks and relationships are discussed. The performance of the methods is compared using different performance indices in two different process data sets and simulations. New variants and methods to improve the diagnosis performance in MSQC are also proposed. The third part of the thesis is devoted to the fault diagnosis in latent-based multivariate statistical process control (Lb-MSPC). The rationale of the most extended methods for fault diagnosis in supervised Lb-MSPC is described and one of our proposals, the Fingerprints contribution plots (FCP) is introduced. Finally the thesis presents and compare the performance results of these diagnosis methods in Lb-MSPC. The diagnosis results in two process data sets are compared using a new strategy based in the use of the overall sensitivity and specificity[ES] La realización de un diagnóstico preciso de los fallos, tanto si se trata de fallos de sensores como si se trata de fallos de procesos, ha llegado a ser algo de vital importancia en la monitorización de procesos (reduce las paradas de planta, incrementa la seguridad de la operación en planta y reduce los costes de producción). Se requieren diagnósticos rápidos y correctos si se quiere poder recuperar los procesos o productos antes de que la seguridad o la calidad de los mismos se pueda ver comprometida. En el estudio de las diferentes metodologías para el diagnóstico de fallos esta tesis distingue dos escenarios diferentes, métodos para el control de estadístico multivariante de la calidad (MSQC) y métodos para el control estadístico de procesos basados en el uso de variables latentes (Lb-MSPC). En la primera parte de esta tesis se introduce el estado del arte sobre el diagnóstico e identificación de fallos (FDI). La segunda parte de la tesis está centrada en el estudio del diagnóstico de fallos en control estadístico multivariante de la calidad. Se describen los fundamentos de los métodos más extendidos para el diagnóstico en escenarios supervisados, sus requerimientos para su implementación sus puntos fuertes y débiles y sus posibles relaciones. Los resultados de diagnóstico de los métodos es comparado usando diferentes índices sobre los datos procedentes de dos procesos reales y de diferentes simulaciones. En la tesis se proponen nuevas variantes que tratan de mejorar los resultados obtenidos en MSQC. La tercera parte de la tesis está dedicada al diagnóstico de fallos en control estadístico multivariante de procesos basados en el uso de modelos de variables latentes (Lb-MSPC). Se describe los fundamentos de los métodos mas extendidos en el diagnóstico de fallos en Lb-MSPC supervisado y se introduce una de nuestras propuestas, el fingerprint contribution plot (FCP). Finalmente la tesis presenta y compara los resultados de diagnóstico de los métodos propuestos en Lb-MSPC. Los resultados son comparados sobre los datos de dos procesos usando una nueva estrategia basada en el uso de la sensitividad y especificidad promedia.[CA] La realització d'un diagnòstic precís de les fallades, tant si es tracta de fallades de sensors com si es tracta de fallades de processos, ha arribat a ser de vital importància en la monitorització de processos (reduïx les parades de planta, incrementa la seguretat de l'operació en planta i reduïx els costos de producció) . Es requerixen diagnòstics ràpids i correctes si es vol poder recuperar els processos o productes abans de que la seguretat o la qualitat dels mateixos es puga veure compromesa. En l'estudi de les diferents metodologies per al diagnòstic de fallades esta tesi distingix dos escenaris diferents, mètodes per al control estadístic multivariant de la qualitat (MSQC) i l mètodes per al control estadístic de processos basats en l'ús de variables latents (Lb-MSPC). En la primera part d'esta tesi s'introduïx l'estat de l'art sobre el diagnòstic i identificació de fallades (FDI). La segona part de la tesi està centrada en l'estudi del diagnòstic de fallades en control estadístic multivariant de la qualitat. Es descriuen els fonaments dels mètodes més estesos per al diagnòstic en escenaris supervisats, els seus requeriments per a la seua implementació els seus punts forts i febles i les seues possibles relacions. Els resultats de diagnòstic dels mètodes és comparat utilitzant diferents índexs sobre les dades procedents de dos processos reals i de diferents simulacions. En la tesi es proposen noves variants que tracten de millorar els resultats obtinguts en MSQC. La tercera part de la tesi està dedicada al diagnòstic de fallades en control estadístic multivariant de processos basat en l'ús de models de variables latents (Lb-MSPC). Es descriu els fonaments dels mètodes més estesos en el diagnòstic de fallades en MSPC supervisat i s'introdueix una nova proposta, el fingerprint contribution plot (FCP). Finalment la tesi presenta i compara els resultats de diagnòstic dels mètodes proposats en MSPC. Els resultats són comparats sobre les dades de dos processos utilitzant una nova estratègia basada en l'ús de la sensibilitat i especificitat mitjana.Vidal Puig, S. (2016). FAULT DIAGNOSIS TOOLS IN MULTIVARIATE STATISTICAL PROCESS AND QUALITY CONTROL [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/61292TESI

Fault detection and root cause diagnosis using dynamic Bayesian network

This thesis presents two real time process fault detection and diagnosis (FDD) techniques incorporating process data and prior knowledge. Unlike supervised monitoring techniques, both these methods can perform without having any prior information of a fault. In the first part of this research, a hybrid methodology is developed combining principal component analysis (PCA), Bayesian network (BN) and multiple uncertain (likelihood) evidence to improve the diagnostic capacity of PCA and existing PCA-BN schemes with hard evidence based updating. A dynamic BN (DBN) based FDD methodology is proposed in the later part of this work which provides detection and accurate diagnosis by a single tool. Furthermore, fault propagation pathway is analyzed using the predictive feature of a BN and cause-effect relationships among the process variables. Proposed frameworks are successfully validated by applying to several process models