Maximizing Model Generalization for Machine Condition Monitoring with Self-Supervised Learning and Federated Learning

Deep Learning (DL) can diagnose faults and assess machine health from raw condition monitoring data without manually designed statistical features. However, practical manufacturing applications remain extremely difficult for existing DL methods. Machine data is often unlabeled and from very few health conditions (e.g., only normal operating data). Furthermore, models often encounter shifts in domain as process parameters change and new categories of faults emerge. Traditional supervised learning may struggle to learn compact, discriminative representations that generalize to these unseen target domains since it depends on having plentiful classes to partition the feature space with decision boundaries. Transfer Learning (TL) with domain adaptation attempts to adapt these models to unlabeled target domains but assumes similar underlying structure that may not be present if new faults emerge. This study proposes focusing on maximizing the feature generality on the source domain and applying TL via weight transfer to copy the model to the target domain. Specifically, Self-Supervised Learning (SSL) with Barlow Twins may produce more discriminative features for monitoring health condition than supervised learning by focusing on semantic properties of the data. Furthermore, Federated Learning (FL) for distributed training may also improve generalization by efficiently expanding the effective size and diversity of training data by sharing information across multiple client machines. Results show that Barlow Twins outperforms supervised learning in an unlabeled target domain with emerging motor faults when the source training data contains very few distinct categories. Incorporating FL may also provide a slight advantage by diffusing knowledge of health conditions between machines

Transfer Learning with Deep Convolutional Neural Network (CNN) for Pneumonia Detection using Chest X-ray

Pneumonia is a life-threatening disease, which occurs in the lungs caused by either bacterial or viral infection. It can be life-endangering if not acted upon in the right time and thus an early diagnosis of pneumonia is vital. The aim of this paper is to automatically detect bacterial and viral pneumonia using digital x-ray images. It provides a detailed report on advances made in making accurate detection of pneumonia and then presents the methodology adopted by the authors. Four different pre-trained deep Convolutional Neural Network (CNN)- AlexNet, ResNet18, DenseNet201, and SqueezeNet were used for transfer learning. 5247 Bacterial, viral and normal chest x-rays images underwent preprocessing techniques and the modified images were trained for the transfer learning based classification task. In this work, the authors have reported three schemes of classifications: normal vs pneumonia, bacterial vs viral pneumonia and normal, bacterial and viral pneumonia. The classification accuracy of normal and pneumonia images, bacterial and viral pneumonia images, and normal, bacterial and viral pneumonia were 98%, 95%, and 93.3% respectively. This is the highest accuracy in any scheme than the accuracies reported in the literature. Therefore, the proposed study can be useful in faster-diagnosing pneumonia by the radiologist and can help in the fast airport screening of pneumonia patients.Comment: 13 Figures, 5 tables. arXiv admin note: text overlap with arXiv:2003.1314

비표지 고장 데이터와 유중가스분석데이터를 이용한 딥러닝기반 주변압기 고장진단 연구

학위논문(박사) -- 서울대학교대학원 : 공과대학 기계항공공학부, 2021.8. 소재웅.오늘날 산업의 급속한 발전과 고도화로 인해 안전하고 신뢰할 수 있는 전력 계통에 대한 수요는 더욱 중요해지고 있다. 따라서 실제 산업 현장에서는 주변압기의 안전한 작동을 위해 상태를 정확하게 진단할 수 있는 prognostics and health management (PHM)와 같은 기술이 필요하다. 주변압기 진단을 위해 개발된 다양한 방법 중 인공지능(AI) 기반 접근법은 산업과 학계에서 많은 관심을 받고 있다. 더욱이 방대한 데이터와 함께 높은 성능을 달성하는 딥 러닝 기술은 주변압기 고장 진단의 학자들에게 높은 관심을 갖게 해줬다. 그 이유는 딥 러닝 기술이 시스템의 도메인 지식을 깊이 이해할 필요 없이 대량의 데이터만 주어진다면 복잡한 시스템이라도 사용자의 목적에 맞게 그 해답을 찾을 수 있기 때문에 딥 러닝에 대한 관심은 주변압기 고장 진단 분야에서 특히 두드러졌다. 그러나, 이러한 뛰어난 진단 성능은 아직 실제 주변압기 산업에서는 많은 관심을 얻고 있지는 못한 것으로 알려졌다. 그 이유는 산업현장의 비표지데이터와 소량의 고장데이터 때문에 우수한 딥러닝기반의 고장 진단 모델들을 개발하기 어렵다. 따라서 본 학위논문에서는 주변압기 산업에서 현재 대두되고 있는 세가지 이슈를 연구하였다. 1) 건전성 평면 시각화 이슈, 2) 데이터 부족 이슈, 3) 심각도 이슈 들을 극복하기 위한 딥 러닝 기반 고장 진단 연구를 진행하였다. 소개된 세가지 이슈들을 개선하기 위해 본 학위논문은 세 가지 연구를 제안하였다. 첫 번째 연구는 보조 감지 작업이 있는 준지도 자동 인코더를 통해 건전성 평면을 제안하였다. 제안된 방법은 변압기 열하 특성을 시각화 할 수 있다. 또한, 준지도 접근법을 활용하기 때문에 방대한 비표지데이터 그리고 소수의 표지데이터만으로 구현될 수 있다. 제안방법은 주변압기 건전성을 건전성 평면과 함께 시각화하고, 매우 적은 소수의 레이블 데이터만으로 주변압기 고장을 진단한다. 두 번째 연구는 규칙 기반 Duval 방법을 AI 기반 deep neural network (DNN)과 융합(bridge)하는 새로운 프레임워크를 제안하였다. 이 방법은 룰기반의 Duval을 사용하여 비표지데이터를 수도 레이블링한다 (pseudo-labeling). 또한, AI 기반 DNN은 정규화 기술과 매개 변수 전이 학습을 적용하여 노이즈가 있는 pseudo-label 데이터를 학습하는데 사용된다. 개발된 기술은 방대한양의 비표지데이터를 룰기반으로 일차적으로 진단한 결과와 소수의 실제 고장데이터와 함께 학습데이터로 훈련하였을 때 기존의 진단 방법보다 획기적인 향상을 가능케 한다. 끝으로, 세 번째 연구는 고장 타입을 진단할 뿐만 아니라 심각도 또한 진단하는 기술을 제안하였다. 이때 두 상태의 레이블링된 고장 타입과 심각도 사이에는 불균일한 데이터 분포로 이루어져 있다. 그 이유는 심각도의 경우 레이블링이 항상 되어 있지만 고장 타입의 경우는 실제 주변압기로부터 고장 타입 데이터를 얻기가 매우 어렵기 때문이다. 따라서, 본 연구에서 세번째로 개발한 기술은 오늘날 데이터 생성에 매우 우수한 성능을 달성하고 있는 generative adversarial network (GAN)를 통해 불균형한 두 상태를 균일화 작업을 수행하는 동시에 고장 모드와 심각도를 진단하는 모델을 개발하였다.Due to the rapid development and advancement of today’s industry, the demand for safe and reliable power distribution and transmission lines is becoming more critical; thus, prognostics and health management (hereafter, PHM) is becoming more important in the power transformer industry. Among various methods developed for power transformer diagnosis, the artificial intelligence (AI) based approach has received considerable interest from academics. Specifically, deep learning technology, which offers excellent performance when used with vast amounts of data, is also rapidly gaining the spotlight in the academic field of transformer fault diagnosis. The interest in deep learning has been especially noticed in the field of fault diagnosis, because deep learning algorithms can be applied to complex systems that have large amounts of data, without the need for a deep understanding of the domain knowledge of the system. However, the outstanding performance of these diagnosis methods has not yet gained much attention in the power transformer PHM industry. The reason is that a large amount of unlabeled and a small amount of fault data always restrict their deep-learning-based diagnosis methods in the power transformer PHM industry. Therefore, in this dissertation research, deep-learning-based fault diagnosis methods are developed to overcome three issues that currently prevent this type of diagnosis in industrial power transformers: 1) the visualization of health feature space issue, 2) the insufficient data issue, and 3) the severity issue. To cope with these challenges, this thesis is composed of three research thrusts. The first research thrust develops a health feature space via a semi-supervised autoencoder with an auxiliary detection task. The proposed method can visualize a monotonic health trendability of the transformer’s degradation properties. Further, thanks to the use of a semi-supervised approach, the method is applicable to situations with a large amount of unlabeled and a small amount labeled data (a situation common in industrial datasets). Next, the second research thrust proposes a new framework, that bridges the rule-based Duval method with an AI-based deep neural network (BDD). In this method, the rule-based Duval method is utilized to pseudo-label a large amount of unlabeled data. Furthermore, the AI-based DNN is used to apply regularization techniques and parameter transfer learning to learn the noisy pseudo-labelled data. Finally, the third thrust not only identifies fault types but also indicates a severity level. However, the balance between labeled fault types and the severity level is imbalanced in real-world data. Therefore, in the proposed method, diagnosis of fault types – with severity levels – under imbalanced conditions is addressed by utilizing a generative adversarial network with an auxiliary classifier. The validity of the proposed methods is demonstrated by studying massive unlabeled dissolved gas analysis (DGA) data, provided by the Korea Electric Power Company (KEPCO), and sparse labeled data, provided by the IEC TC 10 database. Each developed method could be used in industrial fields that use power transformers to monitor the health feature space, consider severity level, and diagnose transformer faults under extremely insufficient labeled fault data.Chapter 1 Introduction 1 1.1 Motivation 1 1.2 Research Scope and Overview 4 1.3 Dissertation Layout 7 Chapter 2 Literature Review 9 2.1 A Brief Overview of Rule-Based Fault Diagnosis 9 2.2 A Brief Overview of Conventional AI-Based Fault Diagnosis 11 Chapter 3 Extracting Health Feature Space via Semi-Supervised Autoencoder with an Auxiliary Task (SAAT) 13 3.1 Backgrounds of Semi-supervised autoencoder (SSAE) 15 3.1.1 Autoencoder: Unsupervised Feature Extraction 15 3.1.2 Softmax Classifier: Supervised Classification 17 3.1.3 Semi-supervised Autoencoder 18 3.2 Input DGA Data Preprocessing 20 3.3 SAAT-Based Fault Diagnosis Method 21 3.3.1 Roles of the Auxiliary Detection Task 23 3.3.2 Architecture of the Proposed SAAT 27 3.3.3 Health Feature Space Visualization 29 3.3.4 Overall Procedure of the Proposed SAAT-based Fault Diagnosis 30 3.4 Performance Evaluation of SAAT 31 3.4.1 Data Description and Implementation 31 3.4.2 An Outline of Four Comparative Studies and Quantitative Evaluation Metrics 33 3.4.3 Experimental Results and Discussion 36 3.5 Summary and Discussion 49 Chapter 4 Learning from Even a Weak Teacher: Bridging Rule-based Duval Weak Supervision and a Deep Neural Network (BDD) for Diagnosing Transformer 51 4.1 Backgrounds of BDD 53 4.1.1 Rule-based method: Duval Method 53 4.1.2 Deep learning Based Method: Deep Neural Network 54 4.1.3 Parameter Transfer 55 4.2 BDD Based Fault Diagnosis 56 4.2.1 Problem Statement 56 4.2.2 Framework of the Proposed BDD 57 4.2.3 Overall Procedure of BDD-based Fault Diagnosis 63 4.3 Performance Evaluation of the BDD 64 4.3.1 Description of Data and the DNN Architecture 64 4.3.2 Experimental Results and Discussion 66 4.4 Summary and Discussion 76 Chapter 5 Generative Adversarial Network with Embedding Severity DGA Level 79 5.1 Backgrounds of Generative Adversarial Network 81 5.2 GANES based Fault Diagnosis 82 5.2.1 Training Strategy of GANES 82 5.2.2 Overall procedure of GANES 87 5.3 Performance Evaluation of GANES 91 5.3.1 Description of Data 91 5.3.2 Outlines of Experiments 91 5.3.3 Preliminary Experimental Results of Various GANs 95 5.3.4 Experiments for the Effectiveness of Embedding Severity DGA Level 99 5.4 Summary and Discussion 105 Chapter 6 Conclusion 106 6.1 Contributions and Significance 106 6.2 Suggestions for Future Research 108 References 110 국문 초록 127박

Few-Shot Learning Approaches for Fault Diagnosis Using Vibration Data: A Comprehensive Review

Fault detection and diagnosis play a crucial role in ensuring the reliability and safety of modern industrial systems. For safety and cost considerations, critical equipment and systems in industrial operations are typically not allowed to operate in severe fault states. Moreover, obtaining labeled samples for fault diagnosis often requires significant human effort. This results in limited labeled data for many application scenarios. Thus, the focus of attention has shifted towards learning from a small amount of data. Few-shot learning has emerged as a solution to this challenge, aiming to develop models that can effectively solve problems with only a few samples. This approach has gained significant traction in various fields, such as computer vision, natural language processing, audio and speech, reinforcement learning, robotics, and data analysis. Surprisingly, despite its wide applicability, there have been limited investigations or reviews on applying few-shot learning to the field of mechanical fault diagnosis. In this paper, we provide a comprehensive review of the relevant work on few-shot learning in mechanical fault diagnosis from 2018 to September 2023. By examining the existing research, we aimed to shed light on the potential of few-shot learning in this domain and offer valuable insights for future research directions

A Novel Ensemble Adaptive Sparse Bayesian Transfer Learning Machine for Nonlinear Large-Scale Process Monitoring.

Process monitoring plays an important role in ensuring the safety and stable operation of equipment in a large-scale process. This paper proposes a novel data-driven process monitoring framework, termed the ensemble adaptive sparse Bayesian transfer learning machine (EAdspB-TLM), for nonlinear fault diagnosis. The proposed framework has the following advantages: Firstly, the probabilistic relevance vector machine (PrRVM) under Bayesian framework is re-derived so that it can be used to forecast the plant operating conditions. Secondly, we extend the PrRVM method and assimilate transfer learning into the sparse Bayesian learning framework to provide it with the transferring ability. Thirdly, the source domain (SD) data are re-enabled to alleviate the issue of insufficient training data. Finally, the proposed EAdspB-TLM framework was effectively applied to monitor a real wastewater treatment process (WWTP) and a Tennessee Eastman chemical process (TECP). The results further demonstrate that the proposed method is feasible

Going Deep in Medical Image Analysis: Concepts, Methods, Challenges and Future Directions

Medical Image Analysis is currently experiencing a paradigm shift due to Deep Learning. This technology has recently attracted so much interest of the Medical Imaging community that it led to a specialized conference in `Medical Imaging with Deep Learning' in the year 2018. This article surveys the recent developments in this direction, and provides a critical review of the related major aspects. We organize the reviewed literature according to the underlying Pattern Recognition tasks, and further sub-categorize it following a taxonomy based on human anatomy. This article does not assume prior knowledge of Deep Learning and makes a significant contribution in explaining the core Deep Learning concepts to the non-experts in the Medical community. Unique to this study is the Computer Vision/Machine Learning perspective taken on the advances of Deep Learning in Medical Imaging. This enables us to single out `lack of appropriately annotated large-scale datasets' as the core challenge (among other challenges) in this research direction. We draw on the insights from the sister research fields of Computer Vision, Pattern Recognition and Machine Learning etc.; where the techniques of dealing with such challenges have already matured, to provide promising directions for the Medical Imaging community to fully harness Deep Learning in the future

Deep learning for diabetic retinopathy detection and classification based on fundus images: A review.

Diabetic Retinopathy is a retina disease caused by diabetes mellitus and it is the leading cause of blindness globally. Early detection and treatment are necessary in order to delay or avoid vision deterioration and vision loss. To that end, many artificial-intelligence-powered methods have been proposed by the research community for the detection and classification of diabetic retinopathy on fundus retina images. This review article provides a thorough analysis of the use of deep learning methods at the various steps of the diabetic retinopathy detection pipeline based on fundus images. We discuss several aspects of that pipeline, ranging from the datasets that are widely used by the research community, the preprocessing techniques employed and how these accelerate and improve the models' performance, to the development of such deep learning models for the diagnosis and grading of the disease as well as the localization of the disease's lesions. We also discuss certain models that have been applied in real clinical settings. Finally, we conclude with some important insights and provide future research directions

Novel deep cross-domain framework for fault diagnosis or rotary machinery in prognostics and health management

Improving the reliability of engineered systems is a crucial problem in many applications in various engineering fields, such as aerospace, nuclear energy, and water declination industries. This requires efficient and effective system health monitoring methods, including processing and analyzing massive machinery data to detect anomalies and performing diagnosis and prognosis. In recent years, deep learning has been a fast-growing field and has shown promising results for Prognostics and Health Management (PHM) in interpreting condition monitoring signals such as vibration, acoustic emission, and pressure due to its capacity to mine complex representations from raw data. This doctoral research provides a systematic review of state-of-the-art deep learning-based PHM frameworks, an empirical analysis on bearing fault diagnosis benchmarks, and a novel multi-source domain adaptation framework. It emphasizes the most recent trends within the field and presents the benefits and potentials of state-of-the-art deep neural networks for system health management. Besides, the limitations and challenges of the existing technologies are discussed, which leads to opportunities for future research. The empirical study of the benchmarks highlights the evaluation results of the existing models on bearing fault diagnosis benchmark datasets in terms of various performance metrics such as accuracy and training time. The result of the study is very important for comparing or testing new models. A novel multi-source domain adaptation framework for fault diagnosis of rotary machinery is also proposed, which aligns the domains in both feature-level and task-level. The proposed framework transfers the knowledge from multiple labeled source domains into a single unlabeled target domain by reducing the feature distribution discrepancy between the target domain and each source domain. Besides, the model can be easily reduced to a single-source domain adaptation problem. Also, the model can be readily updated to unsupervised domain adaptation problems in other fields such as image classification and image segmentation. Further, the proposed model is modified with a novel conditional weighting mechanism that aligns the class-conditional probability of the domains and reduces the effect of irrelevant source domain which is a critical issue in multi-source domain adaptation algorithms. The experimental verification results show the superiority of the proposed framework over state-of-the-art multi-source domain-adaptation models

Can AI help in screening Viral and COVID-19 pneumonia?

Coronavirus disease (COVID-19) is a pandemic disease, which has already caused thousands of causalities and infected several millions of people worldwide. Any technological tool enabling rapid screening of the COVID-19 infection with high accuracy can be crucially helpful to healthcare professionals. The main clinical tool currently in use for the diagnosis of COVID-19 is the Reverse transcription polymerase chain reaction (RT-PCR), which is expensive, less-sensitive and requires specialized medical personnel. X-ray imaging is an easily accessible tool that can be an excellent alternative in the COVID-19 diagnosis. This research was taken to investigate the utility of artificial intelligence (AI) in the rapid and accurate detection of COVID-19 from chest X-ray images. The aim of this paper is to propose a robust technique for automatic detection of COVID-19 pneumonia from digital chest X-ray images applying pre-trained deep-learning algorithms while maximizing the detection accuracy. A public database was created by the authors combining several public databases and also by collecting images from recently published articles. The database contains a mixture of 423 COVID-19, 1485 viral pneumonia, and 1579 normal chest X-ray images. Transfer learning technique was used with the help of image augmentation to train and validate several pre-trained deep Convolutional Neural Networks (CNNs). The networks were trained to classify two different schemes: i) normal and COVID-19 pneumonia; ii) normal, viral and COVID-19 pneumonia with and without image augmentation. The classification accuracy, precision, sensitivity, and specificity for both the schemes were 99.7%, 99.7%, 99.7% and 99.55% and 97.9%, 97.95%, 97.9%, and 98.8%, respectively.Comment: 12 pages, 9 Figure