Wasserstein Distance based Deep Adversarial Transfer Learning for Intelligent Fault Diagnosis

The demand of artificial intelligent adoption for condition-based maintenance strategy is astonishingly increased over the past few years. Intelligent fault diagnosis is one critical topic of maintenance solution for mechanical systems. Deep learning models, such as convolutional neural networks (CNNs), have been successfully applied to fault diagnosis tasks for mechanical systems and achieved promising results. However, for diverse working conditions in the industry, deep learning suffers two difficulties: one is that the well-defined (source domain) and new (target domain) datasets are with different feature distributions; another one is the fact that insufficient or no labelled data in target domain significantly reduce the accuracy of fault diagnosis. As a novel idea, deep transfer learning (DTL) is created to perform learning in the target domain by leveraging information from the relevant source domain. Inspired by Wasserstein distance of optimal transport, in this paper, we propose a novel DTL approach to intelligent fault diagnosis, namely Wasserstein Distance based Deep Transfer Learning (WD-DTL), to learn domain feature representations (generated by a CNN based feature extractor) and to minimize the distributions between the source and target domains through adversarial training. The effectiveness of the proposed WD-DTL is verified through 3 transfer scenarios and 16 transfer fault diagnosis experiments of both unsupervised and supervised (with insufficient labelled data) learning. We also provide a comprehensive analysis of the network visualization of those transfer tasks

Fault Diagnosis of Transfer Learning Equipment Based on Cloud Edge Collaboration + Confrontation Network

With the continuous improvement of product quality, production efficiency, and complexity, higher requirements are put forward for the reliability and stability of equipment, and the difficulty of real-time diagnosis of faults and functional failures is also increasing. The traditional fault diagnosis methods based on signal processing and Convolutional neural network cannot meet the requirements of on-site online real-time fault diagnosis of equipment. One is that the vibration signals on the industrial site are superimposed on each other, nonlinear and unstable and traditional feature extraction methods take a long time, resulting in unstable extraction results. Second, massive data and fault diagnosis algorithms need rich computing and storage resources. The traditional Convolutional neural network method conflicts with the real-time response requirements of fault diagnosis. At the same time, different models of fault diagnosis models have poor generalization ability, and the diagnostic accuracy is not high or even impossible to diagnose. To solve the above problems, this paper proposes a fault diagnosis method based on industrial Internet platform, which is equipment cloud edge collaboration + adaptive countermeasure network Transfer learning. On the edge side, the vibration signals collected from key components of the model are processed using empirical mode decomposition (EEMD) to solve the problem of signal nonlinearity and stationarity. In the cloud, EEMD signals of different models are decomposed into source domain and target domain for confrontation training, which is used as the input of the improved domain adversarial network model DANN (Domain Adversarial Neural Networks), so as to improve the accuracy of fault diagnosis of different models by using cloud computing power and the improved adversarial network Transfer learning algorithm. Through the analysis of experimental data, this paper verifies that the model after the confrontation network Transfer learning is more accurate than the traditional fault diagnosis method. Through the coordination of computing resources and real-time requirements, real-time diagnosis of cloud side collaborative bearing fault is realized

Cross-condition fault diagnosis of an aircraft environmental control system (ECS) by transfer learning

Fault diagnosis models based on machine learning are often subjected to degradation in performance when dealing with data that are differently distributed than the training data. Such an occasion is common in reality because machines usually operate under various conditions. Transfer learning is a solution for the performance degradation of cross-condition fault diagnosis problems. This paper studies how transfer learning algorithms transfer component analysis (TCA) and joint distribution alignment (JDA) improve the cross-condition fault diagnosis accuracy of an aircraft environmental control system (ECS). Both methods work by transforming the source and target domain data into a feature space where their distributions are aligned to allow a uniform classifier to act accurately in both domains. This paper discovered that both TCA and JDA produce significantly more accurate results than traditional methods on target domains with unlabelled ECS data taken at different operating conditions than the source domain. Additionally, when dealing with unlabelled data from unknown conditions bearing a different composition of classes in the target domain, TCA is found to be more robust and accurate, generating an average predictive accuracy of 95.22%, which demonstrates the ability of transfer learning in solving similar problems in the real-world application of fault diagnosis

딥러닝 기반 고장 진단을 위한 정보 활용 극대화 기법 개발

학위논문(박사) -- 서울대학교대학원 : 공과대학 기계항공공학부, 2021.8. 윤병동.기계 시스템의 예기치 않은 고장은 많은 산업 분야에서 막대한 사회적, 경제적 손실을 야기할 수 있다. 갑작스런 고장을 감지하고 예방하여 기계 시스템의 신뢰성을 높이기 위해 데이터 기반 고장 진단 기술을 개발하기 위한 연구가 활발하게 이루어지고 있다. 고장 진단 기술의 목표는 대상 기계 시스템의 고장 발생을 가능한 빨리 감지하고 진단하는 것이다. 최근 합성곱 신경망 기법을 포함한 딥러닝 기반 고장 진단 기술은 자율적인 특성인자(feature) 학습이 가능하고 높은 진단 성능을 얻을 수 있다는 장점이 있어 활발히 연구되고 있다. 그러나 딥러닝 기반의 고장 진단 기술을 개발함에 있어 해결해야 할 몇 가지 문제점들이 존재한다. 먼저, 신경망 구조를 깊게 쌓음으로써 풍부한 계층적 특성인자들을 배울 수 있고, 이를 통해 향상된 성능을 얻을 수 있다. 그러나 기울기(gradient) 정보 흐름의 비효율성과 과적합 문제로 인해 모델이 깊어질수록 학습이 어렵게 된다는 문제가 있다. 다음으로, 높은 성능의 고장 진단 모델을 학습하기 위해서는 충분한 양의 레이블 데이터(labeled data)가 확보돼야 한다. 그러나 실제 현장에서 운용되고 있는 기계 시스템의 경우, 충분한 양의 데이터와 레이블 정보를 얻는 것이 어려운 경우가 많다. 따라서 이러한 문제들을 해결하고 진단 성능을 향상시키기 위한 새로운 딥러닝 기반 고장 진단 기술의 개발이 필요하다. 본 박사학위논문에서는 딥러닝 기반 고장 진단 기술의 성능을 향상시키기 위한 세가지 정보 활용 극대화 기법에 대한 연구로 1) 딥러닝 아키텍처 내 기울기 정보 흐름을 향상시키기 위한 새로운 딥러닝 구조 연구, 2) 파라미터 전이 및 삼중항 손실을 기반으로 불충분한 데이터 및 노이즈 조건 하 강건하고 차별적인 특성인자 학습에 대한 연구, 3) 다른 도메인으로부터 레이블 정보를 전이시켜 사용하는 도메인 적응 기반 고장 진단 기법 연구를 제안한다. 첫 번째 연구에서는 딥러닝 모델 내 기울기 정보 흐름을 개선하기 위한 향상된 합성곱 신경망 기반 구조를 제안한다. 본 연구에서는 다양한 계층의 아웃풋(feature map)을 직접 연결함으로써 향상된 정보 흐름을 얻을 수 있으며, 그 결과 진단 모델을 효율적으로 학습하는 것이 가능하다. 또한 차원 축소 모듈을 통해 학습 파라미터 수를 크게 줄임으로써 학습 효율성을 높일 수 있다. 두 번째 연구에서는 파라미터 전이 및 메트릭 학습 기반 고장 진단 기법을 제안한다. 본 연구는 데이터가 불충분하고 노이즈가 많은 조건 하에서도 높은 고장 진단 성능을 얻기 위해 강건하고 차별적인 특성인자 학습을 가능하게 한다. 먼저, 풍부한 소스 도메인 데이터를 사용해 훈련된 사전학습모델을 타겟 도메인으로 전이해 사용함으로써 강건한 진단 방법을 개발할 수 있다. 또한, semi-hard 삼중항 손실 함수를 사용함으로써 각 상태 레이블에 따라 데이터가 더 잘 분리되도록 해주는 특성인자를 학습할 수 있다. 세 번째 연구에서는 레이블이 지정되지 않은(unlabeled) 대상 도메인에서의 고장 진단 성능을 높이기 위한 레이블 정보 전이 전략을 제안한다. 우리가 목표로 하는 대상 도메인에서의 고장 진단 방법을 개발하기 위해 다른 소스 도메인에서 얻은 레이블 정보가 전이되어 활용된다. 동시에 새롭게 고안한 의미론적 클러스터링 손실(semantic clustering loss)을 여러 특성인자 수준에 적용함으로써 차별적인 도메인 불변 기능을 학습한다. 결과적으로 도메인 불변 특성을 가지며 의미론적으로 잘 분류되는 특성인자를 효과적으로 학습할 수 있음을 증명하였다.Unexpected failures of mechanical systems can lead to substantial social and financial losses in many industries. In order to detect and prevent sudden failures and to enhance the reliability of mechanical systems, significant research efforts have been made to develop data-driven fault diagnosis techniques. The purpose of fault diagnosis techniques is to detect and identify the occurrence of abnormal behaviors in the target mechanical systems as early as possible. Recently, deep learning (DL) based fault diagnosis approaches, including the convolutional neural network (CNN) method, have shown remarkable fault diagnosis performance, thanks to their autonomous feature learning ability. Still, there are several issues that remain to be solved in the development of robust and industry-applicable deep learning-based fault diagnosis techniques. First, by stacking the neural network architectures deeper, enriched hierarchical features can be learned, and therefore, improved performance can be achieved. However, due to inefficiency in the gradient information flow and overfitting problems, deeper models cannot be trained comprehensively. Next, to develop a fault diagnosis model with high performance, it is necessary to obtain sufficient labeled data. However, for mechanical systems that operate in real-world environments, it is not easy to obtain sufficient data and label information. Consequently, novel methods that address these issues should be developed to improve the performance of deep learning based fault diagnosis techniques. This dissertation research investigated three research thrusts aimed toward maximizing the use of information to improve the performance of deep learning based fault diagnosis techniques, specifically: 1) study of the deep learning structure to enhance the gradient information flow within the architecture, 2) study of a robust and discriminative feature learning method under insufficient and noisy data conditions based on parameter transfer and triplet loss, and 3) investigation of a domain adaptation based fault diagnosis method that propagates the label information across different domains. The first research thrust suggests an advanced CNN-based architecture to improve the gradient information flow within the deep learning model. By directly connecting the feature maps of different layers, the diagnosis model can be trained efficiently thanks to enhanced information flow. In addition, the dimension reduction module also can increase the training efficiency by significantly reducing the number of trainable parameters. The second research thrust suggests a parameter transfer and metric learning based fault diagnosis method. The proposed approach facilitates robust and discriminative feature learning to enhance fault diagnosis performance under insufficient and noisy data conditions. The pre-trained model trained using abundant source domain data is transferred and used to develop a robust fault diagnosis method. Moreover, a semi-hard triplet loss function is adopted to learn the features with high separability, according to the class labels. Finally, the last research thrust proposes a label information propagation strategy to increase the fault diagnosis performance in the unlabeled target domain. The label information obtained from the source domain is transferred and utilized for developing fault diagnosis methods in the target domain. Simultaneously, the newly devised semantic clustering loss is applied at multiple feature levels to learn discriminative, domain-invariant features. As a result, features that are not only semantically well-clustered but also domain-invariant can be effectively learned.Chapter 1 Introduction 1 1.1 Motivation 1 1.2 Research Scope and Overview 3 1.3 Dissertation Layout 6 Chapter 2 Technical Background and Literature Review 8 2.1 Fault Diagnosis Techniques for Mechanical Systems 8 2.1.1 Fault Diagnosis Techniques 10 2.1.2 Deep Learning Based Fault Diagnosis Techniques 15 2.2 Transfer Learning 22 2.3 Metric Learning 28 2.4 Summary and Discussion 30 Chapter 3 Direct Connection Based Convolutional Neural Network (DC-CNN) for Fault Diagnosis 31 3.1 Directly Connected Convolutional Module 33 3.2 Dimension Reduction Module 34 3.3 Input Vibration Image Generation 36 3.4 DC-CNN-Based Fault Diagnosis Method 40 3.5 Experimental Studies and Results 45 3.5.1 Experiment and Data Description 45 3.5.2 Compared Methods 48 3.5.3 Diagnosis Performance Results 51 3.5.4 The Number of Trainable Parameters 56 3.5.5 Visualization of the Learned Features 58 3.5.6 Robustness of Diagnosis Performance 62 3.6 Summary and Discussion 67 Chapter 4 Robust and Discriminative Feature Learning for Fault Diagnosis Under Insufficient and Noisy Data Conditions 68 4.1 Parameter transfer learning 70 4.2 Robust Feature Learning Based on the Pre-trained model 72 4.3 Discriminative Feature Learning Based on the Triplet loss 77 4.4 Robust and Discriminative Feature Learning for Fault Diagnosis 80 4.5 Experimental Studies and Results 84 4.5.1 Experiment and Data Description 84 4.5.2 Compared Methods 85 4.5.3 Experimental Results Under Insufficient Data Conditions 86 4.5.4 Experimental Results Under Noisy Data Conditions 92 4.6 Summary and Discussion 95 Chapter 5 A Domain Adaptation with Semantic Clustering (DASC) Method for Fault Diagnosis 96 5.1 Unsupervised Domain Adaptation 101 5.2 CNN-based Diagnosis Model 104 5.3 Learning of Domain-invariant Features 105 5.4 Domain Adaptation with Semantic Clustering 107 5.5 Proposed DASC-based Fault Diagnosis Method 109 5.6 Experimental Studies and Results 114 5.6.1 Experiment and Data Description 114 5.6.2 Compared Methods 117 5.6.3 Scenario I: Different Operating Conditions 118 5.6.4 Scenario II: Different Rotating Machinery 125 5.6.5 Analysis and Discussion 131 5.7 Summary and Discussion 140 Chapter 6 Conclusion 141 6.1 Contributions and Significance 141 6.2 Suggestions for Future Research 143 References 146 국문 초록 154박

A Deep Transfer Model With Wasserstein Distance Guided Multi-Adversarial Networks for Bearing Fault Diagnosis Under Different Working Conditions

In recent years, intelligent fault diagnosis technology with the deep learning algorithm has been widely used in the manufacturing industry for substituting time-consuming human analysis method to enhance the efficiency of fault diagnosis. The rolling bearing as the connection between the rotor and support is the crucial component in rotating equipment. However, the working condition of the rolling bearing is under changing with complex operation demand, which will significantly degrade the performance of the intelligent fault diagnosis method. In this paper, a new deep transfer model based on Wasserstein distance guided multi-adversarial networks (WDMAN) is proposed to address this problem. The WDMAN model exploits complex feature space structures to enable the transfer of different data distributions based on multiple domain critic networks. The essence of our method is learning the shared feature representation by minimizing the Wasserstein distance between the source domain and target domain distribution in an adversarial training way. The experiment results demonstrate that our model outperforms the state-of-the-art methods on rolling bearing fault diagnosis under different working conditions. The t-distributed stochastic neighbor embedding (t-SNE) technology is used to visualize the learned domain invariant feature and investigate the transferability behind the great performance of our proposed model

Smart filter aided domain adversarial neural network: An unsupervised domain adaptation method for fault diagnosis in noisy industrial scenarios

The application of unsupervised domain adaptation (UDA)-based fault diagnosis methods has shown significant efficacy in industrial settings, facilitating the transfer of operational experience and fault signatures between different operating conditions, different units of a fleet or between simulated and real data. However, in real industrial scenarios, unknown levels and types of noise can amplify the difficulty of domain alignment, thus severely affecting the diagnostic performance of deep learning models. To address this issue, we propose an UDA method called Smart Filter-Aided Domain Adversarial Neural Network (SFDANN) for fault diagnosis in noisy industrial scenarios. The proposed methodology comprises two steps. In the first step, we develop a smart filter that dynamically enforces similarity between the source and target domain data in the time-frequency domain. This is achieved by combining a learnable wavelet packet transform network (LWPT) and a traditional wavelet packet transform module. In the second step, we input the data reconstructed by the smart filter into a domain adversarial neural network (DANN). To learn domain-invariant and discriminative features, the learnable modules of SFDANN are trained in a unified manner with three objectives: time-frequency feature proximity, domain alignment, and fault classification. We validate the effectiveness of the proposed SFDANN method based on two fault diagnosis cases: one involving fault diagnosis of bearings in noisy environments and another involving fault diagnosis of slab tracks in a train-track-bridge coupling vibration system, where the transfer task involves transferring from numerical simulations to field measurements. Results show that compared to other representative state of the art UDA methods, SFDANN exhibits superior performance and remarkable stability

Rolling Bearing Incipient Fault Diagnosis Method Based on Improved Transfer Learning with Hybrid Feature Extraction

Data-driven based rolling bearing fault diagnosis has been widely investigated in recent years. However, in real-world industry scenarios, the collected labeled samples are normally in a different data distribution. Moreover, the features of bearing fault in the early stages are extremely inconspicuous. Due to the above mentioned problems, it is difficult to diagnose the incipient fault under different scenarios by adopting the conventional data-driven methods. Therefore, in this paper a new unsupervised rolling bearing incipient fault diagnosis approach based on transfer learning is proposed, with a novel feature extraction method based on a statistical algorithm, wavelet scattering network, and a stacked auto-encoder network. Then, the geodesic flow kernel algorithm is adopted to align the feature vectors on the Grassmann manifold, and the k-nearest neighbor classifier is used for fault classification. The experiment is conducted based on two bearing datasets, the bearing fault dataset of Case Western Reserve University and the bearing fault dataset of Xi’an Jiaotong University. The experiment results illustrate the effectiveness of the proposed approach on solving the different data distribution and incipient bearing fault diagnosis issues

A Novel Transfer Learning Method Utilizing Acoustic and Vibration Signals for Rotating Machinery Fault Diagnosis

Fault diagnosis of rotating machinery plays a important role for the safety and stability of modern industrial systems. However, there is a distribution discrepancy between training data and data of real-world operation scenarios, which causing the decrease of performance of existing systems. This paper proposed a transfer learning based method utilizing acoustic and vibration signal to address this distribution discrepancy. We designed the acoustic and vibration feature fusion MAVgram to offer richer and more reliable information of faults, coordinating with a DNN-based classifier to obtain more effective diagnosis representation. The backbone was pre-trained and then fine-tuned to obtained excellent performance of the target task. Experimental results demonstrate the effectiveness of the proposed method, and achieved improved performance compared to STgram-MFN

불충분한 고장 데이터에 대한 딥러닝 기반 회전 기계 진단기술 학습방법 연구

학위논문(박사)--서울대학교 대학원 :공과대학 기계항공공학부,2020. 2. 윤병동.Deep Learning is a promising approach for fault diagnosis in mechanical applications. Deep learning techniques are capable of processing lots of data in once, and modelling them into desired diagnostic model. In industrial fields, however, we can acquire tons of data but barely useful including fault or failure data because failure in industrial fields is usually unacceptable. To cope with this insufficient fault data problem to train diagnostic model for rotating machinery, this thesis proposes three research thrusts: 1) filter-envelope blocks in convolution neural networks (CNNs) to incorporate the preprocessing steps for vibration signal; frequency filtering and envelope extraction for more optimal solution and reduced efforts in building diagnostic model, 2) cepstrum editing based data augmentation (CEDA) for diagnostic dataset consist of vibration signals from rotating machinery, and 3) selective parameter freezing (SPF) for efficient parameter transfer in transfer learning. The first research thrust proposes noble types of functional blocks for neural networks in order to learn robust feature to the vibration data. Conventional neural networks including convolution neural network (CNN), is tend to learn biased features when the training data is acquired from small cases of conditions. This can leads to unfavorable performance to the different conditions or other similar equipment. Therefore this research propose two neural network blocks which can be incorporated to the conventional neural networks and minimize the preprocessing steps, filter block and envelope block. Each block is designed to learn frequency filter and envelope extraction function respectively, in order to induce the neural network to learn more robust and generalized features from limited vibration samples. The second thrust presents a new data augmentation technique specialized for diagnostic data of vibration signals. Many data augmentation techniques exist for image data with no consideration for properties of vibration data. Conventional techniques for data augmentation, such as flipping, rotating, or shearing are not proper for 1-d vibration data can harm the natural property of vibration signal. To augment vibration data without losing the properties of its physics, the proposed method generate new samples by editing the cepstrum which can be done by adjusting the cepstrum component of interest. By doing reverse transform to the edited cepstrum, the new samples is obtained and this results augmented dataset which leads to higher accuracy for the diagnostic model. The third research thrust suggests a new parameter repurposing method for parameter transfer, which is used for transfer learning. The proposed SPF selectively freezes transferred parameters from source network and re-train only unnecessary parameters for target domain to reduce overfitting and preserve useful source features when the target data is limited to train diagnostic model.딥러닝은 기계 응용 분야의 결함 진단을 위한 유망한 접근 방식이다. 딥러닝 기술은 많은 양의 데이터를 학습하여 진단 모델의 개발을 용이하게 한다. 그러나 산업 분야에서는 많은 양의 데이터를 얻을 수 없거나 얻을 수 있더라도 고장 데이터는 일반적으로 획득하기 매우 어렵기 때문에 딥러닝 방법의 사용은 쉽지 않다. 회전 기계의 진단을 위하여 딥러닝을 학습시킬 때 발생하는 고장 데이터 부족 문제에 대처하기 위해 이 논문은 3 가지 연구를 제안한다. 1) 향상된 진동 특징 학습을 위한 필터-엔벨롭 네트워크 구조 2) 진동데이터 생성을 위한 Cepstrum 기반 데이터 증량법3) 전이 학습에서 효율적인 파라미터 전이를 위한 선택적 파라미터 동결법. 첫 번째 연구는 진동 데이터에 대한 강건한 특징을 배우기 위해 신경망에 대한 새로운 형태의 네트워크 블록들을 제안한다. 합성곱 신경망을 포함하는 종래의 신경망은 학습 데이터가 작은 경우에 데이터로부터 편향된 특징을 배우는 경향이 있으며, 이는 다른 조건에서 작동하는 경우나 다른 시스템에 대해 적용되었을 때 낮은 진단 성능을 보인다. 따라서 본 연구는 기존의 신경망에 함께 사용될 수 있는 필터 블록 및 엔벨롭 블록을 제안한다. 각 블록은 주파수 필터와 엔벨롭 추출 기능을 네트워크 내에서 스스로 학습하여 신경망이 제한된 학습 진동데이터로부터 보다 강건하고 일반화 된 특징을 학습하도록 한다. 두 번째 연구는 진동 신호의 진단 데이터에 특화된 새로운 데이터 증량법을 제안한다. 뒤집기, 회전 또는 전단과 같은 데이터 확대를 위한 이미지 데이터를 위한 기존의 기술이 1 차원 진동 데이터에 적합하지 않으며, 진동 신호의 물리적 특성에 맞지 않는 신호를 생성할 수 있다. 물리적 특성을 잃지 않고 진동 데이터를 증량하기 위해 제안된 방법은 cepstrum의 주요성분을 추출하고 조정하여 역 cepstrum을 수행하는 방식으로 새로운 샘플을 생성한다. 제안된 방법을 통해 데이터를 생성하여 증량돤 데이터세트는 진단 모델 학습에 대해 성능향상을 가져온다. 세 번째 연구는 전이 학습에 사용되는 파라미터 전이를 위한 새로운 파라미터 재학습법을 제안한다. 제안된 선택적 파라미터 동결법은 소스 네트워크에서 전이된 파라미터를 선택적으로 동결하고 대상 도메인에 대해 불필요한 파라미터만 재학습하여 대상 데이터가 진단 모델에 재학습될 때의 과적합을 줄이고 소스 네트워크의 성능을 보존한다. 제안된 세 방법은 독립적으로 또는 동시에 진단모델에 사용되어 부족한 고장데이터로 인한 진단성능의 감소를 경감하거나 더 높은 성능을 이끌어낼 수 있다.Chapter 1 Introduction 13 1.1 Motivation 13 1.2 Research Scope and Overview 15 1.3 Structure of the Thesis 19 Chapter 2 Literature Review 20 2.1 Deep Neural Networks 20 2.2 Transfer Learning and Parameter Transfer 23 Chapter 3 Description of Testbed Data 26 3.1 Bearing Data I: Case Western Reserve University Data 26 3.2 Bearing Data II: Accelerated Life Test Test-bed 27 Chapter 4 Filter-Envelope Blocks in Neural Network for Robust Feature Learning 32 4.1 Preliminary Study of Problems In Use of CNN for Vibration Signals 34 4.1.1 Class Confusion Problem of CNN Model to Different Conditions 34 4.1.2 Benefits of Frequency Filtering and Envelope Extraction for Fault Diagnosis in Vibration Signals 37 4.2 Proposed Network Block 1: Filter Block 41 4.2.1 Spectral Feature Learning in Neural Network 42 4.2.2 FIR Band-pass Filter in Neural Network 45 4.2.3 Result and Discussion 48 4.3 Proposed Neural Block 2: Envelope Block 48 4.3.1 Max-Average Pooling Block for Envelope Extraction 51 4.3.2 Adaptive Average Pooling for Learnable Envelope Extractor 52 4.3.3 Result and Discussion 54 4.4 Filter-Envelope Network for Fault Diagnosis 56 4.4.1 Combinations of Filter-Envelope Blocks for the use of Rolling Element Bearing Fault Diagnosis 56 4.4.2 Summary and Discussion 58 Chapter 5 Cepstrum Editing Based Data Augmentation for Vibration Signals 59 5.1 Brief Review of Data Augmentation for Deep Learning 59 5.1.1 Image Augmentation to Enlarge Training Dataset 59 5.1.2 Data Augmentation for Vibration Signal 61 5.2 Cepstrum Editing based Data Augmentation 62 5.2.1 Cepstrum Editing as a Signal Preprocessing 62 5.2.2 Cepstrum Editing based Data Augmentation 64 5.3 Results and Discussion 65 5.3.1 Performance validation to rolling element bearing diagnosis 65 Chapter 6 Selective Parameter Freezing for Parameter Transfer with Small Dataset 71 6.1 Overall Procedure of Selective Parameter Freezing 72 6.2 Determination Sensitivity of Source Network Parameters 75 6.3 Case Study 1: Transfer to Different Fault Size 76 6.3.1 Performance by hyperparameter α 77 6.3.2 Effect of the number of training samples and network size 79 6.4 Case Study 2: Transfer from Artificial to Natural Fault 81 6.4.1 Diagnostic performance for proposed method 82 6.4.2 Visualization of frozen parameters by hyperparameter α 83 6.4.3 Visual inspection of feature space 85 6.5 Conclusion 87 Chapter 7 91 7.1 Contributions and Significance 91Docto