A Temporal Pyramid Pooling-Based Convolutional Neural Network for Remaining Useful Life Prediction

Remaining Useful Life (RUL) prediction is a key issue in Prognostics and Health Management (PHM). Accurate RUL assessments are crucial for predictive maintenance planning. Deep neural networks such as Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) have been widely applied in RUL prediction due to their powerful feature learning capabilities in dealing with high-dimensional sensor data. The sliding time window method with a predefined window size is typically employed to generate data samples to train such deep neural networks. However, the disadvantage of using a fixed-size time window is that we might not be able to apply the resulting predictive model to predict new sensor data whose length is shorter than the predetermined time window size. Besides, as the length of sensor data varies, the traditional unchanged and subjectively set time window size may be inappropriate and impair the prediction model’s performance. Therefore, we propose a Temporal Pyramid Pooling-Based Convolutional Neural Network (TPP-CNN) to increase model practicability and prediction accuracy. With the temporal pyramid pooling module, we can generate data samples of arbitrary time window sizes and use them as inputs of CNN. In the training phase, CNN can learn to capture temporal dependencies of different lengths since we feed in samples with different time window sizes. In this novel manner, the learned model can be used to test data with arbitrary sizes, and its predictive ability is also improved. The proposed TPP-CNN model is validated on the C-MPASS turbofan engine dataset, and the experiments have demonstrated its effectiveness

Data augmentation to improve the performance of ensemble learning for system failure prediction with limited observations

Ensemble learning has been widely used to improve the performance and robustness of machine learning algorithms on time series data. However, in real operational processes where the observed data is limited, it hinders the capability of ensemble learning algorithms. To address the challenge of limited observed data, this paper proposes a novel three-layer ensemble learning framework by use of data augmentation. Firstly, multiple classical time series augmentation methods are applied to increase the size of the data set. Subsequently, after pre-processing, these augmented data is trained by multiple basic learners with K-fold cross-validation as the first layer of the developed ensemble learning framework. The outputs of the first layer are integrated via LASSO to further improve the prediction performance, which serves as the second layer of the developed framework. Finally, the third-layer output is generated by averaging the prediction of the second layer and the output from an improved Long-Short Term Memory model that provides prediction based on the augmented data. A case study on a real wastewater treatment plant is used to illustrate the effectiveness of the proposed method

A Hybrid Prognostic Approach Based on Deep Learning for the Degradation Prediction of Machinery

Remaining useful life (RUL) prediction is of great significance for prognostic and health management (PHM) as it can achieve more reliable and effective maintenance strategies. With the advances in the field of deep learning, data-driven methods have provided promising prognostic prediction results. Hence, this research presents a data-driven prognostic approach based on deep learning models for predicting the RUL of mechanical systems effectively. Multiple separable convolution layers, a bidirectional Long Short-Term Memory (LSTM) layer, and fully-connected layers (FCL) are included in the proposed network, named the SC-BLSTM, to accomplish more accurate prognostic prediction from the raw degradation data acquired by different sensors. The proposed SC-BLSTM approach aims to learn complex and nonlinear features from the input data and capture temporal dependencies from the learned features. The presented approach in this research is tested and verified on the degradation data of turbofan engines (C-MAPSS dataset) from NASA. The result demonstrated that the SC-BLSTM is able to achieve more effective RUL prediction compared with some existing prognostic models

Blended Multi-Modal Deep ConvNet Features for Diabetic Retinopathy Severity Prediction

Diabetic Retinopathy (DR) is one of the major causes of visual impairment and blindness across the world. It is usually found in patients who suffer from diabetes for a long period. The major focus of this work is to derive optimal representation of retinal images that further helps to improve the performance of DR recognition models. To extract optimal representation, features extracted from multiple pre-trained ConvNet models are blended using proposed multi-modal fusion module. These final representations are used to train a Deep Neural Network (DNN) used for DR identification and severity level prediction. As each ConvNet extracts different features, fusing them using 1D pooling and cross pooling leads to better representation than using features extracted from a single ConvNet. Experimental studies on benchmark Kaggle APTOS 2019 contest dataset reveals that the model trained on proposed blended feature representations is superior to the existing methods. In addition, we notice that cross average pooling based fusion of features from Xception and VGG16 is the most appropriate for DR recognition. With the proposed model, we achieve an accuracy of 97.41%, and a kappa statistic of 94.82 for DR identification and an accuracy of 81.7% and a kappa statistic of 71.1% for severity level prediction. Another interesting observation is that DNN with dropout at input layer converges more quickly when trained using blended features, compared to the same model trained using uni-modal deep features.Comment: 18 pages, 8 figures, published in Electronics MDPI journa

A Literature Review of Fault Diagnosis Based on Ensemble Learning

The accuracy of fault diagnosis is an important indicator to ensure the reliability of key equipment systems. Ensemble learning integrates different weak learning methods to obtain stronger learning and has achieved remarkable results in the field of fault diagnosis. This paper reviews the recent research on ensemble learning from both technical and field application perspectives. The paper summarizes 87 journals in recent web of science and other academic resources, with a total of 209 papers. It summarizes 78 different ensemble learning based fault diagnosis methods, involving 18 public datasets and more than 20 different equipment systems. In detail, the paper summarizes the accuracy rates, fault classification types, fault datasets, used data signals, learners (traditional machine learning or deep learning-based learners), ensemble learning methods (bagging, boosting, stacking and other ensemble models) of these fault diagnosis models. The paper uses accuracy of fault diagnosis as the main evaluation metrics supplemented by generalization and imbalanced data processing ability to evaluate the performance of those ensemble learning methods. The discussion and evaluation of these methods lead to valuable research references in identifying and developing appropriate intelligent fault diagnosis models for various equipment. This paper also discusses and explores the technical challenges, lessons learned from the review and future development directions in the field of ensemble learning based fault diagnosis and intelligent maintenance