Predicting Hard Disk Failures in Data Centers Using Temporal Convolutional Neural Networks

In modern data centers, storage system failures are major contributors to downtimes and maintenance costs. Predicting these failures by collecting measurements from disks and analyzing them with machine learning techniques can effectively reduce their impact, enabling timely maintenance. While there is a vast literature on this subject, most approaches attempt to predict hard disk failures using either classic machine learning solutions, such as Random Forests (RFs) or deep Recurrent Neural Networks (RNNs). In this work, we address hard disk failure prediction using Temporal Convolutional Networks (TCNs), a novel type of deep neural network for time series analysis. Using a real-world dataset, we show that TCNs outperform both RFs and RNNs. Specifically, we can improve the Fault Detection Rate (FDR) of ≈ 7.5% (FDR = 89.1%) compared to the state-of-the-art, while simultaneously reducing the False Alarm Rate (FAR = 0.052%). Moreover, we explore the network architecture design space showing that TCNs are consistently superior to RNNs for a given model size and complexity and that even relatively small TCNs can reach satisfactory performance. All the codes to reproduce the results presented in this paper are available at https://github.com/ABurrello/tcn-hard-disk-failure-prediction

Data-driven Models for Remaining Useful Life Estimation of Aircraft Engines and Hard Disk Drives

Failure of physical devices can cause inconvenience, loss of money, and sometimes even deaths. To improve the reliability of these devices, we need to know the remaining useful life (RUL) of a device at a given point in time. Data-driven approaches use data from a physical device to build a model that can estimate the RUL. They have shown great performance and are often simpler than traditional model-based approaches. Typical statistical and machine learning approaches are often not suited for sequential data prediction. Recurrent Neural Networks are designed to work with sequential data but suffer from the vanishing gradient problem over time. Therefore, I explore the use of Long Short-Term Memory (LSTM) networks for RUL prediction. I perform two experiments. First, I train bidirectional LSTM networks on the Backblaze hard-disk drive dataset. I achieve an accuracy of 96.4\% on a 60 day time window, state-of-the-art performance. Additionally, I use a unique standardization method that standardizes each hard drive instance independently and explore the benefits and downsides of this approach. Finally, I train LSTM models on the NASA N-CMAPSS dataset to predict aircraft engine remaining useful life. I train models on each of the eight sub-datasets, achieving a RMSE of 6.304 on one of the sub-datasets, the second-best in the current literature. I also compare an LSTM network\u27s performance to the performance of a Random Forest and Temporal Convolutional Neural Network model, demonstrating the LSTM network\u27s superior performance. I find that LSTM networks are capable predictors for device remaining useful life and show a thorough model development process that can be reproduced to develop LSTM models for various RUL prediction tasks. These models will be able to improve the reliability of devices such as aircraft engines and hard-disk drives

Model-Augmented Estimation of Conditional Mutual Information for Feature Selection

Markov blanket feature selection, while theoretically optimal, is generally challenging to implement. This is due to the shortcomings of existing approaches to conditional independence (CI) testing, which tend to struggle either with the curse of dimensionality or computational complexity. We propose a novel two-step approach which facilitates Markov blanket feature selection in high dimensions. First, neural networks are used to map features to low-dimensional representations. In the second step, CI testing is performed by applying the $k$-NN conditional mutual information estimator to the learned feature maps. The mappings are designed to ensure that mapped samples both preserve information and share similar information about the target variable if and only if they are close in Euclidean distance. We show that these properties boost the performance of the $k$-NN estimator in the second step. The performance of the proposed method is evaluated on both synthetic and real data.Comment: Accepted to UAI 202

Intelligent fault detection and classification based on hybrid deep learning methods for Hardware-in-the-Loop test of automotive software systems

Hardware-in-the-Loop (HIL) has been recommended by ISO 26262 as an essential test bench for determining the safety and reliability characteristics of automotive software systems (ASSs). However, due to the complexity and the huge amount of data recorded by the HIL platform during the testing process, the conventional data analysis methods used for detecting and classifying faults based on the human expert are not realizable. Therefore, the development of effective means based on the historical data set is required to analyze the records of the testing process in an efficient manner. Even though data-driven fault diagnosis is superior to other approaches, selecting the appropriate technique from the wide range of Deep Learning (DL) techniques is challenging. Moreover, the training data containing the automotive faults are rare and considered highly confidential by the automotive industry. Using hybrid DL techniques, this study proposes a novel intelligent fault detection and classification (FDC) model to be utilized during the V-cycle development process, i.e., the system integration testing phase. To this end, an HIL-based real-time fault injection framework is used to generate faulty data without altering the original system model. In addition, a combination of the Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) is employed to build the model structure. In this study, eight types of sensor faults are considered to cover the most common potential faults in the signals of ASSs. As a case study, a gasoline engine system model is used to demonstrate the capabilities and advantages of the proposed method and to verify the performance of the model. The results prove that the proposed method shows better detection and classification performance compared to other standalone DL methods. Specifically, the overall detection accuracies of the proposed structure in terms of precision, recall and F1-score are 98.86%, 98.90% and 98.88%, respectively. For classification, the experimental results also demonstrate the superiority under unseen test data with an average accuracy of 98.8%

Failure Diagnosis and Prognosis of Safety Critical Systems: Applications in Aerospace Industries

Many safety-critical systems such as aircraft, space crafts, and large power plants are required to operate in a reliable and efficient working condition without any performance degradation. As a result, fault diagnosis and prognosis (FDP) is a research topic of great interest in these systems. FDP systems attempt to use historical and current data of a system, which are collected from various measurements to detect faults, diagnose the types of possible failures, predict and manage failures in advance. This thesis deals with FDP of safety-critical systems. For this purpose, two critical systems including a multifunctional spoiler (MFS) and hydro-control value system are considered, and some challenging issues from the FDP are investigated. This research work consists of three general directions, i.e., monitoring, failure diagnosis, and prognosis. The proposed FDP methods are based on data-driven and model-based approaches. The main aim of the data-driven methods is to utilize measurement data from the system and forecast the remaining useful life (RUL) of the faulty components accurately and efficiently. In this regard, two dierent methods are developed. A modular FDP method based on a divide and conquer strategy is presented for the MFS system. The modular structure contains three components:1) fault diagnosis unit, 2) failure parameter estimation unit and 3) RUL unit. The fault diagnosis unit identifies types of faults based on an integration of neural network (NN) method and discrete wavelet transform (DWT) technique. Failure parameter estimation unit observes the failure parameter via a distributed neural network. Afterward, the RUL of the system is predicted by an adaptive Bayesian method. In another work, an innovative data-driven FDP method is developed for hydro-control valve systems. The idea is to use redundancy in multi-sensor data information and enhance the performance of the FDP system. Therefore, a combination of a feature selection method and support vector machine (SVM) method is applied to select proper sensors for monitoring of the hydro-valve system and isolate types of fault. Then, adaptive neuro-fuzzy inference systems (ANFIS) method is used to estimate the failure path. Similarly, an online Bayesian algorithm is implemented for forecasting RUL. Model-based methods employ high-delity physics-based model of a system for prognosis task. In this thesis, a novel model-based approach based on an integrated extended Kalman lter (EKF) and Bayesian method is introduced for the MFS system. To monitor the MFS system, a residual estimation method using EKF is performed to capture the progress of the failure. Later, a transformation is utilized to obtain a new measure to estimate the degradation path (DP). Moreover, the recursive Bayesian algorithm is invoked to predict the RUL. Finally, relative accuracy (RA) measure is utilized to assess the performance of the proposed methods