How to Do Machine Learning with Small Data? -- A Review from an Industrial Perspective

Artificial intelligence experienced a technological breakthrough in science, industry, and everyday life in the recent few decades. The advancements can be credited to the ever-increasing availability and miniaturization of computational resources that resulted in exponential data growth. However, because of the insufficient amount of data in some cases, employing machine learning in solving complex tasks is not straightforward or even possible. As a result, machine learning with small data experiences rising importance in data science and application in several fields. The authors focus on interpreting the general term of "small data" and their engineering and industrial application role. They give a brief overview of the most important industrial applications of machine learning and small data. Small data is defined in terms of various characteristics compared to big data, and a machine learning formalism was introduced. Five critical challenges of machine learning with small data in industrial applications are presented: unlabeled data, imbalanced data, missing data, insufficient data, and rare events. Based on those definitions, an overview of the considerations in domain representation and data acquisition is given along with a taxonomy of machine learning approaches in the context of small data

Novel deep cross-domain framework for fault diagnosis or rotary machinery in prognostics and health management

Improving the reliability of engineered systems is a crucial problem in many applications in various engineering fields, such as aerospace, nuclear energy, and water declination industries. This requires efficient and effective system health monitoring methods, including processing and analyzing massive machinery data to detect anomalies and performing diagnosis and prognosis. In recent years, deep learning has been a fast-growing field and has shown promising results for Prognostics and Health Management (PHM) in interpreting condition monitoring signals such as vibration, acoustic emission, and pressure due to its capacity to mine complex representations from raw data. This doctoral research provides a systematic review of state-of-the-art deep learning-based PHM frameworks, an empirical analysis on bearing fault diagnosis benchmarks, and a novel multi-source domain adaptation framework. It emphasizes the most recent trends within the field and presents the benefits and potentials of state-of-the-art deep neural networks for system health management. Besides, the limitations and challenges of the existing technologies are discussed, which leads to opportunities for future research. The empirical study of the benchmarks highlights the evaluation results of the existing models on bearing fault diagnosis benchmark datasets in terms of various performance metrics such as accuracy and training time. The result of the study is very important for comparing or testing new models. A novel multi-source domain adaptation framework for fault diagnosis of rotary machinery is also proposed, which aligns the domains in both feature-level and task-level. The proposed framework transfers the knowledge from multiple labeled source domains into a single unlabeled target domain by reducing the feature distribution discrepancy between the target domain and each source domain. Besides, the model can be easily reduced to a single-source domain adaptation problem. Also, the model can be readily updated to unsupervised domain adaptation problems in other fields such as image classification and image segmentation. Further, the proposed model is modified with a novel conditional weighting mechanism that aligns the class-conditional probability of the domains and reduces the effect of irrelevant source domain which is a critical issue in multi-source domain adaptation algorithms. The experimental verification results show the superiority of the proposed framework over state-of-the-art multi-source domain-adaptation models

Signal processing and graph-based semi-supervised learning-based fault diagnosis for direct online induction motors

In this thesis, fault diagnosis approaches for direct online induction motors are proposed using signal processing and graph-based semi-supervised learning (GSSL). These approaches are developed using experimental data obtained in the lab for two identical 0.25 HP three-phase squirrel-cage induction motors. Various electrical and mechanical single- and multi-faults are applied to each motor during experiments. Three-phase stator currents and three-dimensional vibration signals are recorded simultaneously in each experiment. In this thesis, Power Spectral Density (PSD)-based stator current amplitude spectrum analysis and one-dimensional Complex Continuous Wavelet Transform (CWT)-based stator current time-scale spectrum analysis are employed to detect broken rotor bar (BRB) faults. An effective single- and multi-fault diagnosis approach is developed using GSSL, where discrete wavelet transform (DWT) is applied to extract features from experimental stator current and vibration data. Three GSSL algorithms (Local and global consistency (LGC), Gaussian field and harmonic functions (GFHF), and greedy-gradient max-cut (GGMC)) are adopted and compared in this study. To enable machine learning for untested motor operating conditions, mathematical equations to calculate features for untested conditions are developed using curve fitting and features obtained from experimental data of tested conditions

Semi-supervised transfer learning methodology for fault detection and diagnosis in air-handling units

Heating, ventilation and air-conditioning (HVAC) systems are the major energy consumers among buildings’ equipment. Reliable fault detection and diagnosis schemes can effectively reduce their energy consumption and maintenance costs. In this respect, data-driven approaches have shown impressive results, but their accuracy depends on the availability of representative data to train the models, which is not common in real applications. For this reason, transfer learning is attracting growing attention since it tackles the problem by leveraging the knowledge between datasets, increasing the representativeness of fault scenarios. However, to date, research on transfer learning for heating, ventilation and air-conditioning has mostly been focused on learning algorithmic, overlooking the importance of a proper domain similarity analysis over the available data. Thus, this study proposes the design of a transfer learning approach based on a specific data selection methodology to tackle dissimilarity issues. The procedure is supported by neural network models and the analysis of eventual prediction uncertainties resulting from the assessment of the target application samples. To verify the proposed methodology, it is applied to a semi-supervised transfer learning case study composed of two publicly available air-handling unit datasets containing some fault scenarios. Results emphasize the potential of the proposed domain dissimilarity analysis reaching a classification accuracy of 92% under a transfer learning framework, an increase of 37% in comparison to classical approaches.Objectius de Desenvolupament Sostenible::11 - Ciutats i Comunitats SosteniblesObjectius de Desenvolupament Sostenible::12 - Producció i Consum ResponsablesPostprint (published version

A Literature Review of Fault Diagnosis Based on Ensemble Learning

The accuracy of fault diagnosis is an important indicator to ensure the reliability of key equipment systems. Ensemble learning integrates different weak learning methods to obtain stronger learning and has achieved remarkable results in the field of fault diagnosis. This paper reviews the recent research on ensemble learning from both technical and field application perspectives. The paper summarizes 87 journals in recent web of science and other academic resources, with a total of 209 papers. It summarizes 78 different ensemble learning based fault diagnosis methods, involving 18 public datasets and more than 20 different equipment systems. In detail, the paper summarizes the accuracy rates, fault classification types, fault datasets, used data signals, learners (traditional machine learning or deep learning-based learners), ensemble learning methods (bagging, boosting, stacking and other ensemble models) of these fault diagnosis models. The paper uses accuracy of fault diagnosis as the main evaluation metrics supplemented by generalization and imbalanced data processing ability to evaluate the performance of those ensemble learning methods. The discussion and evaluation of these methods lead to valuable research references in identifying and developing appropriate intelligent fault diagnosis models for various equipment. This paper also discusses and explores the technical challenges, lessons learned from the review and future development directions in the field of ensemble learning based fault diagnosis and intelligent maintenance

Advanced Fault Diagnosis and Health Monitoring Techniques for Complex Engineering Systems

Over the last few decades, the field of fault diagnostics and structural health management has been experiencing rapid developments. The reliability, availability, and safety of engineering systems can be significantly improved by implementing multifaceted strategies of in situ diagnostics and prognostics. With the development of intelligence algorithms, smart sensors, and advanced data collection and modeling techniques, this challenging research area has been receiving ever-increasing attention in both fundamental research and engineering applications. This has been strongly supported by the extensive applications ranging from aerospace, automotive, transport, manufacturing, and processing industries to defense and infrastructure industries

FAULT DETECTION FRAMEWORK FOR IMBALANCED AND SPARSELY-LABELED DATA SETS USING SELF-ORGANIZING MAPS

While machine learning techniques developed for fault detection usually assume that the classes in the training data are balanced, in real-world applications, this is seldom the case. These techniques also usually require labeled training data, obtaining which is a costly and time-consuming task. In this context, a data-driven framework is developed to detect faults in systems where the condition monitoring data is either imbalanced or consists of mostly unlabeled observations. To mitigate the problem of class imbalance, self-organizing maps (SOMs) are trained in a supervised manner, using the same map size for both classes of data, prior to performing classification. The optimal SOM size for balancing the classes in the data, the size of the neighborhood function, and the learning rate, are determined by performing multiobjective optimization on SOM quality measures such as quantization error and information entropy; and performance measures such as training time and classification error. For training data sets which contain a majority of unlabeled observations, the transductive semi-supervised approach is used to label the neurons of an unsupervised SOM, before performing supervised SOM classification on the test data set. The developed framework is validated using artificial and real-world fault detection data sets

Machine Learning in Manufacturing towards Industry 4.0: From ‘For Now’ to ‘Four-Know’

While attracting increasing research attention in science and technology, Machine Learning (ML) is playing a critical role in the digitalization of manufacturing operations towards Industry 4.0. Recently, ML has been applied in several fields of production engineering to solve a variety of tasks with different levels of complexity and performance. However, in spite of the enormous number of ML use cases, there is no guidance or standard for developing ML solutions from ideation to deployment. This paper aims to address this problem by proposing an ML application roadmap for the manufacturing industry based on the state-of-the-art published research on the topic. First, this paper presents two dimensions for formulating ML tasks, namely, ’Four-Know’ (Know-what, Know-why, Know-when, Know-how) and ’Four-Level’ (Product, Process, Machine, System). These are used to analyze ML development trends in manufacturing. Then, the paper provides an implementation pipeline starting from the very early stages of ML solution development and summarizes the available ML methods, including supervised learning methods, semi-supervised methods, unsupervised methods, and reinforcement methods, along with their typical applications. Finally, the paper discusses the current challenges during ML applications and provides an outline of possible directions for future developments