Machine Learning with a Reject Option: A survey

Machine learning models always make a prediction, even when it is likely to be inaccurate. This behavior should be avoided in many decision support applications, where mistakes can have severe consequences. Albeit already studied in 1970, machine learning with rejection recently gained interest. This machine learning subfield enables machine learning models to abstain from making a prediction when likely to make a mistake. This survey aims to provide an overview on machine learning with rejection. We introduce the conditions leading to two types of rejection, ambiguity and novelty rejection, which we carefully formalize. Moreover, we review and categorize strategies to evaluate a model's predictive and rejective quality. Additionally, we define the existing architectures for models with rejection and describe the standard techniques for learning such models. Finally, we provide examples of relevant application domains and show how machine learning with rejection relates to other machine learning research areas

Similarity-based anomaly score for fleet-based condition monitoring

An increased number of industrial assets are monitored during their daily use, producing large amounts of data. This data allows us to better monitor the health status of these asset, enabling predictive maintenance to reduce risks and costs caused by unexpected machine failure. Many condition monitoring approaches focus on assessing a machine's health status individually. Often, these approaches require historical data sets or handcrafted fault indicators. However, multiple industrial applications involve monitoring multiple similar operating machines, a fleet. By assuming the healthy behavior for the majority of the machine, deviating signatures can indicate a machine fault. In this work, we extend our previous proposed framework for fleet-based condition monitoring (Hendrickx et al.). This framework uses interpretable machine learning techniques to automatically evaluate assets within a fleet while incorporating domain knowledge if available. It is designed with four building blocks. In the first block, the user defines a similarity measure to compare machines. This measure can be both data-driven as based on domain knowledge. The second block clusters the machines based on this similarity measure. The third block assesses the health status of a machine by assigning an anomaly score where higher scores represent more deviating behavior. Finally, each of these blocks is visualized in the fourth block to guide a domain expert to set up and gain trust in the framework. The anomaly score proposed in our previous work has two shortcomings. First, its value can change very abruptly; a slight deviation can cause a machine's anomaly score to change from very low to very high. Second, the score does not accurately represent the anomalousness of a machine. A machine with the highest anomaly score is not necessarily the most deviating. Finally, the anomaly score is assigned to a group of machines. It is thus hard to assess the health status of an individual machine. As a consequence, this anomaly score offers little insights into a machine's performance. The contribution of this paper is a new implementation of the anomaly score block. Instead of basing our anomaly score on the clustering, we make use of the machine's similarities within the fleet. This solves the shortcomings of the previous anomaly score and defines an individualized, continuous scoring mechanism that represents the anomalousness of a machine.status: Published onlin

Customer-correlated drive cycle generation for realistic fuel consumption and driving emissions testing

Fuel consumption and driving emissions are critical performance targets for the automotive industry. Extensive tests are therefore performed on engine test benches or full vehicle chassis dynamometers. In order to make these laboratory tests an accurate representation of the real-world fuel consumption and driving emissions, it is crucial to apply realistic drive cycles. This work first presents a procedure to characterize the real-world fuel consumption and driving emissions based on a customer fleet measurement database. Second, it will be demonstrated how a customer-correlated drive cycle can be generated which matches this real-world customer usage.status: publishe

A fleet-wide approach for condition monitoring of similar machines using time-series clustering

© Springer Nature Switzerland AG 2019. The application of machine learning to fault diagnosis allows automated condition monitoring of machines, leading to reduced maintenance costs and increased machine availability. Traditional approaches train a machine learning algorithm to identify specific faults or operational settings. Therefore, these approaches cannot always cope with a dynamic industrial environment. However, an industrial installation often contains multiple machines of the same type, which enables a fleet-based analysis. This type of analysis compares machines to tackle the challenges of a dynamic environment. In this paper a novel method is proposed for analyzing a fleet of machines operating under similar conditions in the same area by using inter-machine comparisons. The proposed methodology consists of two steps. First, the inter-machine difference is calculated using dynamic time warping by using the amount of warping as measure. This method allows comparing the measured signals even when fluctuations are present. Second, a clustering method uses the inter-machine similarity to identify groups of machines that operate in a similar manner. The generation of a fault usually causes a change in the raw signals and diagnostic features. As a result, the inter-machine difference between the faulty machine and the rest of the fleet will increase, leading to the creation of a separate group that contains the faulty machine. The methodology is evaluated and validated on phase current signals measured on a fleet of electrical drivetrains, where a phase unbalance fault is introduced in some of the drivetrains for a specific time duration.status: publishe

A general anomaly detection framework for fleet-based condition monitoring of machines

Machine failures decrease up-time and can lead to extra repair costs or even to human casualties and environmental pollution. Recent condition monitoring techniques use \changed{artificial intelligence} in an effort to avoid time-consuming manual analysis and handcrafted feature extraction. Many of these only analyze a single machine and require a large historical data set. In practice, this can be difficult and expensive to collect. However, some industrial condition monitoring applications involve a fleet of similar operating machines. In most of these applications, it is safe to assume healthy conditions for the majority of machines. Deviating machine behavior is then an indicator for a machine fault. This work proposes an unsupervised, generic, anomaly detection framework for fleet-based condition monitoring. It uses generic building blocks and offers three key advantages. First, a historical data set is not required due to online fleet-based comparisons. Second, it allows incorporating domain expertise by user-defined comparison measures. Finally, contrary to most black-box \changed{artificial intelligence} techniques, easy interpretability allows a domain expert to validate the predictions made by the framework. Two use-cases on an electrical machine fleet demonstrate the applicability of the framework to detect a voltage unbalance by means of electrical and vibration signatures.edition: Special Issue on "Machinery Diagnostics and Prognostics Using Artificial Intelligent Techniques"status: Published onlin

Fleet-based health monitoring for end-of-production-line and operational testing

status: accepte