Using Decision Trees to Detect and Isolate Simulated Leaks in the J-2X Rocket Engine

The goal of this work was to use data-driven methods to automatically detect and isolate faults in the J-2X rocket engine. It was decided to use decision trees, since they tend to be easier to interpret than other data-driven methods. The decision tree algorithm automatically "learns" a decision tree by performing a search through the space of possible decision trees to find one that fits the training data. The particular decision tree algorithm used is known as C4.5. Simulated J-2X data from a high-fidelity simulator developed at Pratt & Whitney Rocketdyne and known as the Detailed Real-Time Model (DRTM) was used to "train" and test the decision tree. Fifty-six DRTM simulations were performed for this purpose, with different leak sizes, different leak locations, and different times of leak onset. To make the simulations as realistic as possible, they included simulated sensor noise, and included a gradual degradation in both fuel and oxidizer turbine efficiency. A decision tree was trained using 11 of these simulations, and tested using the remaining 45 simulations. In the training phase, the C4.5 algorithm was provided with labeled examples of data from nominal operation and data including leaks in each leak location. From the data, it "learned" a decision tree that can classify unseen data as having no leak or having a leak in one of the five leak locations. In the test phase, the decision tree produced very low false alarm rates and low missed detection rates on the unseen data. It had very good fault isolation rates for three of the five simulated leak locations, but it tended to confuse the remaining two locations, perhaps because a large leak at one of these two locations can look very similar to a small leak at the other location

Data Mining at NASA: From Theory to Applications

This slide presentation demonstrates the data mining/machine learning capabilities of NASA Ames and Intelligent Data Understanding (IDU) group. This will encompass the work done recently in the group by various group members. The IDU group develops novel algorithms to detect, classify, and predict events in large data streams for scientific and engineering systems. This presentation for Knowledge Discovery and Data Mining 2009 is to demonstrate the data mining/machine learning capabilities of NASA Ames and IDU group. This will encompass the work done re cently in the group by various group members

Anomaly Detection for Next-Generation Space Launch Ground Operations

NASA is developing new capabilities that will enable future human exploration missions while reducing mission risk and cost. The Fault Detection, Isolation, and Recovery (FDIR) project aims to demonstrate the utility of integrated vehicle health management (IVHM) tools in the domain of ground support equipment (GSE) to be used for the next generation launch vehicles. In addition to demonstrating the utility of IVHM tools for GSE, FDIR aims to mature promising tools for use on future missions and document the level of effort - and hence cost - required to implement an application with each selected tool. One of the FDIR capabilities is anomaly detection, i.e., detecting off-nominal behavior. The tool we selected for this task uses a data-driven approach. Unlike rule-based and model-based systems that require manual extraction of system knowledge, data-driven systems take a radically different approach to reasoning. At the basic level, they start with data that represent nominal functioning of the system and automatically learn expected system behavior. The behavior is encoded in a knowledge base that represents "in-family" system operations. During real-time system monitoring or during post-flight analysis, incoming data is compared to that nominal system operating behavior knowledge base; a distance representing deviation from nominal is computed, providing a measure of how far "out of family" current behavior is. We describe the selected tool for FDIR anomaly detection - Inductive Monitoring System (IMS), how it fits into the FDIR architecture, the operations concept for the GSE anomaly monitoring, and some preliminary results of applying IMS to a Space Shuttle GSE anomaly

Propulsion Health Management System Development for Affordable and Reliable Operation of Space Exploration Systems

The constraints of future Exploration Missions will require unique integrated system health management capabilities throughout the mission. An ambitious launch schedule, human-rating requirements, long quiescent periods, limited human access for repair or replacement, and long communication delays, all require an integrated approach to health management that can span distinct, yet interdependent vehicle subsystems, anticipate failure states, provide autonomous remediation and support the Exploration Mission from beginning to end. Propulsion is a critical part of any space exploration mission, and monitoring the health of the propulsion system is an integral part of assuring mission safety and success. Health management is a somewhat ubiquitous technology that encompasses a large spectrum of physical components and logical processes. For this reason, it is essential to develop a systematic plan for propulsion health management system development. This paper provides a high-level perspective of propulsion health management systems, and describes a logical approach for the future planning and early development that are crucial to planned space exploration programs. It also presents an overall approach, or roadmap, for propulsion health management system development and a discussion of the associated roadblocks and challenges

A Framework to Develop Anomaly Detection/Fault Isolation Architecture Using System Engineering Principles

For critical systems, timely recognition of an anomalous condition immediately starts the evaluation process. For complex systems, isolating the fault to a component or subsystem results in corrective action sooner so that undesired consequences may be minimized. There are many unique anomaly detection and fault isolation capabilities available with innovative techniques to quickly discover an issue and identify the underlying problems. This research develops a framework to aid in the selection of appropriate anomaly detection and fault isolation technology to augment a given system. To optimize this process, the framework employs a model based systems engineering approach. Specifically, a SysML model is generated that enables a system-level evaluation of alternative detection and isolation techniques, and subsequently identifies the preferable application(s) from these technologies A case study is conducted on a cryogenic liquid hydrogen system that was used to fuel the Space Shuttles at the Kennedy Space Center, Florida (and will be used to fuel the next generation Space Launch System rocket). This system is operated remotely and supports time-critical and highly hazardous operations making it a good candidate to augment with this technology. As the process depicted by the framework down-selects to potential applications for consideration, these too are tested in their ability to achieve required goals

Space Transportation Propulsion Technology Symposium. Volume 3: Panel Session Summaries and Presentations

The Space Transportation Propulsion Technology Symposium was held at the Pennsylvania State University on June 25 to 29, 1990. Emphasis was placed on propulsion requirements and initiatives to support current, next generation, and future space transportation systems, with the primary objectives of discerning whether proposed designs truly meet future transportation needs and identifying possible technology gaps, overlaps and other programmatic deficiencies. Key space transportation propulsion issues are addressed through four panels with government, industry, and academia membership. The panel focused on systems engineering and integration; development, manufacturing, and certification; operational efficiency; program development; and cultural issues

An Anomaly Detection Method for Satellites Using Monte Carlo Dropout

Recently, there has been a significant amount of interest in satellite telemetry anomaly detection (AD) using neural networks (NN). For AD purposes, the current approaches focus on either forecasting or reconstruction of the time series, and they cannot measure the level of reliability or the probability of correct detection. Although the Bayesian neural network (BNN)-based approaches are well known for time series uncertainty estimation, they are computationally intractable. In this paper, we present a tractable approximation for BNN based on the Monte Carlo (MC) dropout method for capturing the uncertainty in the satellite telemetry time series, without sacrificing accuracy. For time series forecasting, we employ an NN, which consists of several Long Short-Term Memory (LSTM) layers followed by various dense layers. We employ the MC dropout inside each LSTM layer and before the dense layers for uncertainty estimation. With the proposed uncertainty region and by utilizing a post-processing filter, we can effectively capture the anomaly points. Numerical results show that our proposed time series AD approach outperforms the existing methods from both prediction accuracy and AD perspectives

3D Histogram Based Anomaly Detection for Categorical Sensor Data in Internet of Things

The applications of Internet-of-things (IoT) deploy massive number of sensors to monitor the system and environment. Anomaly detection on streaming sensor data is an important task for IoT maintenance and operation. In real IoT applications, many sensors report categorical values rather than numerical readings. Unfortunately, most existing anomaly detection methods are designed only for numerical sensor data. They cannot be used to monitor the categorical sensor data. In this study, we design and develop a 3D Histogram based Categorical Anomaly Detection (HCAD) solution to monitor categorical sensor data in IoT. HCAD constructs the histogram model by three dimensions: categorical value, event duration, and frequency. The histogram models are used to profile normal working states of IoT devices. HCAD automatically determines the range of normal data and anomaly threshold. It only requires very limit parameter setting and can be applied to a wide variety of different IoT devices. We implement HCAD and integrate it into an online monitoring system. We test the proposed solution on real IoT datasets such as telemetry data from satellite sensors, air quality data from chemical sensors, and transportation data from traffic sensors. The results of extensive experiments show that HCAD achieves higher detecting accuracy and efficiency than state-of-the-art methods

Multivariate Real Time Series Data Using Six Unsupervised Machine Learning Algorithms

The development of artificial intelligence (AI) algorithms for classification purpose of undesirable events has gained notoriety in the industrial world. Nevertheless, for AI algorithm training is necessary to have labeled data to identify the normal and anomalous operating conditions of the system. However, labeled data is scarce or nonexistent, as it requires a herculean effort to the specialists of labeling them. Thus, this chapter provides a comparison performance of six unsupervised Machine Learning (ML) algorithms to pattern recognition in multivariate time series data. The algorithms can identify patterns to assist in semiautomatic way the data annotating process for, subsequentially, leverage the training of AI supervised models. To verify the performance of the unsupervised ML algorithms to detect interest/anomaly pattern in real time series data, six algorithms were applied in following two identical cases (i) meteorological data from a hurricane season and (ii) monitoring data from dynamic machinery for predictive maintenance purposes. The performance evaluation was investigated with seven threshold indicators: accuracy, precision, recall, specificity, F1-Score, AUC-ROC and AUC-PRC. The results suggest that algorithms with multivariate approach can be successfully applied in the detection of anomalies in multivariate time series data