Validating a neural network-based online adaptive system

Neural networks are popular models used for online adaptation to accommodate system faults and recuperate against environmental changes in real-time automation and control applications. However, the adaptivity limits the applicability of conventional verification and validation (V&V) techniques to such systems. We investigated the V&V of neural network-based online adaptive systems and developed a novel validation approach consisting of two important methods. (1) An independent novelty detector at the system input layer detects failure conditions and tracks abnormal events/data that may cause unstable learning behavior. (2) At the system output layer, we perform a validity check on the network predictions to validate its accommodation performance.;Our research focuses on the Intelligent Flight Control System (IFCS) for NASA F-15 aircraft as an example of online adaptive control application. We utilized Support Vector Data Description (SVDD), a one-class classifier to examine the data entering the adaptive component and detect potential failures. We developed a decompose and combine strategy to drastically reduce its computational cost, from O(n 3) down to O( n32 log n) such that the novelty detector becomes feasible in real-time.;We define a confidence measure, the validity index, to validate the predictions of the Dynamic Cell Structure (DCS) network in IFCS. The statistical information is collected during adaptation. The validity index is computed to reflect the trustworthiness associated with each neural network output. The computation of validity index in DCS is straightforward and efficient.;Through experimentation with IFCS, we demonstrate that: (1) the SVDD tool detects system failures accurately and provides validation inferences in a real-time manner; (2) the validity index effectively indicates poor fitting within regions characterized by sparse data and/or inadequate learning. The developed methods can be integrated with available online monitoring tools and further generalized to complete a promising validation framework for neural network based online adaptive systems

A survey of outlier detection methodologies

Outlier detection has been used for centuries to detect and, where appropriate, remove anomalous observations from data. Outliers arise due to mechanical faults, changes in system behaviour, fraudulent behaviour, human error, instrument error or simply through natural deviations in populations. Their detection can identify system faults and fraud before they escalate with potentially catastrophic consequences. It can identify errors and remove their contaminating effect on the data set and as such to purify the data for processing. The original outlier detection methods were arbitrary but now, principled and systematic techniques are used, drawn from the full gamut of Computer Science and Statistics. In this paper, we introduce a survey of contemporary techniques for outlier detection. We identify their respective motivations and distinguish their advantages and disadvantages in a comparative review

Aircraft Abnormal Conditions Detection, Identification, and Evaluation Using Innate and Adaptive Immune Systems Interaction

Abnormal flight conditions play a major role in aircraft accidents frequently causing loss of control. To ensure aircraft operation safety in all situations, intelligent system monitoring and adaptation must rely on accurately detecting the presence of abnormal conditions as soon as they take place, identifying their root cause(s), estimating their nature and severity, and predicting their impact on the flight envelope.;Due to the complexity and multidimensionality of the aircraft system under abnormal conditions, these requirements are extremely difficult to satisfy using existing analytical and/or statistical approaches. Moreover, current methodologies have addressed only isolated classes of abnormal conditions and a reduced number of aircraft dynamic parameters within a limited region of the flight envelope.;This research effort aims at developing an integrated and comprehensive framework for the aircraft abnormal conditions detection, identification, and evaluation based on the artificial immune systems paradigm, which has the capability to address the complexity and multidimensionality issues related to aircraft systems.;Within the proposed framework, a novel algorithm was developed for the abnormal conditions detection problem and extended to the abnormal conditions identification and evaluation. The algorithm and its extensions were inspired from the functionality of the biological dendritic cells (an important part of the innate immune system) and their interaction with the different components of the adaptive immune system. Immunity-based methodologies for re-assessing the flight envelope at post-failure and predicting the impact of the abnormal conditions on the performance and handling qualities are also proposed and investigated in this study.;The generality of the approach makes it applicable to any system. Data for artificial immune system development were collected from flight tests of a supersonic research aircraft within a motion-based flight simulator. The abnormal conditions considered in this work include locked actuators (stabilator, aileron, rudder, and throttle), structural damage of the wing, horizontal tail, and vertical tail, malfunctioning sensors, and reduced engine effectiveness. The results of applying the proposed approach to this wide range of abnormal conditions show its high capability in detecting the abnormal conditions with zero false alarms and very high detection rates, correctly identifying the failed subsystem and evaluating the type and severity of the failure. The results also reveal that the post-failure flight envelope can be reasonably predicted within this framework

Aircraft Fault Detection and Classification Using Multi-Level Immune Learning Detection

This work is an extension of a recently developed software tool called MILD (Multi-level Immune Learning Detection), which implements a negative selection algorithm for anomaly and fault detection that is inspired by the human immune system. The immunity-based approach can detect a broad spectrum of known and unforeseen faults. We extend MILD by applying a neural network classifier to identify the pattern of fault detectors that are activated during fault detection. Consequently, MILD now performs fault detection and identification of the system under investigation. This paper describes the application of MILD to detect and classify faults of a generic transport aircraft augmented with an intelligent flight controller. The intelligent control architecture is designed to accommodate faults without the need to explicitly identify them. Adding knowledge about the existence and type of a fault will improve the handling qualities of a degraded aircraft and impact tactical and strategic maneuvering decisions. In addition, providing fault information to the pilot is important for maintaining situational awareness so that he can avoid performing an action that might lead to unexpected behavior - e.g., an action that exceeds the remaining control authority of the damaged aircraft. We discuss the detection and classification results of simulated failures of the aircraft's control system and show that MILD is effective at determining the problem with low false alarm and misclassification rates

Neural Network-Based Sensor Validation for Turboshaft Engines

Sensor failure detection, isolation, and accommodation using a neural network approach is described. An auto-associative neural network is configured to perform dimensionality reduction on the sensor measurement vector and provide estimated sensor values. The sensor validation scheme is applied in a simulation of the T700 turboshaft engine in closed loop operation. Performance is evaluated based on the ability to detect faults correctly and maintain stable and responsive engine operation. The set of sensor outputs used for engine control forms the network input vector. Analytical redundancy is verified by training networks of successively smaller bottleneck layer sizes. Training data generation and strategy are discussed. The engine maintained stable behavior in the presence of sensor hard failures. With proper selection of fault determination thresholds, stability was maintained in the presence of sensor soft failures

Monitoring and Control Framework for Advanced Power Plant Systems Using Artificial Intelligence Techniques

This dissertation presents the design, development, and simulation testing of a monitoring and control framework for dynamic systems using artificial intelligence techniques. A comprehensive monitoring and control system capable of detecting, identifying, evaluating, and accommodating various subsystem failures and upset conditions is presented. The system is developed by synergistically merging concepts inspired from the biological immune system with evolutionary optimization algorithms and adaptive control techniques.;The proposed methodology provides the tools for addressing the complexity and multi-dimensionality of the modern power plants in a comprehensive and integrated manner that classical approaches cannot achieve. Current approaches typically address abnormal condition (AC) detection of isolated subsystems of low complexity, affected by specific AC involving few features with limited identification capability. They do not attempt AC evaluation and mostly rely on control system robustness for accommodation. Addressing the problem of power plant monitoring and control under AC at this level of completeness has not yet been attempted.;Within the proposed framework, a novel algorithm, namely the partition of the universe, was developed for building the artificial immune system self. As compared to the clustering approach, the proposed approach is less computationally intensive and facilitates the use of full-dimensional self for system AC detection, identification, and evaluation. The approach is implemented in conjunction with a modified and improved dendritic cell algorithm. It allows for identifying the failed subsystems without previous training and is extended to address the AC evaluation using a novel approach.;The adaptive control laws are designed to augment the performance and robustness of baseline control laws under normal and abnormal operating conditions. Artificial neural network-based and artificial immune system-based approaches are developed and investigated for an advanced power plant through numerical simulation.;This dissertation also presents the development of an interactive computational environment for the optimization of power plant control system using evolutionary techniques with immunity-inspired enhancements. Several algorithms mimicking mechanisms of the immune system of superior organisms, such as cloning, affinity-based selection, seeding, and vaccination are used. These algorithms are expected to enhance the computational effectiveness, improve convergence, and be more efficient in handling multiple local extrema, through an adequate balance between exploration and exploitation.;The monitoring and control framework formulated in this dissertation applies to a wide range of technical problems. The proposed methodology is demonstrated with promising results using a high validity DynsimRTM model of the acid gas removal unit that is part of the integrated gasification combined cycle power plant available at West Virginia University AVESTAR Center. The obtained results show that the proposed system is an efficient and valuable technique to be applied to a real world application. The implementation of this methodology can potentially have significant impacts on the operational safety of many complex systems

Using Neural Networks for Sensor Validation

This paper presents the results of applying two different types of neural networks in two different approaches to the sensor validation problem. The first approach uses a functional approximation neural network as part of a nonlinear observer in a model-based approach to analytical redundancy. The second approach uses an auto-associative neural network to perform nonlinear principal component analysis on a set of redundant sensors to provide an estimate for a single failed sensor. The approaches are demonstrated using a nonlinear simulation of a turbofan engine. The fault detection and sensor estimation results are presented and the training of the auto-associative neural network to provide sensor estimates is discussed