Missing Value Imputation With Unsupervised Backpropagation

Many data mining and data analysis techniques operate on dense matrices or complete tables of data. Real-world data sets, however, often contain unknown values. Even many classification algorithms that are designed to operate with missing values still exhibit deteriorated accuracy. One approach to handling missing values is to fill in (impute) the missing values. In this paper, we present a technique for unsupervised learning called Unsupervised Backpropagation (UBP), which trains a multi-layer perceptron to fit to the manifold sampled by a set of observed point-vectors. We evaluate UBP with the task of imputing missing values in datasets, and show that UBP is able to predict missing values with significantly lower sum-squared error than other collaborative filtering and imputation techniques. We also demonstrate with 24 datasets and 9 supervised learning algorithms that classification accuracy is usually higher when randomly-withheld values are imputed using UBP, rather than with other methods

Modeling Financial Time Series with Artificial Neural Networks

Financial time series convey the decisions and actions of a population of human actors over time. Econometric and regressive models have been developed in the past decades for analyzing these time series. More recently, biologically inspired artificial neural network models have been shown to overcome some of the main challenges of traditional techniques by better exploiting the non-linear, non-stationary, and oscillatory nature of noisy, chaotic human interactions. This review paper explores the options, benefits, and weaknesses of the various forms of artificial neural networks as compared with regression techniques in the field of financial time series analysis.CELEST, a National Science Foundation Science of Learning Center (SBE-0354378); SyNAPSE program of the Defense Advanced Research Project Agency (HR001109-03-0001

XFSL: A tool for supervised learning of fuzzy systems

This paper presents Xfsl, a tool for the automatic tuning of fuzzy systems using supervised learning algorithms. The tool provides a wide set of learning algorithms, which can be used to tune complex systems. An important issue is that Xfsl is integrated into the fuzzy system development environment Xfuzzy 3.0, and hence, it can be easily employed within the design flow of a fuzzy system.Comisión Interministerial de Ciencia y Tecnología TIC98-0869Fondo Europeo de Desarrollo Regional 1FD97-0956-C3-0

Developmental Flight Test of a Powered Approach Stability Augmentation System on the U.S. Navy\u27s E- 2C Hawkeye 2000 Aircraft

The E-2C aircraft is a Navy carrier based high-wing, twin engine turboprop powered aircraft used for the Airborne Early Warning (AEW) mission. In the power approach configuration, the aircraft displays strong adverse yaw, weak directional stability, and excessive rudder control power. These antagonistic characteristics, when coupled together, result in an extremely high workload for the pilot during both carrier and field landings. Although the aircraft has a yaw axis stability augmentation system, it is currently only applicable to cruise conditions. Engaging the stability augmentation in the power approach configuration results in a 1 Hz directional oscillation due to the system’s high gain schedule. Additionally, another attribute of the existing system design results in extremely high rudder pedal forces while maintaining sideslip in crosswind conditions. Northrop Grumman developed Flight Control Computer (FCC) software patches designed to improve the handling qualities on landing approaches. These patches are designed to change the rudder control gain schedule to allow the use of stability augmentation in the power approach configuration and suppress the divergent Phugoid characteristic throughout the flight envelope. The system is a directional axis controller only and termed the Powered Approach Stability Augmentation System (PASAS). Initial flight tests on a developmental system provided the design parameters for the production system, which was eventually installed in the Navy’s newest E-2C variant, termed Hawkeye 2000. The ensuing flight test program consisted of land based test flights during the summer of 2001, and culminated in a ship trial consisting of multiple landings on the USS Truman in March of 2002

Adaptive dynamic programming with eligibility traces and complexity reduction of high-dimensional systems

This dissertation investigates the application of a variety of computational intelligence techniques, particularly clustering and adaptive dynamic programming (ADP) designs especially heuristic dynamic programming (HDP) and dual heuristic programming (DHP). Moreover, a one-step temporal-difference (TD(0)) and n-step TD (TD(λ)) with their gradients are utilized as learning algorithms to train and online-adapt the families of ADP. The dissertation is organized into seven papers. The first paper demonstrates the robustness of model order reduction (MOR) for simulating complex dynamical systems. Agglomerative hierarchical clustering based on performance evaluation is introduced for MOR. This method computes the reduced order denominator of the transfer function by clustering system poles in a hierarchical dendrogram. Several numerical examples of reducing techniques are taken from the literature to compare with our work. In the second paper, a HDP is combined with the Dyna algorithm for path planning. The third paper uses DHP with an eligibility trace parameter (λ) to track a reference trajectory under uncertainties for a nonholonomic mobile robot by using a first-order Sugeno fuzzy neural network structure for the critic and actor networks. In the fourth and fifth papers, a stability analysis for a model-free action-dependent HDP(λ) is demonstrated with batch- and online-implementation learning, respectively. The sixth work combines two different gradient prediction levels of critic networks. In this work, we provide a convergence proofs. The seventh paper develops a two-hybrid recurrent fuzzy neural network structures for both critic and actor networks. They use a novel n-step gradient temporal-difference (gradient of TD(λ)) of an advanced ADP algorithm called value-gradient learning (VGL(λ)), and convergence proofs are given. Furthermore, the seventh paper is the first to combine the single network adaptive critic with VGL(λ). --Abstract, page iv

Metaheuristic design of feedforward neural networks: a review of two decades of research

Over the past two decades, the feedforward neural network (FNN) optimization has been a key interest among the researchers and practitioners of multiple disciplines. The FNN optimization is often viewed from the various perspectives: the optimization of weights, network architecture, activation nodes, learning parameters, learning environment, etc. Researchers adopted such different viewpoints mainly to improve the FNN's generalization ability. The gradient-descent algorithm such as backpropagation has been widely applied to optimize the FNNs. Its success is evident from the FNN's application to numerous real-world problems. However, due to the limitations of the gradient-based optimization methods, the metaheuristic algorithms including the evolutionary algorithms, swarm intelligence, etc., are still being widely explored by the researchers aiming to obtain generalized FNN for a given problem. This article attempts to summarize a broad spectrum of FNN optimization methodologies including conventional and metaheuristic approaches. This article also tries to connect various research directions emerged out of the FNN optimization practices, such as evolving neural network (NN), cooperative coevolution NN, complex-valued NN, deep learning, extreme learning machine, quantum NN, etc. Additionally, it provides interesting research challenges for future research to cope-up with the present information processing era