Mining Aircraft Telemetry Data With Evolutionary Algorithms

The Ganged Phased Array Radar - Risk Mitigation System (GPAR-RMS) was a mobile ground-based sense-and-avoid system for Unmanned Aircraft System (UAS) operations developed by the University of North Dakota. GPAR-RMS detected proximate aircraft with various sensor systems, including a 2D radar and an Automatic Dependent Surveillance - Broadcast (ADS-B) receiver. Information about those aircraft was then displayed to UAS operators via visualization software developed by the University of North Dakota. The Risk Mitigation (RM) subsystem for GPAR-RMS was designed to estimate the current risk of midair collision, between the Unmanned Aircraft (UA) and a General Aviation (GA) aircraft flying under Visual Flight Rules (VFR) in the surrounding airspace, for UAS operations in Class E airspace (i.e. below 18,000 feet MSL). However, accurate probabilistic models for the behavior of pilots of GA aircraft flying under VFR in Class E airspace were needed before the RM subsystem could be implemented. In this dissertation the author presents the results of data mining an aircraft telemetry data set from a consecutive nine month period in 2011. This aircraft telemetry data set consisted of Flight Data Monitoring (FDM) data obtained from Garmin G1000 devices onboard every Cessna 172 in the University of North Dakota\u27s training fleet. Data from aircraft which were potentially within the controlled airspace surrounding controlled airports were excluded. Also, GA aircraft in the FDM data flying in Class E airspace were assumed to be flying under VFR, which is usually a valid assumption. Complex subpaths were discovered from the aircraft telemetry data set using a novel application of an ant colony algorithm. Then, probabilistic models were data mined from those subpaths using extensions of the Genetic K-Means (GKA) and Expectation- Maximization (EM) algorithms. The results obtained from the subpath discovery and data mining suggest a pilot flying a GA aircraft near to an uncontrolled airport will perform different maneuvers than a pilot flying a GA aircraft far from an uncontrolled airport, irrespective of the altitude of the GA aircraft. However, since only aircraft telemetry data from the University of North Dakota\u27s training fleet were data mined, these results are not likely to be applicable to GA aircraft operating in a non-training environment

Novel Computationally Intelligent Machine Learning Algorithms for Data Mining and Knowledge Discovery

This thesis addresses three major issues in data mining regarding feature subset selection in large dimensionality domains, plausible reconstruction of incomplete data in cross-sectional applications, and forecasting univariate time series. For the automated selection of an optimal subset of features in real time, we present an improved hybrid algorithm: SAGA. SAGA combines the ability to avoid being trapped in local minima of Simulated Annealing with the very high convergence rate of the crossover operator of Genetic Algorithms, the strong local search ability of greedy algorithms and the high computational efficiency of generalized regression neural networks (GRNN). For imputing missing values and forecasting univariate time series, we propose a homogeneous neural network ensemble. The proposed ensemble consists of a committee of Generalized Regression Neural Networks (GRNNs) trained on different subsets of features generated by SAGA and the predictions of base classifiers are combined by a fusion rule. This approach makes it possible to discover all important interrelations between the values of the target variable and the input features. The proposed ensemble scheme has two innovative features which make it stand out amongst ensemble learning algorithms: (1) the ensemble makeup is optimized automatically by SAGA; and (2) GRNN is used for both base classifiers and the top level combiner classifier. Because of GRNN, the proposed ensemble is a dynamic weighting scheme. This is in contrast to the existing ensemble approaches which belong to the simple voting and static weighting strategy. The basic idea of the dynamic weighting procedure is to give a higher reliability weight to those scenarios that are similar to the new ones. The simulation results demonstrate the validity of the proposed ensemble model

Fault analysis using state-of-the-art classifiers

Fault Analysis is the detection and diagnosis of malfunction in machine operation or process control. Early fault analysis techniques were reserved for high critical plants such as nuclear or chemical industries where abnormal event prevention is given utmost importance. The techniques developed were a result of decades of technical research and models based on extensive characterization of equipment behavior. This requires in-depth knowledge of the system and expert analysis to apply these methods for the application at hand. Since machine learning algorithms depend on past process data for creating a system model, a generic autonomous diagnostic system can be developed which can be used for application in common industrial setups. In this thesis, we look into some of the techniques used for fault detection and diagnosis multi-class and one-class classifiers. First we study Feature Selection techniques and the classifier performance is analyzed against the number of selected features. The aim of feature selection is to reduce the impact of irrelevant variables and to reduce computation burden on the learning algorithm. We introduce the feature selection algorithms as a literature survey. Only few algorithms are implemented to obtain the results. Fault data from a Radio Frequency (RF) generator is used to perform fault detection and diagnosis. Comparison between continuous and discrete fault data is conducted for the Support Vector Machines (SVM) and Radial Basis Function Network (RBF) classifiers. In the second part we look into one-class classification techniques and their application to fault detection. One-class techniques were primarily developed to identify one class of objects from all other possible objects. Since all fault occurrences in a system cannot be simulated or recorded, one-class techniques help in identifying abnormal events. We introduce four one-class classifiers and analyze them using Receiver-Operating Characteristic (ROC) curve. We also develop a feature extraction method for the RF generator data which is used to obtain results for one-class classifiers and Radial Basis Function Network two class classification. To apply these techniques for real-time verification, the RIT Fault Prediction software is built. LabView environment is used to build a basic data management and fault detection using Radial Basis Function Network. This software is stand alone and acts as foundation for future implementations

Efficient Learning Machines

Computer scienc

Intelligent energy management system - techniques and methods

ABSTRACT Our environment is an asset to be managed carefully and is not an expendable resource to be taken for granted. The main original contribution of this thesis is in formulating intelligent techniques and simulating case studies to demonstrate the significance of the present approach for achieving a low carbon economy. Energy boosts crop production, drives industry and increases employment. Wise energy use is the first step to ensuring sustainable energy for present and future generations. Energy services are essential for meeting internationally agreed development goals. Energy management system lies at the heart of all infrastructures from communications, economy, and society’s transportation to the society. This has made the system more complex and more interdependent. The increasing number of disturbances occurring in the system has raised the priority of energy management system infrastructure which has been improved with the aid of technology and investment; suitable methods have been presented to optimize the system in this thesis. Since the current system is facing various problems from increasing disturbances, the system is operating on the limit, aging equipments, load change etc, therefore an improvement is essential to minimize these problems. To enhance the current system and resolve the issues that it is facing, smart grid has been proposed as a solution to resolve power problems and to prevent future failures. This thesis argues that smart grid consists of computational intelligence and smart meters to improve the reliability, stability and security of power. In comparison with the current system, it is more intelligent, reliable, stable and secure, and will reduce the number of blackouts and other failures that occur on the power grid system. Also, the thesis has reported that smart metering is technically feasible to improve energy efficiency. In the thesis, a new technique using wavelet transforms, floating point genetic algorithm and artificial neural network based hybrid model for gaining accurate prediction of short-term load forecast has been developed. Adopting the new model is more accuracy than radial basis function network. Actual data has been used to test the proposed new method and it has been demonstrated that this integrated intelligent technique is very effective for the load forecast. Choosing the appropriate algorithm is important to implement the optimization during the daily task in the power system. The potential for application of swarm intelligence to Optimal Reactive Power Dispatch (ORPD) has been shown in this thesis. After making the comparison of the results derived from swarm intelligence, improved genetic algorithm and a conventional gradient-based optimization method, it was concluded that swam intelligence is better in terms of performance and precision in solving optimal reactive power dispatch problems

Twitter Mining for Syndromic Surveillance

Enormous amounts of personalised data is generated daily from social media platforms today. Twitter in particular, generates vast textual streams in real-time, accompanied with personal information. This big social media data offers a potential avenue for inferring public and social patterns. This PhD thesis investigates the use of Twitter data to deliver signals for syndromic surveillance in order to assess its ability to augment existing syndromic surveillance efforts and give a better understanding of symptomatic people who do not seek healthcare advice directly. We focus on a specific syndrome - asthma/difficulty breathing. We seek to develop means of extracting reliable signals from the Twitter signal, to be used for syndromic surveillance purposes. We begin by outlining our data collection and preprocessing methods. However, we observe that even with keyword-based data collection, many of the collected tweets are not relevant because they represent chatter, or talk of awareness instead of an individual suffering a particular condition. In light of this, we set out to identify relevant tweets to collect a strong and reliable signal. We first develop novel features based on the emoji content of Tweets and apply semi-supervised learning techniques to filter Tweets. Next, we investigate the effectiveness of deep learning at this task. We pro-pose a novel classification algorithm based on neural language models, and compare it to existing successful and popular deep learning algorithms. Following this, we go on to propose an attentive bi-directional Recurrent Neural Network architecture for filtering Tweets which also offers additional syndromic surveillance utility by identifying keywords among syndromic Tweets. In doing so, we are not only able to detect alarms, but also have some clues into what the alarm involves. Lastly, we look towards optimizing the Twitter syndromic surveillance pipeline by selecting the best possible keywords to be supplied to the Twitter API. We developed algorithms to intelligently and automatically select keywords such that the quality, in terms of relevance, and quantity of Tweets collected is maximised

Intelligent energy management system : techniques and methods

ABSTRACT Our environment is an asset to be managed carefully and is not an expendable resource to be taken for granted. The main original contribution of this thesis is in formulating intelligent techniques and simulating case studies to demonstrate the significance of the present approach for achieving a low carbon economy. Energy boosts crop production, drives industry and increases employment. Wise energy use is the first step to ensuring sustainable energy for present and future generations. Energy services are essential for meeting internationally agreed development goals. Energy management system lies at the heart of all infrastructures from communications, economy, and society’s transportation to the society. This has made the system more complex and more interdependent. The increasing number of disturbances occurring in the system has raised the priority of energy management system infrastructure which has been improved with the aid of technology and investment; suitable methods have been presented to optimize the system in this thesis. Since the current system is facing various problems from increasing disturbances, the system is operating on the limit, aging equipments, load change etc, therefore an improvement is essential to minimize these problems. To enhance the current system and resolve the issues that it is facing, smart grid has been proposed as a solution to resolve power problems and to prevent future failures. This thesis argues that smart grid consists of computational intelligence and smart meters to improve the reliability, stability and security of power. In comparison with the current system, it is more intelligent, reliable, stable and secure, and will reduce the number of blackouts and other failures that occur on the power grid system. Also, the thesis has reported that smart metering is technically feasible to improve energy efficiency. In the thesis, a new technique using wavelet transforms, floating point genetic algorithm and artificial neural network based hybrid model for gaining accurate prediction of short-term load forecast has been developed. Adopting the new model is more accuracy than radial basis function network. Actual data has been used to test the proposed new method and it has been demonstrated that this integrated intelligent technique is very effective for the load forecast. Choosing the appropriate algorithm is important to implement the optimization during the daily task in the power system. The potential for application of swarm intelligence to Optimal Reactive Power Dispatch (ORPD) has been shown in this thesis. After making the comparison of the results derived from swarm intelligence, improved genetic algorithm and a conventional gradient-based optimization method, it was concluded that swam intelligence is better in terms of performance and precision in solving optimal reactive power dispatch problems.EThOS - Electronic Theses Online ServiceGBUnited Kingdo

Optimal Model-parameter Determination for Feedforward Artificial Neural Networks

Neural Networks are an immensely versatile tool for state-of-the-art prediction problems. However, they require a training process that involves numerous hyper-parameters. This creates a training process that demands expert knowledge to configure and is often described as a trial-and-error process. The result is a training process that needs to be executed multiple times and this is highly time expensive. Currently, one solution to this problem is to perform a Grid-Search algorithm. This is where a set of possible values (essentially guesses) is declared for each hyper-parameter. Then each combination of hyper-parameters is used to configure the training session. Once the training of each model (hyper-parameter combination) is completed, the best performing model is retained, and the rest are discarded. The problem with this is that it can be wasteful as it explores hyper-parameter combinations that predictably produce poor models. It is also very time consuming and scales poorly with the size of the model. A number of methods are proposed in this {thesis} to efficiently derive hyper-parameters and model parameters and the empirical results are presented. These methods are split into two categories, Weight-Direct Determination (WDD), and Simple Effective Evolutionary Method. The former category exhibits success in certain cases whereas the latter exhibits a broad success across Classification and Regression; amongst a large number of samples and features and small number of samples and features. The thesis concludes that the WDD is only effective on small datasets (both in terms of the number of samples and number of input features). This is due to its dependence on Delaunay Triangulation which exhibits a quadratic time complexity with-respect-to the number of input samples. It is deemed that the WDD methods developed in this research are not optimal for achieving general-purpose application of Multi-Layer Perceptrons. However, the Complete Simple Effective Evolutionary Method (CSEEM) from the SEEM Chapter shows great promise as it is able to perform effectively on the `Knowledge Extraction based on Evolutionary Learning' (KEEL) Datasets for both Regression and Classification. This method can achieve this effectiveness whilst only requiring a single hyper-parameter (the number of children in a population) that is fairly invariant across datasets. In this {thesis}, CSEEM is applied to real-world regression and classification problems. It is also compared to RMSProp (gradient-dependent iterative method) to compare its performance with an existing gradient-dependent method. In both categories, CSEEM consistently performs with a lower normalized square loss and higher classification accuracy, respectively, versus the number hidden nodes when compared to RMSProp