Hybrid Metaheuristic Methods for Ensemble Classification in Non-stationary Data Streams

The extensive growth of digital technologies has led to new challenges in terms of processing and distilling insights from data that generated continuously in real-time. To address this challenge, several data stream mining techniques, where each instance of data is typically processed once on its arrival (i.e. online), have been proposed. However, such techniques of-ten perform poorly over non-stationary data streams, where the distribution of data evolves over time in unforeseen ways. To ensure the predictive ability of a computational model working with evolving data, appropriate data-stream mining techniques capable of adapting to different types of concept drifts are required. So far, ensemble-based learning methods are among the most popular techniques employed for performing data stream classification tasks in the presence of concept drifts. In ensemble learning, multiple learners forming an ensemble are trained to obtain a better predictive performance compared to that of a single learner. This thesis aims to propose and investigate novel hybrid metaheuristic methods for per-forming classification tasks in non-stationary environments. In particular, the thesis offers the following three main contributions. First, it presents the Evolutionary Adaptation to Concept Drifts (EACD) method that uses two evolutionary algorithms, namely, Replicator Dynamics (RD) and Genetic algorithm (GA). According to this method, an ensemble of different classification types is created based on various feature sets (called subspaces) randomly drawn from the target data stream. These subspaces are allowed to grow or shrink based on their performance using RD, while their combinations are optimised using GA. As the second contribution, this thesis proposes the REplicator Dynamics & GENEtic (RED-GENE)algorithm. RED-GENE builds upon the EACD method and employs the same approach to creating different classification types and GA optimisation technique. At the same time, RED-GENE improves the EACD method by proposing three different modified versions of RD to accelerate the concept drift adaptation process. The third contribution of the thesis is the REplicator Dynamics & Particle Swarm Optimisation (RED-PSO) algorithm that is based on a three-layer architecture to produce classification types of different sizes. The selected feature combinations in all classification types are optimised using a non-canonical version of the Particle Swarm Optimisation (PSO) technique for each layer individually. An extensive set of experiments using both synthetic and real-world data streams proves the effectiveness of the three proposed methods along with their statistical significance to the state-of-the-art algorithms. The proposed methods in this dissertation are consequently compared with each other that proves each of the proposed methods has its strengths to-wards concept drift adaptation in non-stationary data stream classification. This has led us to formulate a list of suggestions on when to use each of the proposed methods with regards to different applications and environments

Memetic algorithms for training feedforward neural networks: an approach based on gravitational search algorithm

The backpropagation (BP) algorithm is a gradient-based algorithm used for training a feedforward neural network (FNN). Despite the fact that BP is still used today when FNNs are trained, it has some disadvantages, including the following: (i) it fails when non-differentiable functions are addressed, (ii) it can become trapped in local minima, and (iii) it has slow convergence. In order to solve some of these problems, metaheuristic algorithms have been used to train FNN. Although they have good exploration skills, they are not as good as gradient-based algorithms at exploitation tasks. The main contribution of this article lies in its application of novel memetic approaches based on the Gravitational Search Algorithm (GSA) and Chaotic Gravitational Search Algorithm (CGSA) algorithms, called respectively Memetic Gravitational Search Algorithm (MGSA) and Memetic Chaotic Gravitational Search Algorithm (MCGSA), to train FNNs in three classical benchmark problems: the XOR problem, the approximation of a continuous function, and classification tasks. The results show that both approaches constitute suitable alternatives for training FNNs, even improving on the performance of other state-of-the-art metaheuristic algorithms such as ParticleSwarm Optimization (PSO), the Genetic Algorithm (GA), the Adaptive Differential Evolution algorithm with Repaired crossover rate (Rcr-JADE), and the Covariance matrix learning and Bimodal distribution parameter setting Differential Evolution (COBIDE) algorithm. Swarm optimization, the genetic algorithm, the adaptive differential evolution algorithm with repaired crossover rate, and the covariance matrix learning and bimodal distribution parameter setting differential evolution algorithm.El algoritmo de retropropagación (BP) es un algoritmo basado en gradientes que se utiliza para entrenar una red neuronal feedforward (FNN). A pesar de que BP todavía se usa hoy en día cuando se entrenan las FNN, tiene algunas desventajas, incluidas las siguientes: (i) falla cuando se abordan funciones no diferenciables, (ii) puede quedar atrapada en mínimos locales y (iii) ) tiene convergencia lenta. Para resolver algunos de estos problemas, se han utilizado algoritmos metaheurísticos para entrenar FNN. Aunque tienen buenas habilidades de exploración, no son tan buenos como los algoritmos basados ​​en gradientes en las tareas de explotación. La principal contribución de este artículo radica en la aplicación de nuevos enfoques meméticos basados ​​en los algoritmos Gravitational Search Algorithm (GSA) y Chaotic Gravitational Search Algorithm (CGSA), llamados respectivamente Algoritmo de búsqueda gravitacional memético (MGSA) y Algoritmo de búsqueda gravitacional caótico memético (MCGSA), para entrenar FNN en tres problemas de referencia clásicos: el problema XOR, la aproximación de una función continua y tareas de clasificación. Los resultados muestran que ambos enfoques constituyen alternativas adecuadas para el entrenamiento de FNN, incluso mejorando el rendimiento de otros algoritmos metaheurísticos de última generación como ParticleSwarm Optimization (PSO), el Algoritmo Genético (GA), el algoritmo de Evolución Diferencial Adaptativa con Tasa de cruce reparada (Rcr-JADE) y el algoritmo de evolución diferencial (COBIDE) de configuración de parámetros de distribución bimodal y aprendizaje de matriz de covarianza. Optimización de enjambre, el algoritmo genético, el algoritmo de evolución diferencial adaptativo con tasa de cruce reparada

Mining Aircraft Telemetry Data With Evolutionary Algorithms

The Ganged Phased Array Radar - Risk Mitigation System (GPAR-RMS) was a mobile ground-based sense-and-avoid system for Unmanned Aircraft System (UAS) operations developed by the University of North Dakota. GPAR-RMS detected proximate aircraft with various sensor systems, including a 2D radar and an Automatic Dependent Surveillance - Broadcast (ADS-B) receiver. Information about those aircraft was then displayed to UAS operators via visualization software developed by the University of North Dakota. The Risk Mitigation (RM) subsystem for GPAR-RMS was designed to estimate the current risk of midair collision, between the Unmanned Aircraft (UA) and a General Aviation (GA) aircraft flying under Visual Flight Rules (VFR) in the surrounding airspace, for UAS operations in Class E airspace (i.e. below 18,000 feet MSL). However, accurate probabilistic models for the behavior of pilots of GA aircraft flying under VFR in Class E airspace were needed before the RM subsystem could be implemented. In this dissertation the author presents the results of data mining an aircraft telemetry data set from a consecutive nine month period in 2011. This aircraft telemetry data set consisted of Flight Data Monitoring (FDM) data obtained from Garmin G1000 devices onboard every Cessna 172 in the University of North Dakota\u27s training fleet. Data from aircraft which were potentially within the controlled airspace surrounding controlled airports were excluded. Also, GA aircraft in the FDM data flying in Class E airspace were assumed to be flying under VFR, which is usually a valid assumption. Complex subpaths were discovered from the aircraft telemetry data set using a novel application of an ant colony algorithm. Then, probabilistic models were data mined from those subpaths using extensions of the Genetic K-Means (GKA) and Expectation- Maximization (EM) algorithms. The results obtained from the subpath discovery and data mining suggest a pilot flying a GA aircraft near to an uncontrolled airport will perform different maneuvers than a pilot flying a GA aircraft far from an uncontrolled airport, irrespective of the altitude of the GA aircraft. However, since only aircraft telemetry data from the University of North Dakota\u27s training fleet were data mined, these results are not likely to be applicable to GA aircraft operating in a non-training environment

A survey on metaheuristics for stochastic combinatorial optimization

Metaheuristics are general algorithmic frameworks, often nature-inspired, designed to solve complex optimization problems, and they are a growing research area since a few decades. In recent years, metaheuristics are emerging as successful alternatives to more classical approaches also for solving optimization problems that include in their mathematical formulation uncertain, stochastic, and dynamic information. In this paper metaheuristics such as Ant Colony Optimization, Evolutionary Computation, Simulated Annealing, Tabu Search and others are introduced, and their applications to the class of Stochastic Combinatorial Optimization Problems (SCOPs) is thoroughly reviewed. Issues common to all metaheuristics, open problems, and possible directions of research are proposed and discussed. In this survey, the reader familiar to metaheuristics finds also pointers to classical algorithmic approaches to optimization under uncertainty, and useful informations to start working on this problem domain, while the reader new to metaheuristics should find a good tutorial in those metaheuristics that are currently being applied to optimization under uncertainty, and motivations for interest in this fiel

Prediction of sulfur content in diesel fuel using fluorescence spectroscopy and a hybrid ant colony : Tabu Search algorithm with polynomial bases expansion

It is widely accepted that feature selection is an essential step in predictive modeling. There are several approaches to feature selection, from filter techniques to meta-heuristics wrapper methods. In this paper, we propose a compilation of tools to optimize the fitting of black-box linear models. The proposed AnTSbe algorithm combines Ant Colony Optimization and Tabu Search memory list for the selection of features and uses l1 and l2 regularization norms to fit the linear models. In addition, a polynomial combination of input features was introduced to further explore the information contained in the original data. As a case study, excitation-emission matrix fluorescence data were used as the primary measurements to predict total sulfur concentration in diesel fuel samples. The sample dataset was divided into S10 (less than 10 ppm of total sulfur), and S100 (mean sulfur content of 100 ppm) groups and local linear models were fit with AnTSbe. For the Diesel S100 local models, using only 5 out of the original 1467 fluorescence pairs, combined with bases expansion, we were able to satisfactorily predict total sulfur content in samples with MAPE of less than 4% and RMSE of 4.68 ppm, for the test subset. For the Diesel S10 local models, the use of 4 Ex/Em pairs was sufficient to predict sulfur content with MAPE 0.24%, and RMSE of 0.015 ppm, for the test subset. Our experimental results demonstrate that the proposed methodology was able to satisfactorily optimize the fitting of linear models to predict sulfur content in diesel fuel samples without need of chemical of physical pre-treatment, and was superior to classic PLS regression methods and also to our previous results with ant colony optimization studies in the same dataset. The proposed AnTSbe can be directly applied to data from other sources without need for adaptations

Adaptive sliding windows for improved estimation of data center resource utilization

Accurate prediction of data center resource utilization is required for capacity planning, job scheduling, energy saving, workload placement, and load balancing to utilize the resources efficiently. However, accurately predicting those resources is challenging due to dynamic workloads, heterogeneous infrastructures, and multi-tenant co-hosted applications. Existing prediction methods use fixed size observation windows which cannot produce accurate results because of not being adaptively adjusted to capture local trends in the most recent data. Therefore, those methods train on large fixed sliding windows using an irrelevant large number of observations yielding to inaccurate estimations or fall for inaccuracy due to degradation of estimations with short windows on quick changing trends. In this paper we propose a deep learning-based adaptive window size selection method, dynamically limiting the sliding window size to capture the trend for the latest resource utilization, then build an estimation model for each trend period. We evaluate the proposed method against multiple baseline and state-of-the-art methods, using real data-center workload data sets. The experimental evaluation shows that the proposed solution outperforms those state-of-the-art approaches and yields 16 to 54% improved prediction accuracy compared to the baseline methods.This work is partially supported by the European ResearchCouncil (ERC) under the EU Horizon 2020 programme(GA 639595), the Spanish Ministry of Economy, Industry andCompetitiveness (TIN2015-65316-P and IJCI2016-27485), theGeneralitat de Catalunya, Spain (2014-SGR-1051) and Universityof the Punjab, Pakistan. The statements made herein are solelythe responsibility of the authors.Peer ReviewedPostprint (published version

Development of Hybrid PS-FW GMPPT Algorithm for improving PV System Performance Under Partial Shading Conditions

A global maximum power point tracking (MPPT) algorithm hybrid based on Particle Swarm Fireworks (PS-FW) algorithm is proposed which is formed with Particle Swarm Optimization and Fireworks Algorithm. The algorithm tracks the global maximum power point (MPP) when conventional MPPT methods fail due to occurrence of partial shading conditions. With the applied strategies and operators, PS-FW algorithm obtains superior performances verified under simulation and experimental setup with multiple cases of shading patterns