Studies in particle swarm optimization technique for global optimization.

Ph. D. University of KwaZulu-Natal, Durban 2013.Abstract available in the digital copy.Articles found within the main body of the thesis in the print version is found at the end of the thesis in the digital version

Feature selection using enhanced particle swarm optimisation for classification models.

In this research, we propose two Particle Swarm Optimisation (PSO) variants to undertake feature selection tasks. The aim is to overcome two major shortcomings of the original PSO model, i.e., premature convergence and weak exploitation around the near optimal solutions. The first proposed PSO variant incorporates four key operations, including a modified PSO operation with rectified personal and global best signals, spiral search based local exploitation, Gaussian distribution-based swarm leader enhancement, and mirroring and mutation operations for worst solution improvement. The second proposed PSO model enhances the first one through four new strategies, i.e., an adaptive exemplar breeding mechanism incorporating multiple optimal signals, nonlinear function oriented search coefficients, exponential and scattering schemes for swarm leader, and worst solution enhancement, respectively. In comparison with a set of 15 classical and advanced search methods, the proposed models illustrate statistical superiority for discriminative feature selection for a total of 13 data sets

Development of registration methods for cardiovascular anatomy and function using advanced 3T MRI, 320-slice CT and PET imaging

Different medical imaging modalities provide complementary anatomical and functional information. One increasingly important use of such information is in the clinical management of cardiovascular disease. Multi-modality data is helping improve diagnosis accuracy, and individualize treatment. The Clinical Research Imaging Centre at the University of Edinburgh, has been involved in a number of cardiovascular clinical trials using longitudinal computed tomography (CT) and multi-parametric magnetic resonance (MR) imaging. The critical image processing technique that combines the information from all these different datasets is known as image registration, which is the topic of this thesis. Image registration, especially multi-modality and multi-parametric registration, remains a challenging field in medical image analysis. The new registration methods described in this work were all developed in response to genuine challenges in on-going clinical studies. These methods have been evaluated using data from these studies. In order to gain an insight into the building blocks of image registration methods, the thesis begins with a comprehensive literature review of state-of-the-art algorithms. This is followed by a description of the first registration method I developed to help track inflammation in aortic abdominal aneurysms. It registers multi-modality and multi-parametric images, with new contrast agents. The registration framework uses a semi-automatically generated region of interest around the aorta. The aorta is aligned based on a combination of the centres of the regions of interest and intensity matching. The method achieved sub-voxel accuracy. The second clinical study involved cardiac data. The first framework failed to register many of these datasets, because the cardiac data suffers from a common artefact of magnetic resonance images, namely intensity inhomogeneity. Thus I developed a new preprocessing technique that is able to correct the artefacts in the functional data using data from the anatomical scans. The registration framework, with this preprocessing step and new particle swarm optimizer, achieved significantly improved registration results on the cardiac data, and was validated quantitatively using neuro images from a clinical study of neonates. Although on average the new framework achieved accurate results, when processing data corrupted by severe artefacts and noise, premature convergence of the optimizer is still a common problem. To overcome this, I invented a new optimization method, that achieves more robust convergence by encoding prior knowledge of registration. The registration results from this new registration-oriented optimizer are more accurate than other general-purpose particle swarm optimization methods commonly applied to registration problems. In summary, this thesis describes a series of novel developments to an image registration framework, aimed to improve accuracy, robustness and speed. The resulting registration framework was applied to, and validated by, different types of images taken from several ongoing clinical trials. In the future, this framework could be extended to include more diverse transformation models, aided by new machine learning techniques. It may also be applied to the registration of other types and modalities of imaging data

Chaotic Sand Cat Swarm Optimization

In this study, a new hybrid metaheuristic algorithm named Chaotic Sand Cat Swarm Optimization (CSCSO) is proposed for constrained and complex optimization problems. This algorithm combines the features of the recently introduced SCSO with the concept of chaos. The basic aim of the proposed algorithm is to integrate the chaos feature of non-recurring locations into SCSO’s core search process to improve global search performance and convergence behavior. Thus, randomness in SCSO can be replaced by a chaotic map due to similar randomness features with better statistical and dynamic properties. In addition to these advantages, low search consistency, local optimum trap, inefficiency search, and low population diversity issues are also provided. In the proposed CSCSO, several chaotic maps are implemented for more efficient behavior in the exploration and exploitation phases. Experiments are conducted on a wide variety of well-known test functions to increase the reliability of the results, as well as real-world problems. In this study, the proposed algorithm was applied to a total of 39 functions and multidisciplinary problems. It found 76.3% better responses compared to a best-developed SCSO variant and other chaotic-based metaheuristics tested. This extensive experiment indicates that the CSCSO algorithm excels in providing acceptable results

Computational Optimizations for Machine Learning

The present book contains the 10 articles finally accepted for publication in the Special Issue “Computational Optimizations for Machine Learning” of the MDPI journal Mathematics, which cover a wide range of topics connected to the theory and applications of machine learning, neural networks and artificial intelligence. These topics include, among others, various types of machine learning classes, such as supervised, unsupervised and reinforcement learning, deep neural networks, convolutional neural networks, GANs, decision trees, linear regression, SVM, K-means clustering, Q-learning, temporal difference, deep adversarial networks and more. It is hoped that the book will be interesting and useful to those developing mathematical algorithms and applications in the domain of artificial intelligence and machine learning as well as for those having the appropriate mathematical background and willing to become familiar with recent advances of machine learning computational optimization mathematics, which has nowadays permeated into almost all sectors of human life and activity

Evolving machine learning and deep learning models using evolutionary algorithms

Despite the great success in data mining, machine learning and deep learning models are yet subject to material obstacles when tackling real-life challenges, such as feature selection, initialization sensitivity, as well as hyperparameter optimization. The prevalence of these obstacles has severely constrained conventional machine learning and deep learning methods from fulfilling their potentials. In this research, three evolving machine learning and one evolving deep learning models are proposed to eliminate above bottlenecks, i.e. improving model initialization, enhancing feature representation, as well as optimizing model configuration, respectively, through hybridization between the advanced evolutionary algorithms and the conventional ML and DL methods. Specifically, two Firefly Algorithm based evolutionary clustering models are proposed to optimize cluster centroids in K-means and overcome initialization sensitivity as well as local stagnation. Secondly, a Particle Swarm Optimization based evolving feature selection model is developed for automatic identification of the most effective feature subset and reduction of feature dimensionality for tackling classification problems. Lastly, a Grey Wolf Optimizer based evolving Convolutional Neural Network-Long Short-Term Memory method is devised for automatic generation of the optimal topological and learning configurations for Convolutional Neural Network-Long Short-Term Memory networks to undertake multivariate time series prediction problems. Moreover, a variety of tailored search strategies are proposed to eliminate the intrinsic limitations embedded in the search mechanisms of the three employed evolutionary algorithms, i.e. the dictation of the global best signal in Particle Swarm Optimization, the constraint of the diagonal movement in Firefly Algorithm, as well as the acute contraction of search territory in Grey Wolf Optimizer, respectively. The remedy strategies include the diversification of guiding signals, the adaptive nonlinear search parameters, the hybrid position updating mechanisms, as well as the enhancement of population leaders. As such, the enhanced Particle Swarm Optimization, Firefly Algorithm, and Grey Wolf Optimizer variants are more likely to attain global optimality on complex search landscapes embedded in data mining problems, owing to the elevated search diversity as well as the achievement of advanced trade-offs between exploration and exploitation

Dynamic Optimization Algorithms for Baseload Power Plant Cycling under Variable Renewable Energy

The growing deployment of variable renewable energy (VRE) sources, such as wind and solar, is mainly due to the decline in the cost of renewable technologies and the increase of societal and cultural pressures. Solar and wind power generation are also known to have zero marginal costs and fuel emissions during dispatch. Thereby, the VRE from these sources should be prioritized when available. However, the rapid deployment of VRE has heightened concerns regarding the challenges in the integration between fossil-fueled and renewable energy systems. The high variability introduced by the VRE as well as the limited alignment between demand and wind/solar power generation led to the increased need of dispatchable energy sources such as baseload natural gas- and coal-fired power plants to cycle their power outputs more often to reliably supply the net load. The increasing power plant cycling can introduce unexpected inefficiencies into the system that potentially incur higher costs, emissions, and wear-and-tear, as the power plants are no longer operating at their optimal design points. In this dissertation, dynamic optimization algorithms are developed and implemented for baseload power plant cycling under VRE penetration. Specifically, two different dynamic optimization strategies are developed for the minute and hourly time scales of grid operation. The minute-level strategy is based on a mixed-integer linear programming (MILP) formulation for dynamic dispatch of energy systems, such as natural gas- and coal-fired power plants and sodium sulfur batteries, under VRE while considering power plant equipment health-related constraints. The hourly-level strategy is based on a Nonlinear Multi-objective dynamic real-time Predictive Optimization (NMPO) implemented in a supercritical pulverized coal-fired (SCPC) power plant with a postcombustion carbon capture system (CCS), considering economic and environmental objectives. Different strategies are employed and explored to improve computational tractability, such as mathematical reformulations, automatic differentiation (AD), and parallelization of a metaheuristic particle swarm optimization (PSO) component. The MILP-based dynamic dispatch framework is used to simulate case studies considering different loads and renewable penetration levels for a suite of energy systems. The results show that grid flexibility is mostly provided by the natural gas power plant, while the batteries are used sparingly. Additionally, considering the post-optimization equivalent carbon analysis, the environmental performance is intrinsically connected to grid flexibility and the level of VRE penetration. The stress results reinforce the necessity of further considering and including equipment health-related constraints during dispatch. The results of the NMPO successfully implemented for a large-scale SCPC-CCS show that the optimal compromise is automatically chosen from the Pareto front according to a set of weights for the objectives with minimal interaction between the framework and the decision maker. They also indicate that to setup the optimization thresholds and constraints, knowledge of the power system operations is essential. Finally, the market and carbon policies have an impact on the optimal compromise between the economic and environmental objectives

Traveling Salesman Problem

The idea behind TSP was conceived by Austrian mathematician Karl Menger in mid 1930s who invited the research community to consider a problem from the everyday life from a mathematical point of view. A traveling salesman has to visit exactly once each one of a list of m cities and then return to the home city. He knows the cost of traveling from any city i to any other city j. Thus, which is the tour of least possible cost the salesman can take? In this book the problem of finding algorithmic technique leading to good/optimal solutions for TSP (or for some other strictly related problems) is considered. TSP is a very attractive problem for the research community because it arises as a natural subproblem in many applications concerning the every day life. Indeed, each application, in which an optimal ordering of a number of items has to be chosen in a way that the total cost of a solution is determined by adding up the costs arising from two successively items, can be modelled as a TSP instance. Thus, studying TSP can never be considered as an abstract research with no real importance

Metaheuristic Optimization of Power and Energy Systems: Underlying Principles and Main Issues of the `Rush to Heuristics'

In the power and energy systems area, a progressive increase of literature contributions that contain applications of metaheuristic algorithms is occurring. In many cases, these applications are merely aimed at proposing the testing of an existing metaheuristic algorithm on a specific problem, claiming that the proposed method is better than other methods that are based on weak comparisons. This ‘rush to heuristics’ does not happen in the evolutionary computation domain, where the rules for setting up rigorous comparisons are stricter but are typical of the domains of application of the metaheuristics. This paper considers the applications to power and energy systems and aims at providing a comprehensive view of the main issues that concern the use of metaheuristics for global optimization problems. A set of underlying principles that characterize the metaheuristic algorithms is presented. The customization of metaheuristic algorithms to fit the constraints of specific problems is discussed. Some weaknesses and pitfalls that are found in literature contributions are identified, and specific guidelines are provided regarding how to prepare sound contributions on the application of metaheuristic algorithms to specific problems

Particle Swarm Optimization

Particle swarm optimization (PSO) is a population based stochastic optimization technique influenced by the social behavior of bird flocking or fish schooling.PSO shares many similarities with evolutionary computation techniques such as Genetic Algorithms (GA). The system is initialized with a population of random solutions and searches for optima by updating generations. However, unlike GA, PSO has no evolution operators such as crossover and mutation. In PSO, the potential solutions, called particles, fly through the problem space by following the current optimum particles. This book represents the contributions of the top researchers in this field and will serve as a valuable tool for professionals in this interdisciplinary field