A Comprehensive Survey on Particle Swarm Optimization Algorithm and Its Applications

Particle swarm optimization (PSO) is a heuristic global optimization method, proposed originally by Kennedy and Eberhart in 1995. It is now one of the most commonly used optimization techniques. This survey presented a comprehensive investigation of PSO. On one hand, we provided advances with PSO, including its modifications (including quantum-behaved PSO, bare-bones PSO, chaotic PSO, and fuzzy PSO), population topology (as fully connected, von Neumann, ring, star, random, etc.), hybridization (with genetic algorithm, simulated annealing, Tabu search, artificial immune system, ant colony algorithm, artificial bee colony, differential evolution, harmonic search, and biogeography-based optimization), extensions (to multiobjective, constrained, discrete, and binary optimization), theoretical analysis (parameter selection and tuning, and convergence analysis), and parallel implementation (in multicore, multiprocessor, GPU, and cloud computing forms). On the other hand, we offered a survey on applications of PSO to the following eight fields: electrical and electronic engineering, automation control systems, communication theory, operations research, mechanical engineering, fuel and energy, medicine, chemistry, and biology. It is hoped that this survey would be beneficial for the researchers studying PSO algorithms

Introductory Review of Swarm Intelligence Techniques

With the rapid upliftment of technology, there has emerged a dire need to fine-tune or optimize certain processes, software, models or structures, with utmost accuracy and efficiency. Optimization algorithms are preferred over other methods of optimization through experimentation or simulation, for their generic problem-solving abilities and promising efficacy with the least human intervention. In recent times, the inducement of natural phenomena into algorithm design has immensely triggered the efficiency of optimization process for even complex multi-dimensional, non-continuous, non-differentiable and noisy problem search spaces. This chapter deals with the Swarm intelligence (SI) based algorithms or Swarm Optimization Algorithms, which are a subset of the greater Nature Inspired Optimization Algorithms (NIOAs). Swarm intelligence involves the collective study of individuals and their mutual interactions leading to intelligent behavior of the swarm. The chapter presents various population-based SI algorithms, their fundamental structures along with their mathematical models.Comment: Submitted to Springe

Optimized task scheduling based on hybrid symbiotic organisms search algorithms for cloud computing environment

In Cloud Computing model, users are charged according to the usage of resources and desired Quality of Service (QoS). Task scheduling algorithms are responsible for specifying adequate set of resources to execute user applications in the form of tasks, and schedule decisions of task scheduling algorithms are based on QoS requirements defined by the user. Task scheduling problem is an NP-Complete problem, due to the NP-Complete nature of task scheduling problems and huge search space presented by large scale problem instances, many of the existing solution algorithms incur high computational complexity and cannot effectively obtain global optimum solutions. Recently, Symbiotic Organisms Search (SOS) has been applied to various optimization problems and results obtained were found to be competitive with state-of-the-art metaheuristic algorithms. However, similar to the case other metaheuristic optimization algorithms, the efficiency of SOS algorithm deteriorates as the size of the search space increases. Moreover, SOS suffers from local optima entrapment and its static control parameters cannot maintain a balance between local and global search. In this study, Cooperative Coevolutionary Constrained Multiobjective Symbiotic Organisms Search (CC-CMSOS), Cooperative Coevolutionary Constrained Multi-objective Memetic Symbiotic Organisms Search (CC-CMMSOS), and Cooperative Coevolutionary Constrained Multi-objective Adaptive Benefit Factor Symbiotic Organisms Search (CC-CMABFSOS) algorithms are proposed to solve constrained multi-objective large scale task scheduling optimization problem on IaaS cloud computing environment. To address the issue of scalability, the concept of Cooperative Coevolutionary for enhancing SOS named CC-CMSOS make SOS more efficient for solving large scale task scheduling problems. CC-CMMSOS algorithm further improves the performance of SOS algorithm by hybridizing with Simulated Annealing (SA) to avoid entrapment in local optima for global convergence. Finally, CC-CMABFSOS algorithm adaptively turn SOS control parameters to balance the local and global search procedure for faster convergence speed. The performance of the proposed CC-CMSOS, CC-CMMSOS, and CC-CMABFSOS algorithms are evaluated on CloudSim simulator, using both standard workload traces and synthesized workloads for larger problem instances of up to 5000. Moreover, CC-CMSOS, CC-CMMSOS, and CC-CMABFSOS algorithms are compared with multi-objective optimization algorithms, namely, EMS-C, ECMSMOO, and BOGA. The CC-CMSOS, CC-CMMSOS, and CC-CMABFSOS algorithms obtained significant improved optimal trade-offs between execution time (makespan) and financial cost (cost) while meeting deadline constraints with no computational overhead. The performance improvements obtained by the proposed algorithms in terms of hypervolume ranges from 8.72% to 37.95% across the workloads. Therefore, the proposed algorithms have potentials to improve the performance of QoS delivery

A PSO Application in Skull Prosthesis Modelling by Superellipse

This paper presents a method to create the geometric model of skull defects to be applied in anatomic prosthesis modelling. The approach is to generate an image that represents the missing information in the skull when bone`s defect is non-symmetric. We are proposing the use of superellipse concept to recover the parameters that represents the geometric shape of a skull bone curvature in tomography. If the superellipse is properly adjusted in each computed tomography slice, the arcs that represent the piece of missing bone can be modelled in 3D. The problem is that many similar ellipses can be created, and the best solution must be found. This research applies the Particle Swarm Optimization (PSO) algorithm in order to find the best solution for each tomographic slice. Once the solution found for each slice, the whole 3D missing information can be virtually rebuilt as an adjusted prosthesis model image

A PSO Application in Skull Prosthesis Modelling by Superellipse

This paper presents a method to create the geometric model of skull defects to be applied in anatomic prosthesis modelling. The approach is to generate an image that represents the missing information in the skull when bone's defect is non-symmetric. We are proposing the use of superellipse concept to recover the parameters that represents the geometric shape of a skull bone curvature in tomography. If the superellipse is properly adjusted in each computed tomography slice, the arcs that represent the piece of missing bone can be modelled in 3D. The problem is that many similar ellipses can be created, and the best solution must be found. This research applies the Particle Swarm Optimization (PSO) algorithm in order to find the best solution for each tomographic slice. Once the solution found for each slice, the whole 3D missing information can be virtually rebuilt as an adjusted prosthesis model image

Filter � GA Based Approach to Feature Selection for Classification

This paper presents a new approach to select reduced number of features in databases. Every database has a given number of features but it is observed that some of these features can be redundant and can be harmful as well as and can confuse the process of classification. The proposed method applies filter attribute measure and binary coded Genetic Algorithm to select a small subset of features. The importance of these features is judged by applying K-nearest neighbor (KNN) method of classification. The best reduced subset of features which has high classification accuracy on given databases is adopted. The classification accuracy obtained by proposed method is compared with that reported recently in publications on twenty eight databases. It is noted that proposed method performs satisfactory on these databases and achieves higher classification accuracy but with smaller number of features

Evolving machine learning and deep learning models using evolutionary algorithms

Despite the great success in data mining, machine learning and deep learning models are yet subject to material obstacles when tackling real-life challenges, such as feature selection, initialization sensitivity, as well as hyperparameter optimization. The prevalence of these obstacles has severely constrained conventional machine learning and deep learning methods from fulfilling their potentials. In this research, three evolving machine learning and one evolving deep learning models are proposed to eliminate above bottlenecks, i.e. improving model initialization, enhancing feature representation, as well as optimizing model configuration, respectively, through hybridization between the advanced evolutionary algorithms and the conventional ML and DL methods. Specifically, two Firefly Algorithm based evolutionary clustering models are proposed to optimize cluster centroids in K-means and overcome initialization sensitivity as well as local stagnation. Secondly, a Particle Swarm Optimization based evolving feature selection model is developed for automatic identification of the most effective feature subset and reduction of feature dimensionality for tackling classification problems. Lastly, a Grey Wolf Optimizer based evolving Convolutional Neural Network-Long Short-Term Memory method is devised for automatic generation of the optimal topological and learning configurations for Convolutional Neural Network-Long Short-Term Memory networks to undertake multivariate time series prediction problems. Moreover, a variety of tailored search strategies are proposed to eliminate the intrinsic limitations embedded in the search mechanisms of the three employed evolutionary algorithms, i.e. the dictation of the global best signal in Particle Swarm Optimization, the constraint of the diagonal movement in Firefly Algorithm, as well as the acute contraction of search territory in Grey Wolf Optimizer, respectively. The remedy strategies include the diversification of guiding signals, the adaptive nonlinear search parameters, the hybrid position updating mechanisms, as well as the enhancement of population leaders. As such, the enhanced Particle Swarm Optimization, Firefly Algorithm, and Grey Wolf Optimizer variants are more likely to attain global optimality on complex search landscapes embedded in data mining problems, owing to the elevated search diversity as well as the achievement of advanced trade-offs between exploration and exploitation

Comprehensive Taxonomies of Nature- and Bio-inspired Optimization: Inspiration versus Algorithmic Behavior, Critical Analysis and Recommendations

In recent years, a great variety of nature- and bio-inspired algorithms has been reported in the literature. This algorithmic family simulates different biological processes observed in Nature in order to efficiently address complex optimization problems. In the last years the number of bio-inspired optimization approaches in literature has grown considerably, reaching unprecedented levels that dark the future prospects of this field of research. This paper addresses this problem by proposing two comprehensive, principle-based taxonomies that allow researchers to organize existing and future algorithmic developments into well-defined categories, considering two different criteria: the source of inspiration and the behavior of each algorithm. Using these taxonomies we review more than three hundred publications dealing with nature-inspired and bio-inspired algorithms, and proposals falling within each of these categories are examined, leading to a critical summary of design trends and similarities between them, and the identification of the most similar classical algorithm for each reviewed paper. From our analysis we conclude that a poor relationship is often found between the natural inspiration of an algorithm and its behavior. Furthermore, similarities in terms of behavior between different algorithms are greater than what is claimed in their public disclosure: specifically, we show that more than one-third of the reviewed bio-inspired solvers are versions of classical algorithms. Grounded on the conclusions of our critical analysis, we give several recommendations and points of improvement for better methodological practices in this active and growing research field.Comment: 76 pages, 6 figure

Component-wise analysis of metaheuristic algorithms for novel fuzzy-meta classifier

Metaheuristic research has proposed promising results in science, business, and engineering problems. But, mostly high-level analysis is performed on metaheuristic performances. This leaves several critical questions unanswered due to black-box issue that does not reveal why certain metaheuristic algorithms performed better on some problems and not on others. To address the significant gap between theory and practice in metaheuristic research, this study proposed in-depth analysis approach using component-view of metaheuristic algorithms and diversity measurement for determining exploration and exploitation abilities. This research selected three commonly used swarm-based metaheuristic algorithms – Particle Swarm Optimization (PSO), Artificial Bee Colony (ABC), and Cuckoo Search (CS) – to perform component-wise analysis. As a result, the study able to address premature convergence problem in PSO, poor exploitation in ABC, and imbalanced exploration and exploitation issue in CS. The proposed improved PSO (iPSO), improved ABC (iABC), and improved CS (iCS) outperformed standard algorithms and variants from existing literature, as well as, Grey Wolf Optimization (GWO) and Animal Migration Optimization (AMO) on ten numerical optimization problems with varying modalities. The proposed iPSO, iABC, and iCS were then employed on proposed novel Fuzzy-Meta Classifier (FMC) which offered highly reduced model complexity and high accuracy as compared to Adaptive Neuro-Fuzzy Inference System (ANFIS). The proposed three-layer FMC produced efficient rules that generated nearly 100% accuracies on ten different classification datasets, with significantly reduced number of trainable parameters and number of nodes in the network architecture, as compared to ANFIS

Computational Optimizations for Machine Learning

The present book contains the 10 articles finally accepted for publication in the Special Issue “Computational Optimizations for Machine Learning” of the MDPI journal Mathematics, which cover a wide range of topics connected to the theory and applications of machine learning, neural networks and artificial intelligence. These topics include, among others, various types of machine learning classes, such as supervised, unsupervised and reinforcement learning, deep neural networks, convolutional neural networks, GANs, decision trees, linear regression, SVM, K-means clustering, Q-learning, temporal difference, deep adversarial networks and more. It is hoped that the book will be interesting and useful to those developing mathematical algorithms and applications in the domain of artificial intelligence and machine learning as well as for those having the appropriate mathematical background and willing to become familiar with recent advances of machine learning computational optimization mathematics, which has nowadays permeated into almost all sectors of human life and activity