Particle swarm optimization : understanding order-2 stability guarantees

This paper’s primary aim is to provide clarity on which guarantees about particle stability can actually be made. The particle swarm optimization algorithm has undergone a considerable amount of theoretical analysis. However, with this abundance of theory has come some terminological inconstancies, and as a result it is easy for a practitioner to be misguided by overloaded terminology. Specifically, the criteria for both order-1 and order-2 stability are well studied, but the exact definition of order-2 stability is not consistent amongst researchers. A consequence of this inconsistency in terminology is that the existing theory may in fact misguide practitioners instead of assisting them. In this paper it is theoretically and empirically demonstrated which practical guarantees can in fact be made about particle stability. Specifically, it is shown that the definition of order-2 stability which accurately reflects PSO behavior is that of convergence in second order moment to a constant, and not to zero.http://link.springer.combookseries/5582020-03-30hj2020Computer Scienc

The importance of component-wise stochasticity in particle swarm optimization

This paper illustrates the importance of independent, component-wise stochastic scaling values, from both a theoretical and empirical perspective. It is shown that a swarm employing scalar stochasticity is unable to express every point in the search space if the problem dimensionality is sufficiently large in comparison to the swarm size. The theoretical result is emphasized by an empirical experiment, comparing the performance of a scalar swarm on benchmarks with reachable and unreachable optima. It is shown that a swarm using scalar stochasticity performs significantly worse when the optimum is not in the span of its initial positions. Lastly, it is demonstrated that a scalar swarm performs significantly worse than a swarm with component-wise stochasticity on a large range of benchmark functions, even when the problem dimensionality allows the scalar swarm to reach the optima.The National Research Foundation (NRF) of South Africa (Grant Number 46712).http://link.springer.combookseries/5582019-10-03hj2018Computer Scienc

Stability analysis of the multi-objective multi-guided particle swarm optimizer

At present particle swarm optimizers (PSO) designed for multi-objective optimization have undergone no form of theoretical stability analysis. This paper derives the sufficient and necessary conditions for order-1 and order-2 stability of the recently proposed multi-guided PSO (MGPSO), which was designed specifically for multi-objective optimization. The paper utilizes a recently published theorem for performing stability analysis on PSO variants, which requires minimal modeling assumptions. It is vital for PSO practitioners to know the actual criteria for particle stability of the given PSO variant being used, as it been shown that particle stability has a considerable impact on PSO’s performance. This paper empirically validates its theoretical findings by comparing the derived stability criteria against those of an assumption free MGPSO algorithm. It was found that the derived criteria for order-1 and order-2 stability are an accurate predictor of the unsimplified MGPSO’s particle behavior.http://link.springer.combookseries/5582019-10-03hj2018Computer Scienc

Critical considerations on angle modulated particle swarm optimisers

This article investigates various aspects of angle modulated particle swarm optimisers (AMPSO). Previous attempts at improving the algorithm have only been able to produce better results in a handful of test cases. With no clear understanding of when and why the algorithm fails, improving the algorithm’s performance has proved to be a difficult and sometimes blind undertaking. Therefore, the aim of this study is to identify the circumstances under which the algorithm might fail, and to understand and provide evidence for such cases. It is shown that the general assumption that good solutions are grouped together in the search space does not hold for the standard AMPSO algorithm or any of its existing variants. The problem is explained by specific characteristics of the generating function used in AMPSO. Furthermore, it is shown that the generating function also prevents particle velocities from decreasing, hindering the algorithm’s ability to exploit the binary solution space. Methods are proposed to both confirm and potentially solve the problems found in this study. In particular, this study addresses the problem of finding suitable generating functions for the first time. It is shown that the potential of a generating function to solve arbitrary binary optimisation problems can be quantified. It is further shown that a novel generating function with a single coefficient is able to generate solutions to binary optimisation problems with fewer than four dimensions. The use of ensemble generating functions is proposed as a method to solve binary optimisation problems with more than 16 dimensions.National Research Foundation (South Africa

Boundary constraint handling techniques for particle swarm optimization in high dimensional problem spaces

This paper investigates the use of boundary constraint handling mechanisms to prevent unwanted particle roaming behaviour in high dimensional spaces. The paper tests a range of strategies on a benchmark for large scale optimization. The empirical analysis shows that the hyperbolic strategy, which scales down a particle’s velocity as it approaches the boundary, performs statistically significantly better than the other methods considered in terms of the best objective function value achieved. The hyperbolic strategy directly addresses the velocity explosion, thereby preventing unwanted roaming.The National Research Foundation (NRF) of South Africa (Grant Number 46712).http://link.springer.combookseries/5582019-10-03hj2018Computer Scienc