Structural bias in population-based algorithms

Challenging optimisation problems are abundant in all areas of science and industry. Since the 1950s, scientists have responded to this by developing ever-diversifying families of 'black box' optimisation algorithms. The latter are designed to be able to address any optimisation problem, requiring only that the quality of any candidate solution can be calculated via a 'fitness function' specific to the problem. For such algorithms to be successful, at least three properties are required: (i) an effective informed sampling strategy, that guides the generation of new candidates on the basis of the fitnesses and locations of previously visited candidates; (ii) mechanisms to ensure efficiency, so that (for example) the same candidates are not repeatedly visited; and (iii) the absence of structural bias, which, if present, would predispose the algorithm towards limiting its search to specific regions of the solution space. The first two of these properties have been extensively investigated, however the third is little understood and rarely explored. In this article we provide theoretical and empirical analyses that contribute to the understanding of structural bias. In particular, we state and prove a theorem concerning the dynamics of population variance in the case of real-valued search spaces and a 'flat' fitness landscape. This reveals how structural bias can arise and manifest as non-uniform clustering of the population over time. Critically, theory predicts that structural bias is exacerbated with (independently) increasing population size, and increasing problem difficulty. These predictions, supported by our empirical analyses, reveal two previously unrecognised aspects of structural bias that would seem vital for algorithm designers and practitioners. Respectively, (i) increasing the population size, though ostensibly promoting diversity, will magnify any inherent structural bias, and (ii) the effects of structural bias are more apparent when faced with (many classes of) 'difficult' problems. Our theoretical result also contributes to the 'exploitation/exploration' conundrum in optimisation algorithm design, by suggesting that two commonly used approaches to enhancing exploration - increasing the population size, and increasing the disruptiveness of search operators - have quite distinct implications in terms of structural bias

区分線形系粒子群最適化法における粒子間ネットワークに関する性能評価

これまでに著者は決定論的な粒子群最適化手法である区分線形系粒子群最適化法(PPSO) を提案した．PPSO の粒子は収束モードと発散モードの二つの回転モードを持ち，探索過程に伴い動的に二つの回転モードが切り替わることで解空間上の探索を行う．PPSO において，粒子群は多次元解空間の座標軸方向への探索に偏らずに解空間上を自由に飛び回ることができるため，回転問題に対する探索性能が高いことが示されている．本論文ではPPSO の探索性能を向上させるために，粒子間の情報共有に近傍構造を導入したN-PPSO(PPSO with Neighborhood Topology) を提案する．N-PPSO の有効性を示すために，粒子群最適化法の粒子間に近傍構造を導入したN-PSO (PSO with Neighborhood Topology) との比較実験を行う．Piecewise-linear particle swarm optimizer (PPSO) was proposed, which is a deterministic particle swarm optimization. In PPSO, each particle has two search modes which are a convergence mode and a divergence mode, and switches both search modes irregularly. PPSO is effective to solve rotated problems, because PPSO particles can move in solution spaces toward various directions. Here, in order to improve search performances of PPSO, a neighborhood topology between particles is introduced to PPSO (N-PPSO). We compared the search performances of N-PPSO with those of PSO with the neighborhood topology (N-PSO) through the numerical simulations

Analysis of behaviours in swarm systems

In nature animal species often exist in groups. We talk of insect swarms, flocks of birds, packs of lions, herds of wildebeest etc. These are characterised by individuals interacting by following their own rules, privy only to local information. Robotic swarms or simulations can be used explore such interactions. Mathematical formulations can be constructed that encode similar ideas and allow us to explore the emergent group behaviours. Some behaviours show characteristics reminiscent of the phenomena of criticality. A bird flock may show near instantaneous collective shifts in direction: velocity changes that appear to correlated over distances much larger individual separations. Here we examine swarm systems inspired by flocks of birds and the role played by criticality. The first system, Particle Swarm Optimisation (PSO), is shown to behave optimally when operating close to criticality. The presence of a critical point in the algorithm’s operation is shown to derive from the swarm’s properties as a random dynamical system. Empirical results demonstrate that the optimality lies on or near this point. A modified PSO algorithm is presented which uses measures of the swarm’s diversity as a feedback signal to adjust the behaviour of the swarm. This achieves a statistically balanced mixture of exploration and exploitation behaviours in the resultant swarm. The problems of stagnation and parameter tuning often encountered in PSO are automatically avoided. The second system, Swarm Chemistry, consists of heterogeneous particles combined with kinetic update rules. It is known that, depending upon the parametric configuration, numerous structures visually reminiscent of biological forms are found in this system. The parameter set discovered here results in a cell-division-like behaviour (in the sense of prokaryotic fission). Extensions to the swarm system produces a swarm that shows repeated cell division. As such, this model demonstrates a behaviour of interest to theories regarding the origin of life

Evolutionary Algorithms and Computational Methods for Derivatives Pricing

This work aims to provide novel computational solutions to the problem of derivative pricing. To achieve this, a novel hybrid evolutionary algorithm (EA) based on particle swarm optimisation (PSO) and differential evolution (DE) is introduced and applied, along with various other state-of-the-art variants of PSO and DE, to the problem of calibrating the Heston stochastic volatility model. It is found that state-of-the-art DEs provide excellent calibration performance, and that previous use of rudimentary DEs in the literature undervalued the use of these methods. The use of neural networks with EAs for approximating the solution to derivatives pricing models is next investigated. A set of neural networks are trained from Monte Carlo (MC) simulation data to approximate the closed form solution for European, Asian and American style options. The results are comparable to MC pricing, but with offline evaluation of the price using the neural networks being orders of magnitudes faster and computationally more efficient. Finally, the use of custom hardware for numerical pricing of derivatives is introduced. The solver presented here provides an energy efficient data-flow implementation for pricing derivatives, which has the potential to be incorporated into larger high-speed/low energy trading systems