A generalized theoretical deterministic particle swarm model

A number of theoretical studies of particle swarm optimization (PSO) have been done to gain a better understanding of the dynamics of the algorithm and the behavior of the particles under different conditions. These theoretical analyses have been performed for both the deterministic PSO model and more recently for the stochastic model. However, all current theoretical analyses of the PSO algorithm were based on the stagnation assumption, in some form or another. The analysis done under the stagnation assumption is one where the personal best and neighborhood best positions are assumed to be non-changing. While analysis under the stagnation assumption is very informative, it could never provide a complete description of a PSO’s behavior. Furthermore, the assumption implicitly removes the notion of a social network structure from the analysis. This paper presents a generalization to the theoretical deterministicPSOmodel. Under the generalized model, conditions for particle convergence to a point are derived. The model used in this paper greatly weakens the stagnation assumption, by instead assuming that each particle’s personal best and neighborhood best can occupy an arbitrarily large number of unique positions. It was found that the conditions derived in previous theoretical deterministic PSO research could be obtained as a specialization of the new generalized model proposed. Empirical results are presented to support the theoretical findings.http://link.springer.com/journal/11721hb201

Statistical Mechanics of Canonical-Dissipative Systems and Applications to Swarm Dynamics

We develop the theory of canonical-dissipative systems, based on the assumption that both the conservative and the dissipative elements of the dynamics are determined by invariants of motion. In this case, known solutions for conservative systems can be used for an extension of the dynamics, which also includes elements such as the take-up/dissipation of energy. This way, a rather complex dynamics can be mapped to an analytically tractable model, while still covering important features of non-equilibrium systems. In our paper, this approach is used to derive a rather general swarm model that considers (a) the energetic conditions of swarming, i.e. for active motion, (b) interactions between the particles based on global couplings. We derive analytical expressions for the non-equilibrium velocity distribution and the mean squared displacement of the swarm. Further, we investigate the influence of different global couplings on the overall behavior of the swarm by means of particle-based computer simulations and compare them with the analytical estimations.Comment: 14 pages incl. 13 figures. v2: misprints in Eq. (40) corrected, ref. updated. For related work see also: http://summa.physik.hu-berlin.de/~frank/active.htm

Nash Equilibria, collusion in games and the coevolutionary particle swarm algorithm

In recent work, we presented a deterministic algorithm to investigate collusion between players in a game where the players’ payoff functions are subject to a variational inequality describing the equilibrium of a transportation system. In investigating the potential for collusion between players, the diagonalization algorithm returned a local optimum. In this paper, we apply a coevolutionary particle swarm optimization (PSO) algorithm developed in earlier research in an attempt to return the global maximum. A numerical experiment is used to verify the performance of the algorithm in overcoming local optimum

A comparison of general-purpose optimization algorithms forfinding optimal approximate experimental designs

Several common general purpose optimization algorithms are compared for findingA- and D-optimal designs for different types of statistical models of varying complexity,including high dimensional models with five and more factors. The algorithms of interestinclude exact methods, such as the interior point method, the Nelder–Mead method, theactive set method, the sequential quadratic programming, and metaheuristic algorithms,such as particle swarm optimization, simulated annealing and genetic algorithms.Several simulations are performed, which provide general recommendations on theutility and performance of each method, including hybridized versions of metaheuristicalgorithms for finding optimal experimental designs. A key result is that general-purposeoptimization algorithms, both exact methods and metaheuristic algorithms, perform wellfor finding optimal approximate experimental designs

Generalised additive multiscale wavelet models constructed using particle swarm optimisation and mutual information for spatio-temporal evolutionary system representation

A new class of generalised additive multiscale wavelet models (GAMWMs) is introduced for high dimensional spatio-temporal evolutionary (STE) system identification. A novel two-stage hybrid learning scheme is developed for constructing such an additive wavelet model. In the first stage, a new orthogonal projection pursuit (OPP) method, implemented using a particle swarm optimisation(PSO) algorithm, is proposed for successively augmenting an initial coarse wavelet model, where relevant parameters of the associated wavelets are optimised using a particle swarm optimiser. The resultant network model, obtained in the first stage, may however be a redundant model. In the second stage, a forward orthogonal regression (FOR) algorithm, implemented using a mutual information method, is then applied to refine and improve the initially constructed wavelet model. The proposed two-stage hybrid method can generally produce a parsimonious wavelet model, where a ranked list of wavelet functions, according to the capability of each wavelet to represent the total variance in the desired system output signal is produced. The proposed new modelling framework is applied to real observed images, relative to a chemical reaction exhibiting a spatio-temporal evolutionary behaviour, and the associated identification results show that the new modelling framework is applicable and effective for handling high dimensional identification problems of spatio-temporal evolution sytems

Lattice dynamical wavelet neural networks implemented using particle swarm optimisation for spatio-temporal system identification

Starting from the basic concept of coupled map lattices, a new family of adaptive wavelet neural networks, called lattice dynamical wavelet neural networks (LDWNN), is introduced for spatiotemporal system identification, by combining an efficient wavelet representation with a coupled map lattice model. A new orthogonal projection pursuit (OPP) method, coupled with a particle swarm optimisation (PSO) algorithm, is proposed for augmenting the proposed network. A novel two-stage hybrid training scheme is developed for constructing a parsimonious network model. In the first stage, by applying the orthogonal projection pursuit algorithm, significant wavelet-neurons are adaptively and successively recruited into the network, where adjustable parameters of the associated waveletneurons are optimised using a particle swarm optimiser. The resultant network model, obtained in the first stage, may however be redundant. In the second stage, an orthogonal least squares (OLS) algorithm is then applied to refine and improve the initially trained network by removing redundant wavelet-neurons from the network. The proposed two-stage hybrid training procedure can generally produce a parsimonious network model, where a ranked list of wavelet-neurons, according to the capability of each neuron to represent the total variance in the system output signal is produced. Two spatio-temporal system identification examples are presented to demonstrate the performance of the proposed new modelling framework

A Tutorial on Clique Problems in Communications and Signal Processing

Since its first use by Euler on the problem of the seven bridges of K\"onigsberg, graph theory has shown excellent abilities in solving and unveiling the properties of multiple discrete optimization problems. The study of the structure of some integer programs reveals equivalence with graph theory problems making a large body of the literature readily available for solving and characterizing the complexity of these problems. This tutorial presents a framework for utilizing a particular graph theory problem, known as the clique problem, for solving communications and signal processing problems. In particular, the paper aims to illustrate the structural properties of integer programs that can be formulated as clique problems through multiple examples in communications and signal processing. To that end, the first part of the tutorial provides various optimal and heuristic solutions for the maximum clique, maximum weight clique, and $k$-clique problems. The tutorial, further, illustrates the use of the clique formulation through numerous contemporary examples in communications and signal processing, mainly in maximum access for non-orthogonal multiple access networks, throughput maximization using index and instantly decodable network coding, collision-free radio frequency identification networks, and resource allocation in cloud-radio access networks. Finally, the tutorial sheds light on the recent advances of such applications, and provides technical insights on ways of dealing with mixed discrete-continuous optimization problems