Improving a Particle Swarm Optimization-based Clustering Method

This thesis discusses clustering related works with emphasis on Particle Swarm Optimization (PSO) principles. Specifically, we review in detail the PSO clustering algorithm proposed by Van Der Merwe & Engelbrecht, the particle swarm clustering (PSC) algorithm proposed by Cohen & de Castro, Szabo’s modified PSC (mPSC), and Georgieva & Engelbrecht’s Cooperative-Multi-Population PSO (CMPSO). In this thesis, an improvement over Van Der Merwe & Engelbrecht’s PSO clustering has been proposed and tested for standard datasets. The improvements observed in those experiments vary from slight to moderate, both in terms of minimizing the cost function, and in terms of run time

Improving a Particle Swarm Optimization-based Clustering Method

This thesis discusses clustering related works with emphasis on Particle Swarm Optimization (PSO) principles. Specifically, we review in detail the PSO clustering algorithm proposed by Van Der Merwe & Engelbrecht, the particle swarm clustering (PSC) algorithm proposed by Cohen & de Castro, Szabo’s modified PSC (mPSC), and Georgieva & Engelbrecht’s Cooperative-Multi-Population PSO (CMPSO). In this thesis, an improvement over Van Der Merwe & Engelbrecht’s PSO clustering has been proposed and tested for standard datasets. The improvements observed in those experiments vary from slight to moderate, both in terms of minimizing the cost function, and in terms of run time

Evidence of Critical Dynamics in Movements of Bees inside a Hive

Social insects such as honey bees exhibit complex behavioral patterns, and their distributed behavioral coordination enables decision-making at the colony level. It has, therefore, been proposed that a high-level description of their collective behavior might share commonalities with the dynamics of neural processes in brains. Here, we investigated this proposal by focusing on the possibility that brains are poised at the edge of a critical phase transition and that such a state is enabling increased computational power and adaptability. We applied mathematical tools developed in computational neuroscience to a dataset of bee movement trajectories that were recorded within the hive during the course of many days. We found that certain characteristics of the activity of the bee hive system are consistent with the Ising model when it operates at a critical temperature, and that the system’s behavioral dynamics share features with the human brain in the resting state.journal articl

Incorporating Memory and Learning Mechanisms Into Meta-RaPS

Due to the rapid increase of dimensions and complexity of real life problems, it has become more difficult to find optimal solutions using only exact mathematical methods. The need to find near-optimal solutions in an acceptable amount of time is a challenge when developing more sophisticated approaches. A proper answer to this challenge can be through the implementation of metaheuristic approaches. However, a more powerful answer might be reached by incorporating intelligence into metaheuristics. Meta-RaPS (Metaheuristic for Randomized Priority Search) is a metaheuristic that creates high quality solutions for discrete optimization problems. It is proposed that incorporating memory and learning mechanisms into Meta-RaPS, which is currently classified as a memoryless metaheuristic, can help the algorithm produce higher quality results. The proposed Meta-RaPS versions were created by taking different perspectives of learning. The first approach taken is Estimation of Distribution Algorithms (EDA), a stochastic learning technique that creates a probability distribution for each decision variable to generate new solutions. The second Meta-RaPS version was developed by utilizing a machine learning algorithm, Q Learning, which has been successfully applied to optimization problems whose output is a sequence of actions. In the third Meta-RaPS version, Path Relinking (PR) was implemented as a post-optimization method in which the new algorithm learns the good attributes by memorizing best solutions, and follows them to reach better solutions. The fourth proposed version of Meta-RaPS presented another form of learning with its ability to adaptively tune parameters. The efficiency of these approaches motivated us to redesign Meta-RaPS by removing the improvement phase and adding a more sophisticated Path Relinking method. The new Meta-RaPS could solve even the largest problems in much less time while keeping up the quality of its solutions. To evaluate their performance, all introduced versions were tested using the 0-1 Multidimensional Knapsack Problem (MKP). After comparing the proposed algorithms, Meta-RaPS PR and Meta-RaPS Q Learning appeared to be the algorithms with the best and worst performance, respectively. On the other hand, they could all show superior performance than other approaches to the 0-1 MKP in the literature

Metode RCE-Kmeans untuk Clustering Data

Abstrak Telah banyak metode yang dikembangkan untuk memecahkan berbagai masalah clustering. Salah satunya menggunakan metode-metode dari bidang kecerdasan kelompok seperti Particle Swarm Optimization (PSO). Metode Rapid Centroid Estimation (RCE) merupakan salah satu metode clustering yang berbasis PSO. RCE, seperti varian PSO clustering lainnya, memiliki kelebihan yaitu hasil clustering tidak tergantung pada inisialisasi pusat cluster awal. RCE juga memiliki waktu komputasi yang jauh lebih cepat dibandingkan dengan metode sebelumnya yaitu Particle Swarm Clustering (PSC) dan modified Particle Swarm Clustering (mPSC), tetapi metode RCE memiliki standar deviasi kualitas skema clustering yang lebih tinggi dibandingkan PSC dan mPSC dimana  ini berpengaruh terhadap variansi hasil clustering. Hal ini terjadi karena equilibrium state, yaitu kondisi dimana posisi partikel tidak mengalami perubahan lagi, kurang tepat pada saat kriteria berhenti tercapai. Penelitian ini mengusulkan metode RCE-Kmeans yaitu metode yang mengaplikasikan K-means setelah equilibrium state metode RCE tercapai untuk memperbarui posisi partikel yang dihasilkan dari metode RCE. Hasil penelitian menunjukkan bahwa dari sepuluh dataset, metode RCE-Kmeans memiliki nilai kualitas skema clustering yang lebih baik pada 7 dataset dibandingkan K-means dan lebih baik pada 8 dataset dibandingkan dengan metode RCE. Penggunaan K-means pada metode RCE juga mampu menurunkan nilai standar deviasi dari metode RCE.   Kata kunci—Clustering Data, Particle Swarm, K-means, Rapid Centroid Estimation.   Abstract  There have been many methods developed to solve the clustering problem. One of them is method in swarm intelligence field such as Particle Swarm Optimization (PSO). Rapid Centroid Estimation (RCE) is a method of clustering based Particle Swarm Optimization. RCE, like other variants of PSO clustering, does not depend on initial cluster centers. Moreover, RCE has faster computational time than the previous method like PSC and mPSC. However, RCE has higher standar deviation value than PSC and mPSC in which has impact in the variance of clustering result. It is happaned because of improper equilibrium state, a condition in which the position of the particle does not change anymore, when  the stopping criteria is reached. This study proposes RCE-Kmeans which is a  method applying K-means after the equilibrium state of RCE  reached to update the particle's position which is generated from the RCE method. The results showed that RCE-Kmeans has better quality of the clustering scheme in 7 of 10 datasets compared to K-means and better in 8 of 10 dataset then RCE method. The use of K-means clustering on the RCE method is also able to reduce the standard deviation from RCE method.   Keywords—Data Clustering, Particle Swarm, K-means, Rapid Centroid Estimation

Metode RCE-Kmeans untuk Clustering Data

AbstrakTelah banyak metode yang dikembangkan untuk memecahkan berbagai masalah clustering. Salah satunya menggunakan metode-metode dari bidang kecerdasan kelompok seperti Particle Swarm Optimization (PSO). Metode Rapid Centroid Estimation (RCE) merupakan salah satu metode clustering yang berbasis PSO. RCE, seperti varian PSO clustering lainnya, memiliki kelebihan yaitu hasil clustering tidak tergantung pada inisialisasi pusat cluster awal. RCE juga memiliki waktu komputasi yang jauh lebih cepat dibandingkan dengan metode sebelumnya yaitu Particle Swarm Clustering (PSC) dan modified Particle Swarm Clustering (mPSC), tetapi metode RCE memiliki standar deviasi kualitas skema clustering yang lebih tinggi dibandingkan PSC dan mPSC dimana  ini berpengaruh terhadap variansi hasil clustering. Hal ini terjadi karena equilibrium state, yaitu kondisi dimana posisi partikel tidak mengalami perubahan lagi, kurang tepat pada saat kriteria berhenti tercapai. Penelitian ini mengusulkan metode RCE-Kmeans yaitu metode yang mengaplikasikan K-means setelah equilibrium state metode RCE tercapai untuk memperbarui posisi partikel yang dihasilkan dari metode RCE. Hasil penelitian menunjukkan bahwa dari sepuluh dataset, metode RCE-Kmeans memiliki nilai kualitas skema clustering yang lebih baik pada 7 dataset dibandingkan K-means dan lebih baik pada 8 dataset dibandingkan dengan metode RCE. Penggunaan K-means pada metode RCE juga mampu menurunkan nilai standar deviasi dari metode RCE.  Kata kunci—Clustering Data, Particle Swarm, K-means, Rapid Centroid Estimation.  Abstract There have been many methods developed to solve the clustering problem. One of them is method in swarm intelligence field such as Particle Swarm Optimization (PSO). Rapid Centroid Estimation (RCE) is a method of clustering based Particle Swarm Optimization. RCE, like other variants of PSO clustering, does not depend on initial cluster centers. Moreover, RCE has faster computational time than the previous method like PSC and mPSC. However, RCE has higher standar deviation value than PSC and mPSC in which has impact in the variance of clustering result. It is happaned because of improper equilibrium state, a condition in which the position of the particle does not change anymore, when  the stopping criteria is reached. This study proposes RCE-Kmeans which is a  method applying K-means after the equilibrium state of RCE  reached to update the particle's position which is generated from the RCE method. The results showed that RCE-Kmeans has better quality of the clustering scheme in 7 of 10 datasets compared to K-means and better in 8 of 10 dataset then RCE method. The use of K-means clustering on the RCE method is also able to reduce the standard deviation from RCE method. Keywords—Data Clustering, Particle Swarm, K-means, Rapid Centroid Estimation.   

Swarm Robotic Systems with Minimal Information Processing

This thesis is concerned with the design and analysis of behaviors in swarm robotic systems using minimal information acquisition and processing. The motivation for this work is to contribute in paving the way for the implementation of swarm robotic systems at physically small scales, which will open up new application domains for their operation. At these scales, the space and energy available for the integration of sensors and computational hardware within the individual robots is at a premium. As a result, trade-offs in performance can be justified if a task can be achieved in a more parsimonious way. A framework is developed whereby meaningful collective behaviors in swarms of robots can be shown to emerge without the robots, in principle, possessing any run-time memory or performing any arithmetic computations. This is achieved by the robots having only discrete-valued sensors, and purely reactive controllers. Black-box search methods are used to automatically synthesize these controllers for desired collective behaviors. This framework is successfully applied to two canonical tasks in swarm robotics: self-organized aggregation of robots, and self-organized clustering of objects by robots. In the case of aggregation, the robots are equipped with one binary sensor, which informs them whether or not there is another robot in their line of sight. This makes the structure of the robots’ controller simple enough that its entire space can be systematically searched to locate the optimal controller (within a finite resolution). In the case of object clustering, the robots’ sensor is extended to have three states, distinguishing between robots, objects, and the background. This still requires no run-time memory or arithmetic computations on the part of the robots. It is statistically shown that the extension of the sensor to have three states leads to a better performance as compared to the cases where the sensor is binary, and cannot distinguish between robots and objects, or robots and the background

Game theoretical models for clustering and resource sharing in macro-femtocells networks

One of the main challenges of cellular network operators is to keep a good network quality for their users. In most cases, network quality decreases in indoor environments causing users to switch from one operator to another. A promising solution to cope with this issue is the deployment of femtocells that are used mainly at homes to enhance the mobile network coverage. In fact, higher penetration of broadband and mobile phones with high requirements of new applications such as video conferencing and internet games are promoting femtocell market. However, the deployment of femtocells in existing macrocell networks is a very challenging task due to the high complexity of the resource allocation. In this thesis, we focus on proposing several solutions to address the resource allocation problem in macro-femtocell networks with dense deployment of femtocells based on clustering techniques. Clustering techniques are used to reduce the resource allocation complexity of dense-femtocell networks since the resources are allocated locally within each cluster. Furthermore, a cluster head is responsible for the allocation of resources to femtocells within the cluster which avoids the co-tier interference. The clustering techniques have been widely used for distributed resource allocation in heterogeneous networks through the use of game theory models. In this work, three distributed resource allocation algorithms based on cooperative and evolutionary games are proposed. In the first part, we discuss the resource allocation problem for the non-dense deployment of femtocells. Toward this goal, a coalitional game is used to incentive femtocells in the formation of clusters. The approach decomposes in: (i) a base station selection algorithm for public users, (ii) a clustering algorithm based on cooperative game theory and (iii) a resource allocation within each cluster based on the PSO technique. Besides, an interference control mechanism enabled femtocells to leave its current cluster when the interference levels are higher than an interference threshold. In the second part, we focus on a fair allocation of resources for macro-femtocell networks. We develop a clustering algorithm based on a cooperative game for non-dense femtocell network. The Shapley value is applied to find the marginal contribution of every femtocell to all the possible groups of femtocells, thus, finding the fair amount of resources to be allocated to each femtocell within a cluster. This solution is only applied for non-dense femtocell deployment due to that the complexity of calculating the Shapley value increases significantly with a large number of femtocells. Stability criteria based on the ε-concept of game theory is utilized to find the set of stable clusters. Finally, the analysis of the resource allocation for dense-femtocell deployment is addressed through an evolutionary game theory (EGT) model. It is assumed that EGT requires bounded rationality from players, this reduces the complexity and allows the dense deployment of femtocells. In addition, we demonstrate that the set of clusters formed with EGT are stable by means of the replicator dynamics. The proposed model also includes system analysis for users with low mobility such as pedestrians and cyclists

Collaborative Control of Autonomous Swarms with Resource Constraints

This dissertation focuses on the collaborative control of homogeneous UAV swarms. A two-level scheme is proposed by combining the high-level path planning and the lowlevel vehicle motion control. A decentralized artificial potential function (APF) based approach, which mimics the bacteria foraging process, is studied for the high-level path planning. The deterministic potential based approach, however, suffers from the local minima entrapment dilemma, which motivate us to fix the "flaw" that is naturally embedded. An innovative decentralized stochastic approach based on the Markov Random Filed (MRF) theory is proposed; this approach traditionally used in statistical mechanics and in image processing. By modeling the local interactions as Gibbs potentials, the movements of vehicles are then decided by using Gibbs sampler based simulated annealing (SA) algorithm. A two-step sampling scheme is proposed to coordinate vehicle networks: in the first sampling step, a vehicle is picked through a properly designed, configuration-dependent proposal distribution, and in the second sampling step, the vehicle makes a move by using the local characteristics of the Gibbs distribution. Convergence properties are established theoretically and confirmed with simulations. In order to reduce the communication cost and the delay, a fully parallel sampling algorithm is studied and analyzed accordingly. In practice, the stochastic nature of the proposed algorithm might lead to a high traveling cost. To mitigate this problem, a hybrid algorithm is eveloped by combining the Gibbs sampler based method with the deterministic gradient-flow method to gain the advantages of both approaches. The robustness of the Gibbs sampler based algorithm is also studied. The convergence properties are investigated for different types sensor errors including range-error and random-error. Error bounds are derived to guarantee the convergence of the stochastic algorithm. In the low-level motion control module, a model predictive control (MPC) approach is investigated for car-like UAV model. Multiple control objectives, for example, minimizing tracking error, avoiding actuator/state saturation, and minimizing control effort, are easily encoded in the objective function. Two numerical optimization approaches, gradient descendent approach and dynamic programming approach, are studied to strike the balance between computation time and complexity