Improving K-means clustering with enhanced Firefly Algorithms

In this research, we propose two variants of the Firefly Algorithm (FA), namely inward intensified exploration FA (IIEFA) and compound intensified exploration FA (CIEFA), for undertaking the obstinate problems of initialization sensitivity and local optima traps of the K-means clustering model. To enhance the capability of both exploitation and exploration, matrix-based search parameters and dispersing mechanisms are incorporated into the two proposed FA models. We first replace the attractiveness coefficient with a randomized control matrix in the IIEFA model to release the FA from the constraints of biological law, as the exploitation capability in the neighbourhood is elevated from a one-dimensional to multi-dimensional search mechanism with enhanced diversity in search scopes, scales, and directions. Besides that, we employ a dispersing mechanism in the second CIEFA model to dispatch fireflies with high similarities to new positions out of the close neighbourhood to perform global exploration. This dispersing mechanism ensures sufficient variance between fireflies in comparison to increase search efficiency. The ALL-IDB2 database, a skin lesion data set, and a total of 15 UCI data sets are employed to evaluate efficiency of the proposed FA models on clustering tasks. The minimum Redundancy Maximum Relevance (mRMR)-based feature selection method is also adopted to reduce feature dimensionality. The empirical results indicate that the proposed FA models demonstrate statistically significant superiority in both distance and performance measures for clustering tasks in comparison with conventional K-means clustering, five classical search methods, and five advanced FA variants

Improved Firefly Algorithm with Variable Neighborhood Search for Data Clustering

من بين الخوارزميات الأدلة العليا (الميتاهيورستك)، تعد الخوارزميات القائمة على البحوث المتعددة (المجتمع) خوارزمية بحث استكشافية متفوقة كخوارزمية البحث المحلية من حيث استكشاف مساحة البحث للعثور على الحلول المثلى العالمية. ومع ذلك، فإن الجانب السلبي الأساسي للخوارزميات القائمة على البحوث المتعددة (المجتمع) هو قدرتها الاستغلالية المنخفضة، مما يمنع توسع منطقة البحث عن الحلول المثلى. خوارزمية اليَرَاعَة المضيئة (Firefly (FA هي خوارزمية تعتمد على المجتمع والتي تم استخدامها على نطاق واسع في مشاكل التجميع. ومع ذلك، فإن FA مقيد بتقاربها السابق لأوانه عندما لا يتم استخدام استراتيجيات بحث محلي لتحسين جودة حلول المجموعات في منطقة المجاورة واستكشاف المناطق العالمية في مساحة البحث. على هذا الأساس، فإن الهدف من هذا العمل هو تحسين FA باستخدام البحث المتغير في الأحياء (VNS) كطريقة بحث محلية (FA-VNS)، وبالتالي توفير فائدة VNS للمفاضلة بين قدرات الاستكشاف والاستغلال. يسمح FA-VNS المقترح لليراعات بتحسين حلول التجميع مع القدرة على تعزيز حلول التجميع والحفاظ على تنوع حلول التجميع أثناء عملية البحث باستخدام مشغلي الاضطراب في VNS. لتقييم أداء الخوارزمية، يتم استخدام ثماني مجموعات بيانات معيارية مع أربع خوارزميات تجميع معروفة. تشير المقارنة وفقًا لمقاييس التقييم الداخلية والخارجية إلى أن FA-VNS المقترحة يمكن أن تنتج حلول تجميع أكثر إحكاما من خوارزميات التجميع المعروفة.Among the metaheuristic algorithms, population-based algorithms are an explorative search algorithm superior to the local search algorithm in terms of exploring the search space to find globally optimal solutions. However, the primary downside of such algorithms is their low exploitative capability, which prevents the expansion of the search space neighborhood for more optimal solutions. The firefly algorithm (FA) is a population-based algorithm that has been widely used in clustering problems. However, FA is limited in terms of its premature convergence when no neighborhood search strategies are employed to improve the quality of clustering solutions in the neighborhood region and exploring the global regions in the search space. On these bases, this work aims to improve FA using variable neighborhood search (VNS) as a local search method, providing VNS the benefit of the trade-off between the exploration and exploitation abilities. The proposed FA-VNS allows fireflies to improve the clustering solutions with the ability to enhance the clustering solutions and maintain the diversity of the clustering solutions during the search process using the perturbation operators of VNS. To evaluate the performance of the algorithm, eight benchmark datasets are utilized with four well-known clustering algorithms. The comparison according to the internal and external evaluation metrics indicates that the proposed FA-VNS can produce more compact clustering solutions than the well-known clustering algorithms

Evolving machine learning and deep learning models using evolutionary algorithms

Despite the great success in data mining, machine learning and deep learning models are yet subject to material obstacles when tackling real-life challenges, such as feature selection, initialization sensitivity, as well as hyperparameter optimization. The prevalence of these obstacles has severely constrained conventional machine learning and deep learning methods from fulfilling their potentials. In this research, three evolving machine learning and one evolving deep learning models are proposed to eliminate above bottlenecks, i.e. improving model initialization, enhancing feature representation, as well as optimizing model configuration, respectively, through hybridization between the advanced evolutionary algorithms and the conventional ML and DL methods. Specifically, two Firefly Algorithm based evolutionary clustering models are proposed to optimize cluster centroids in K-means and overcome initialization sensitivity as well as local stagnation. Secondly, a Particle Swarm Optimization based evolving feature selection model is developed for automatic identification of the most effective feature subset and reduction of feature dimensionality for tackling classification problems. Lastly, a Grey Wolf Optimizer based evolving Convolutional Neural Network-Long Short-Term Memory method is devised for automatic generation of the optimal topological and learning configurations for Convolutional Neural Network-Long Short-Term Memory networks to undertake multivariate time series prediction problems. Moreover, a variety of tailored search strategies are proposed to eliminate the intrinsic limitations embedded in the search mechanisms of the three employed evolutionary algorithms, i.e. the dictation of the global best signal in Particle Swarm Optimization, the constraint of the diagonal movement in Firefly Algorithm, as well as the acute contraction of search territory in Grey Wolf Optimizer, respectively. The remedy strategies include the diversification of guiding signals, the adaptive nonlinear search parameters, the hybrid position updating mechanisms, as well as the enhancement of population leaders. As such, the enhanced Particle Swarm Optimization, Firefly Algorithm, and Grey Wolf Optimizer variants are more likely to attain global optimality on complex search landscapes embedded in data mining problems, owing to the elevated search diversity as well as the achievement of advanced trade-offs between exploration and exploitation