Balancing Exploitation And Exploration Search Behavior On Nature-Inspired Clustering Algorithms

Nature-inspired optimization-based clustering techniques are powerful, robust and more sophisticated than the conventional clustering methods due to their stochastic and heuristic characteristics. Unfortunately, these algorithms suffer with several drawbacks such as the tendency to be trapped or stagnate into local optima and slow convergence rates. The latter drawbacks are consequences of the difficulty in balancing the exploration and exploitation processes which directly affect the final quality of the clustering solutions. Hence, this research has proposed three enhanced frameworks, namely, Optimized Gravitational-based (OGC), Density-Based Particle Swarm Optimization (DPSO), and Variance-based Differential Evolution with an Optional Crossover (VDEO) frameworks for data clustering. In the OGC framework, the exhibited explorative search behavior of the Gravitational Clustering (GC) algorithm has been addressed by (i) eliminating the agent velocity accumulation, and (ii) integrating an initialization method of agents using variance and median to subrogate the exploration process. Moreover, the balance between the exploration and exploitation processes in the DPSO framework is considered using a combination of (i) a kernel density estimation technique associated with new bandwidth estimation method and (ii) estimated multi-dimensional gravitational learning coefficients. Lastly, (i) a single-based solution representation, (ii) a switchable mutation scheme, (iii) a vector-based estimation of the mutation factor, and (iv) an optional crossover strategy are proposed in the VDEO framework. The overall performances of the three proposed frameworks have been compared with several current state-of-the-art clustering algorithms on 15 benchmark datasets from the UCI repository. The experimental results are also thoroughly evaluated and verified via non-parametric statistical analysis. Based on the obtained experimental results, the OGC, DPSO, and VDEO frameworks achieved an average enhancement up to 24.36%, 9.38%, and 11.98% of classification accuracy, respectively. All the frameworks also achieved the first rank by the Friedman aligned-ranks (FA) test in all evaluation metrics. Moreover, the three frameworks provided convergent performances in terms of the repeatability. Meanwhile, the OGC framework obtained a significant performance in terms of the classification accuracy, where the VDEO framework presented a significant performance in terms of cluster compactness. On the other hand, the DPSO framework favored the balanced state by producing very competitive results compared to the OGC and DPSO in both evaluation metrics. As a conclusion, balancing the search behavior notably enhanced the overall performance of the three proposed frameworks and made each of them an excellent tool for data clustering

Improved sine cosine algorithm with simulated annealing and singer chaotic map for Hadith classification

Feature selection (FS) represents an important task in classification. Hadith represents an example in which we can apply FS on it. Hadiths are the second major source of Islam after the Quran. Thousands of Hadiths are available in Islam, and these Hadiths are grouped into a number of classes. In the literature, there are many studies conducted for Hadiths classification. Sine Cosine Algorithm (SCA) is a new metaheuristic optimization algorithm. SCA algorithm is mainly based on exploring the search space using sine and cosine mathematical formulas to find the optimal solution. However, SCA, like other Optimization Algorithm (OA), suffers from the problem of local optima and solution diversity. In this paper, to overcome SCA problems and use it for the FS problem, two major improvements were introduced to the standard SCA algorithm. The first improvement includes the use of singer chaotic map within SCA to improve solutions diversity. The second improvement includes the use of the Simulated Annealing (SA) algorithm as a local search operator within SCA to improve its exploitation. In addition, the Gini Index (GI) is used to filter the resulted selected features to reduce the number of features to be explored by SCA. Furthermore, three new Hadith datasets were created. To evaluate the proposed Improved SCA (ISCA), the new three Hadiths datasets were used in our experiments. Furthermore, to confirm the generality of ISCA, we also applied it on 14 benchmark datasets from the UCI repository. The ISCA results were compared with the original SCA and the state-of-the-art algorithms such as Particle Swarm Optimization (PSO), Genetic Algorithm (GA), Grasshopper Optimization Algorithm (GOA), and the most recent optimization algorithm, Harris Hawks Optimizer (HHO). The obtained results confirm the clear outperformance of ISCA in comparison with other optimization algorithms and Hadith classification baseline works. From the obtained results, it is inferred that ISCA can simultaneously improve the classification accuracy while it selects the most informative features

TPPSO: A Novel Two-Phase Particle Swarm Optimization

Particle swarm optimization (PSO) is a stout and rapid searching algorithm that has been used in various applications. Nevertheless, its major drawback is the stagnation problem that arises in the later phases of the search process. To solve this problem, a proper balance between investigation and manipulation throughout the search process should be maintained. This article proposes a new PSO variant named two-phases PSO (TPPSO). The concept of TPPSO is to split the search process into two phases. The first phase performs the original PSO operations with linearly decreasing inertia weight, and its objective is to focus on exploration. The second phase focuses on exploitation by generating two random positions in each iteration that are close to the global best position. The two generated positions are compared with the global best position sequentially. If a generated position performs better than the global best position, then it replaces the global best position. To prove the effectiveness of the proposed algorithm, sixteen popular unimodal, multimodal, shifted, and rotated benchmarking functions have been used to compare its performance with other existing well-known PSO variants and non-PSO algorithms. Simulation results show that TPPSO outperforms the other modified and hybrid PSO variants regarding solution quality, convergence speed, and robustness. The convergence speed of TPPSO is extremely fast, making it a suitable optimizer for real-world optimization problems

Constraint Model for the Satellite Image Mosaic Selection Problem

peer reviewedSatellite imagery solutions are widely used to study and monitor different regions of the Earth. However, a single satellite image can cover only a limited area. In cases where a larger area of interest is studied, several images must be stitched together to create a single larger image, called a mosaic, that can cover the area. Today, with the increasing number of satellite images available for commercial use, selecting the images to build the mosaic is challenging, especially when the user wants to optimize one or more parameters, such as the total cost and the cloud coverage percentage in the mosaic. More precisely, for this problem the input is an area of interest, several satellite images intersecting the area, a list of requirements relative to the image and the mosaic, such as cloud coverage percentage, image resolution, and a list of objectives to optimize. We contribute to the constraint and mixed integer lineal programming formulation of this new problem, which we call the satellite image mosaic selection problem, which is a multi-objective extension of the polygon cover problem. We propose a dataset of realistic and challenging instances, where the images were captured by the satellite constellations SPOT, Pléiades and Pléiades Neo. We evaluate and compare the two proposed models and show their efficiency for large instances, up to 200 images

MOFL/D: A Federated Multi-objective Learning Framework with Decomposition

peer reviewedMulti-objective learning problems occur in all aspects of life and have been studied for decades, including in the field of machine learning. Many such problems also exist in distributed settings, where data cannot easily be shared. In recent years, joint machine learning has been made possible in such settings through the development of the Federated Learning (FL) paradigm. However, there is as of now very little research on the general problem of extending the FL concept to multi- objective learning, limiting such problems to non-cooperative individual learning. We address this gap by presenting a general framework for multi-objective FL, based on decomposition (MOFL/D). Our framework addresses the a posteriori type of multi-objective problem, where user preferences are not known during the optimisation process, allowing multiple participants to jointly find a set of solutions, each optimised for some distribution of preferences. We present an instantiation of the framework and validate it through experiments on a set of multi-objective benchmarking problems that are extended from well-known single- objective benchmarks.U-AGR-8025 - ILNAS PC2 (01/01/2021 - 31/12/2024) - BOUVRY Pasca

A Simple and Effective Approach Based on a Multi-Level Feature Selection for Automated Parkinson’s Disease Detection

Parkinson’s disease (PD), which is a slowly progressing neurodegenerative disorder, negatively affects people’s daily lives. Early diagnosis is of great importance to minimize the effects of PD. One of the most important symptoms in the early diagnosis of PD disease is the monotony and distortion of speech. Artificial intelligence-based approaches can help specialists and physicians to automatically detect these disorders. In this study, a new and powerful approach based on multi-level feature selection was proposed to detect PD from features containing voice recordings of already-diagnosed cases. At the first level, feature selection was performed with the Chi-square and L1-Norm SVM algorithms (CLS). Then, the features that were extracted from these algorithms were combined to increase the representation power of the samples. At the last level, those samples that were highly distinctive from the combined feature set were selected with feature importance weights using the ReliefF algorithm. In the classification stage, popular classifiers such as KNN, SVM, and DT were used for machine learning, and the best performance was achieved with the KNN classifier. Moreover, the hyperparameters of the KNN classifier were selected with the Bayesian optimization algorithm, and the performance of the proposed approach was further improved. The proposed approach was evaluated using a 10-fold cross-validation technique on a dataset containing PD and normal classes, and a classification accuracy of 95.4% was achieved

Dynamic Butterfly Optimization Algorithm for Feature Selection

Feature selection represents an essential pre-processing step for a wide range of Machine Learning approaches. Datasets typically contain irrelevant features that may negatively affect the classifier performance. A feature selector can reduce the number of these features and maximise the classifier accuracy. This paper proposes a Dynamic Butterfly Optimization Algorithm (DBOA) as an improved variant to Butterfly Optimization Algorithm (BOA) for feature selection problems. BOA represents one of the most recently proposed optimization algorithms. BOA has demonstrated its ability to solve different types of problems with competitive results compared to other optimization algorithms. However, the original BOA algorithm has problems when optimising high-dimensional problems. Such issues include stagnation into local optima and lacking solutions diversity during the optimization process. To alleviate these weaknesses of the original BOA, two significant improvements are introduced in the original BOA: the development of a Local Search Algorithm Based on Mutation (LSAM) operator to avoid local optima problem and the use of LSAM to improve BOA solutions diversity. To demonstrate the efficiency and superiority of the proposed DBOA algorithm, 20 benchmark datasets from the UCI repository are employed. The classification accuracy, the fitness values, the number of selected features, the statistical results, and convergence curves are reported for DBOA and its competing algorithms. These results demonstrate that DBOA significantly outperforms the comparative algorithms on the majority of the used performance metrics