Optimal Clustering Framework for Hyperspectral Band Selection

Band selection, by choosing a set of representative bands in hyperspectral image (HSI), is an effective method to reduce the redundant information without compromising the original contents. Recently, various unsupervised band selection methods have been proposed, but most of them are based on approximation algorithms which can only obtain suboptimal solutions toward a specific objective function. This paper focuses on clustering-based band selection, and proposes a new framework to solve the above dilemma, claiming the following contributions: 1) An optimal clustering framework (OCF), which can obtain the optimal clustering result for a particular form of objective function under a reasonable constraint. 2) A rank on clusters strategy (RCS), which provides an effective criterion to select bands on existing clustering structure. 3) An automatic method to determine the number of the required bands, which can better evaluate the distinctive information produced by certain number of bands. In experiments, the proposed algorithm is compared to some state-of-the-art competitors. According to the experimental results, the proposed algorithm is robust and significantly outperform the other methods on various data sets

Feed-Forward Neural Network Soft-Sensor Modeling of Flotation Process Based on Particle Swarm Optimization and Gravitational Search Algorithm

For predicting the key technology indicators (concentrate grade and tailings recovery rate) of flotation process, a feed-forward neural network (FNN) based soft-sensor model optimized by the hybrid algorithm combining particle swarm optimization (PSO) algorithm and gravitational search algorithm (GSA) is proposed. Although GSA has better optimization capability, it has slow convergence velocity and is easy to fall into local optimum. So in this paper, the velocity vector and position vector of GSA are adjusted by PSO algorithm in order to improve its convergence speed and prediction accuracy. Finally, the proposed hybrid algorithm is adopted to optimize the parameters of FNN soft-sensor model. Simulation results show that the model has better generalization and prediction accuracy for the concentrate grade and tailings recovery rate to meet the online soft-sensor requirements of the real-time control in the flotation process

Multidimensional Particle Swarm Optimization for Machine Learning

Particle Swarm Optimization (PSO) is a stochastic nature-inspired optimization method. It has been successfully used in several application domains since it was introduced in 1995. It has been especially successful when applied to complicated multimodal problems, where simpler optimization methods, e.g., gradient descent, are not able to find satisfactory results. Multidimensional Particle Swarm Optimization (MD-PSO) and Fractional Global Best Formation (FGBF) are extensions of the basic PSO. MD-PSO allows searching for an optimum also when the solution dimensionality is unknown. With a dedicated dimensional PSO process, MD-PSO can search for optimal solution dimensionality. An interleaved positional PSO process simultaneously searches for the optimal solution in that dimensionality. Both the basic PSO and its multidimensional extension MD-PSO are susceptible to premature convergence. FGBF is a plug-in to (MD-)PSO that can help avoid premature convergence and find desired solutions faster. This thesis focuses on applications of MD-PSO and FGBF in different machine learning tasks.Multiswarm versions of MD-PSO and FGBF are introduced to perform dynamic optimization tasks. In dynamic optimization, the search space slowly changes. The locations of optima move and a former local optimum may transform into a global optimum and vice versa. We exploit multiple swarms to track different optima.In order to apply MD-PSO for clustering tasks, two key questions need to be answered: 1) How to encode the particles to represent different data partitions? 2) How to evaluate the fitness of the particles to evaluate the quality of the solutions proposed by the particle positions? The second question is considered especially carefully in this thesis. An extensive comparison of Clustering Validity Indices (CVIs) commonly used as fitness functions in Particle Swarm Clustering (PSC) is conducted. Furthermore, a novel approach to carry out fitness evaluation, namely Fitness Evaluation with Computational Centroids (FECC) is introduced. FECC gives the same fitness to any particle positions that lead to the same data partition. Therefore, it may save some computational efforts and, above all, it can significantly improve the results obtained by using any of the best performing CVIs as the PSC fitness function.MD-PSO can also be used to evolve different neural networks. The results of training Multilayer Perceptrons (MLPs) using the common Backpropagation (BP) algorithm and a global technique based on PSO are compared. The pros and cons of BP and (MD-)PSO in MLP training are discussed. For training Radial Basis Function Neural Networks (RBFNNs), a novel technique based on class-specific clustering of the training samples is introduced. The proposed approach is compared to the common input and input-output clustering approaches and the benefits of using the class-specific approach are experimentally demonstrated. With the class-specific approach, the training complexity is reduced, while the classification performance of the trained RBFNNs may be improved.Collective Network of Binary Classifiers (CNBC) is an evolutionary semantic classifier consisting of several Networks of Binary Classifiers (NBCs) trained to recognize a certain semantic class. NBCs in turn consist of several Binary Classifiers (BCs), which are trained for a certain feature type. Thanks to its topology and the use of MD-PSO as its evolution technique, incremental training can be easily applied to add new training items, classes, and/or features.In feature synthesis, the objective is to exploit ground truth information to transform the original low-level features into more discriminative ones. To learn an efficient synthesis for a dataset, only a fraction of the data needs to be labeled. The learned synthesis can then be applied on unlabeled data to improve classification or retrieval results. In this thesis, two different feature synthesis techniques are introduced. In the first one, MD-PSO is directly used to find proper arithmetic operations to be applied on the elements of the original low-level feature vectors. In the second approach, feature synthesis is carried out using one-against-all perceptrons. In the latter technique, the best results were obtained when MD-PSO was used to train the perceptrons.In all the mentioned applications excluding MLP training, MD-PSO is used together with FGBF. Overall, MD-PSO and FGBF are indeed versatile tools in machine learning. However, computational limitations constrain their use in currently emerging machine learning systems operating on Big Data. Therefore, in the future, it is necessary to divide complex tasks into smaller subproblems and to conquer the large problems via solving the subproblems where the use of MD-PSO and FGBF becomes feasible. Several applications discussed in this thesis already exploit the divide-and-conquer operation model

K-Means and Alternative Clustering Methods in Modern Power Systems

As power systems evolve by integrating renewable energy sources, distributed generation, and electric vehicles, the complexity of managing these systems increases. With the increase in data accessibility and advancements in computational capabilities, clustering algorithms, including K-means, are becoming essential tools for researchers in analyzing, optimizing, and modernizing power systems. This paper presents a comprehensive review of over 440 articles published through 2022, emphasizing the application of K-means clustering, a widely recognized and frequently used algorithm, along with its alternative clustering methods within modern power systems. The main contributions of this study include a bibliometric analysis to understand the historical development and wide-ranging applications of K-means clustering in power systems. This research also thoroughly examines K-means, its various variants, potential limitations, and advantages. Furthermore, the study explores alternative clustering algorithms that can complete or substitute K-means. Some prominent examples include K-medoids, Time-series K-means, BIRCH, Bayesian clustering, HDBSCAN, CLIQUE, SPECTRAL, SOMs, TICC, and swarm-based methods, broadening the understanding and applications of clustering methodologies in modern power systems. The paper highlights the wide-ranging applications of these techniques, from load forecasting and fault detection to power quality analysis and system security assessment. Throughout the examination, it has been observed that the number of publications employing clustering algorithms within modern power systems is following an exponential upward trend. This emphasizes the necessity for professionals to understand various clustering methods, including their benefits and potential challenges, to incorporate the most suitable ones into their studies

Object Tracking in Video Images based on Image Segmentation and Pattern Matching

The moving object tracking in video pictures [1] has attracted a great deal of interest in computer vision. For object recognition, navigation systems and surveillance systems [10], object tracking is an indispensable first-step. We propose a novel algorithm for object tracking in video pictures, based on image segmentation and pattern matching [1]. With the image segmentation, we can detect all objects in images no matter whether they are moving or not. Using image segmentation results of successive frames, we exploit pattern matching in a simple feature space for tracking of the objects. Consequently, the proposed algorithm can be applied to multiple moving and still objects even in the case of a moving camera. We describe the algorithm in detail and perform simulation experiments on object tracking which verify the tracking algorithm‘s efficiency. VLSI implementation of the proposed algorithm is possible. The conventional approach to object tracking is based on the difference between the current image and the background image. However, algorithms based on the difference image cannot simultaneously detect still objects. Furthermore, they cannot be applied to the case of a moving camera. Algorithms including the camera motion information have been proposed previously, but, they still contain problems in separating the information from the background. The proposed algorithm, consisting of four stages i.e. image segmentation, feature extraction as well as object tracking and motion vector determination [12]. Here Image Segmentation is done in 3 ways and the efficiency of the tracking is compared in these three ways, the segmentation techniques used are ―Fuzzy C means clustering using Particle Swarm Optimization [5],[6],[17]”, ”Otsu’s global thresholding [16]”, ”Histogram based thresholding by manual threshold selection”, after image segmentation the features of each object are taken and Pattern Matching [10],[11],[20] algorithm is run on consecutive frames of video sequence, so that the pattern of extracted features is matched in the next frame , the motion of the object from reference frame to present frame is calculated in both X and Y directions, the mask is moved in the image accordingly, hence the moving object in the video sequences will be tracked

Evolving machine learning and deep learning models using evolutionary algorithms

Despite the great success in data mining, machine learning and deep learning models are yet subject to material obstacles when tackling real-life challenges, such as feature selection, initialization sensitivity, as well as hyperparameter optimization. The prevalence of these obstacles has severely constrained conventional machine learning and deep learning methods from fulfilling their potentials. In this research, three evolving machine learning and one evolving deep learning models are proposed to eliminate above bottlenecks, i.e. improving model initialization, enhancing feature representation, as well as optimizing model configuration, respectively, through hybridization between the advanced evolutionary algorithms and the conventional ML and DL methods. Specifically, two Firefly Algorithm based evolutionary clustering models are proposed to optimize cluster centroids in K-means and overcome initialization sensitivity as well as local stagnation. Secondly, a Particle Swarm Optimization based evolving feature selection model is developed for automatic identification of the most effective feature subset and reduction of feature dimensionality for tackling classification problems. Lastly, a Grey Wolf Optimizer based evolving Convolutional Neural Network-Long Short-Term Memory method is devised for automatic generation of the optimal topological and learning configurations for Convolutional Neural Network-Long Short-Term Memory networks to undertake multivariate time series prediction problems. Moreover, a variety of tailored search strategies are proposed to eliminate the intrinsic limitations embedded in the search mechanisms of the three employed evolutionary algorithms, i.e. the dictation of the global best signal in Particle Swarm Optimization, the constraint of the diagonal movement in Firefly Algorithm, as well as the acute contraction of search territory in Grey Wolf Optimizer, respectively. The remedy strategies include the diversification of guiding signals, the adaptive nonlinear search parameters, the hybrid position updating mechanisms, as well as the enhancement of population leaders. As such, the enhanced Particle Swarm Optimization, Firefly Algorithm, and Grey Wolf Optimizer variants are more likely to attain global optimality on complex search landscapes embedded in data mining problems, owing to the elevated search diversity as well as the achievement of advanced trade-offs between exploration and exploitation

Automatic object classification for surveillance videos.

PhDThe recent popularity of surveillance video systems, specially located in urban scenarios, demands the development of visual techniques for monitoring purposes. A primary step towards intelligent surveillance video systems consists on automatic object classification, which still remains an open research problem and the keystone for the development of more specific applications. Typically, object representation is based on the inherent visual features. However, psychological studies have demonstrated that human beings can routinely categorise objects according to their behaviour. The existing gap in the understanding between the features automatically extracted by a computer, such as appearance-based features, and the concepts unconsciously perceived by human beings but unattainable for machines, or the behaviour features, is most commonly known as semantic gap. Consequently, this thesis proposes to narrow the semantic gap and bring together machine and human understanding towards object classification. Thus, a Surveillance Media Management is proposed to automatically detect and classify objects by analysing the physical properties inherent in their appearance (machine understanding) and the behaviour patterns which require a higher level of understanding (human understanding). Finally, a probabilistic multimodal fusion algorithm bridges the gap performing an automatic classification considering both machine and human understanding. The performance of the proposed Surveillance Media Management framework has been thoroughly evaluated on outdoor surveillance datasets. The experiments conducted demonstrated that the combination of machine and human understanding substantially enhanced the object classification performance. Finally, the inclusion of human reasoning and understanding provides the essential information to bridge the semantic gap towards smart surveillance video systems

A population-based optimization method using Newton fractal

Department of Mathematical SciencesMetaheuristic is a general procedure to draw an agreement in a group based on the decision making of each individual beyond heuristic. For last decade, there have been many attempts to develop metaheuristic methods based on swarm intelligence to solve global optimization such as particle swarm optimizer, ant colony optimizer, firefly optimizer. These methods are mostly stochastic and independent on specific problems. Since metaheuristic methods based on swarm intelligence require no central coordination (or minimal, if any), they are especially well-applicable to those problems which have distributed or parallel structures. Each individual follows few simple rules, keeping the searching cost at a decent level. Despite its simplicity, the methods often yield a fast approximation in good precision, compared to conventional methods. Exploration and exploitation are two important features that we need to consider to find a global optimum in a high dimensional domain, especially when prior information is not given. Exploration is to investigate the unknown space without using the information from history to find undiscovered optimum. Exploitation is to trace the neighborhood of the current best to improve it using the information from history. Because these two concepts are at opposite ends of spectrum, the tradeoff significantly affects the performance at the limited cost of search. In this work, we develop a chaos-based metaheuristic method, ???Newton Particle Optimization(NPO)???, to solve global optimization problems. The method is based on the Newton method which is a well-established mathematical root-finding procedure. It actively utilizes the chaotic nature of the Newton method to place a proper balance between exploration and exploitation. While most current population-based methods adopt stochastic effects to maximize exploration, they often suffer from weak exploitation. In addition, stochastic methods generally show poor reproducing ability and premature convergence. It has been argued that an alternative approach using chaos may mitigate such disadvantages. The unpredictability of chaos is correspondent with the randomness of stochastic methods. Chaos-based methods are deterministic and therefore easy to reproduce the results with less memory. It has been shown that chaos avoids local optimum better than stochastic methods and buffers the premature convergence issue. Newton method is deterministic but shows chaotic movements near the roots. It is such complexity that enables the particles to search the space for global optimization. We initialize the particle???s position randomly at first and choose the ???leading particles??? to attract other particles near them. We can make a polynomial function whose roots are those leading particles, called ???a guiding function???. Then we update the positions of particles using the guiding function by Newton method. Since the roots are not updated by Newton method, the leading particles survive after update. For diverse movements of particles, we use modified newton method, which has a coefficient $m$ in the variation of movements for each particle. Efficiency in local search is closely related to the value of m which determines the convergence rate of the Newton method. We can control the balance between exploration and exploitation by choice of leading particles. It is interesting that selection of excellent particles as leading particles not always results in the best result. Including mediocre particles in the roots of guiding function maintains the diversity of particles in position. Though diversity seems to be inefficient at first, those particles contribute to the exploration for global search finally. We study the conditions for the convergence of NPO. NPO enjoys the well-established analysis of the Newton method. This contrasts with other ???nature-inspired??? algorithms which have often been criticized for lack of rigorous mathematical ground. We compare the results of NPO with those of two popular metaheuristic methods, particle swarm optimizer(PSO) and firefly optimizer(FO). Though it has been shown that there are no such algorithms superior to all problems by no free lunch theorem, that is why the researchers are concerned about adaptable global optimizer for specific problems. NPO shows good performance to CEC 2013 competition test problems comparing to PSO and FO.ope

A Novel Histogram-Based Multi-Threshold Searching Algorithm for Multilevel Color Thresholding

[[abstract]]Image segmentation is an important preliminary process required in object tracking applications. This paper addresses the issue of unsupervised multi‐colour thresholding design for colour‐based multiple objects segmentation. Most of the current unsupervised colour thresholding techniques require adopting a supervised training algorithm or a cluster‐number decision algorithm to obtain optimal threshold values of each colour channel for a colour‐of‐interest. In this paper, a novel unsupervised multi‐threshold searching algorithm is proposed to automatically search the optimal threshold values for segmenting multiple colour objects. To achieve this, a novel ratio‐map image computation method is proposed to efficiently enhance the contrast between colour and non¬colour pixels. The Otsu’s method is then applied to the ratio‐map image to extract all colour objects from the image. Finally, a new histogram‐based multi‐threshold searching algorithm is developed to search the optimal upper‐bound and lower‐bound threshold values of hue, saturation and brightness components for each colour object. Experimental results show that the proposed method not only succeeds in separating all colour objects-of-interest in colour images, but also provides satisfactory colour thresholding results compared with an existing multilevel thresholding method.[[notice]]補正完畢[[incitationindex]]SCI[[incitationindex]]EI[[booktype]]電子版[[booktype]]紙