An adaptive Michigan approach PSO for nearest prototype classification

Proceedings of: Second International Work-Conference on the Interplay Between Natural and Artificial Computation, IWINAC 2007, La Manga del Mar Menor, Spain, June 18-21, 2007.Nearest Prototype methods can be quite successful on many pattern classification problems. In these methods, a collection of prototypes has to be found that accurately represents the input patterns. The classifier then assigns classes based on the nearest prototype in this collection. In this paper we develop a new algorithm (called AMPSO), based on the Particle Swarm Optimization (PSO) algorithm, that can be used to find those prototypes. Each particle in a swarm represents a single prototype in the solution; the swarm evolves using modified PSO equations with both particle competition and cooperation. Experimentation includes an artificial problem and six common application problems from the UCI data sets. The results show that the AMPSO algorithm is able to find solutions with a reduced number of prototypes that classify data with comparable or better accuracy than the 1-NN classifier. The algorithm can also be compared or improves the results of many classical algorithms in each of those problems; and the results show that AMPSO also performs significantly better than any tested algorithm in one of the problems.This article has been financed by the Spanish founded research MEC project OPLINK::UC3M, Ref: TIN2005-08818-C04-02 and CAM project UC3M-TEC-05-029

AMPSO: A new Particle Swarm Method for Nearest Neighborhood Classification

Nearest prototype methods can be quite successful on many pattern classification problems. In these methods, a collection of prototypes has to be found that accurately represents the input patterns. The classifier then assigns classes based on the nearest prototype in this collection. In this paper, we first use the standard particle swarm optimizer (PSO) algorithm to find those prototypes. Second, we present a new algorithm, called adaptive Michigan PSO (AMPSO) in order to reduce the dimension of the search space and provide more flexibility than the former in this application. AMPSO is based on a different approach to particle swarms as each particle in the swarm represents a single prototype in the solution. The swarm does not converge to a single solution; instead, each particle is a local classifier, and the whole swarm is taken as the solution to the problem. It uses modified PSO equations with both particle competition and cooperation and a dynamic neighborhood. As an additional feature, in AMPSO, the number of prototypes represented in the swarm is able to adapt to the problem, increasing as needed the number of prototypes and classes of the prototypes that make the solution to the problem. We compared the results of the standard PSO and AMPSO in several benchmark problems from the University of California, Irvine, data sets and find that AMPSO always found a better solution than the standard PSO. We also found that it was able to improve the results of the Nearest Neighbor classifiers, and it is also competitive with some of the algorithms most commonly used for classification.This work was supported by the Spanish founded research Project MSTAR::UC3M, Ref: TIN2008-06491-C04-03 and CAM Project CCG06-UC3M/ESP-0774.Publicad

Michigan Particle Swarm Optimization for Prototype Reduction in Classification Problems

This paper presents a new approach to Particle Swarm Optimization, called Michigan Approach PSO (MPSO), and its applica- tion to continuous classi cation problems as a Nearest Prototype (NP) classi er. In Nearest Prototype classi ers, a collection of prototypes has to be found that accurately represents the input patterns. The classi er then assigns classes based on the nearest prototype in this collection. The MPSO algorithm is used to process training data to nd those prototypes. In the MPSO algorithm each particle in a swarm represents a single pro- totype in the solution and it uses modi ed movement rules with particle competition and cooperation that ensure particle diversity. The proposed method is tested both with arti cial problems and with real benchmark problems and compared with several algorithms of the same family. Re- sults show that the particles are able to recognize clusters, nd decision boundaries and reach stable situations that also retain adaptation po- tential. The MPSO algorithm is able to improve the accuracy of 1-NN classi ers, obtains results comparable to the best among other classi ers, and improves the accuracy reported in literature for one of the problems.Publicad

Clasificación mediante enjambre de prototipos

Los Clasificadores por Vecino ms Prximo (K-NN) han recibido un impulso renovado con la aplicación de metaheursticas de búsqueda (evolutivas, etc.) que permiten optimizar su funcionamiento, mediante la optimización de atributos, reducción de prototipos, y evolución de medidas globales o locales de proximidad; ello permite desarrollar clasificadores robustos e insensibles al ruido, competitivos con otros paradigmas de clasificación. Se aporta a este campo un nuevo algoritmo denominado Clasificador mediante Enjambre de Prototipos (CEP o PSC). Se inspira en el algoritmo de Optimización mediante Enjambres de Partículas (PSO), pero introduce un nuevo enfoque que permite abordar problemas de clasificación de forma flexible y escalable. El algoritmo utiliza un enfoque de Michigan para codificar las posiciones de un conjunto de prototipos, que se desplazan por el espacio de los atributos del problema mediante interacciones de tipo atractivo y repulsivo caractersticas de PSO. Se comporta como un optimizador para una función de evaluación multimodal y dinámica, que mide la calidad de cada prototipo. Los resultados experimentales son competitivos con una variedad de algoritmos de referencia. Como conclusión, se proponen campos de aplicación alternativos, y se avanza la posibilidad de generalizar la propuesta como nueva perspectiva dentro de en la Inteligencia de Enjambre _____________________________________________Nearest Neighbor Classifiers are a subject of renewed interest with the application of search metaheuristics (such as evolutionary techniques), that may improve their performance by means of attribute selection, prototype reduction and global or local proximity measures; these methods provide the ability to develop robust and noise-insensitive classifiers that may be competitive with other classification paradigms. We contribute to this field a new algorithm, called Prototype Swarm Classifier. It is inspired in the Particle Swarm Optimization algorithm, but it develops a new approach that enables the algorithm to deal with classification problems in a flexible and scalable way. The algorithm uses a Michigan approach to encode positions of a set of prototypes, that move in attribute space using the attractive and repulsive forces that are characteristic of PSO. It behaves by optimization of a multimodal dynamic local fitness function, that measures the quality of each prototype. Experimental results are competitive with those of a set of reference algoritms. In the conclusions, we propose other fields of application, and we introduce the possibility of generalization of the proposal as a new perspective in the field of Swarm Intelligence

Instance-based Learning with Prototype Reduction for Real-Time Proportional Myocontrol: A Randomized User Study Demonstrating Accuracy-preserving Data Reduction for Prosthetic Embedded Systems

This work presents the design, implementation and validation of learning techniques based on the kNN scheme for gesture detection in prosthetic control. To cope with high computational demands in instance-based prediction, methods of dataset reduction are evaluated considering real-time determinism to allow for the reliable integration into battery-powered portable devices. The influence of parameterization and varying proportionality schemes is analyzed, utilizing an eight-channel-sEMG armband. Besides offline cross-validation accuracy, success rates in real-time pilot experiments (online target achievement tests) are determined. Based on the assessment of specific dataset reduction techniques' adequacy for embedded control applications regarding accuracy and timing behaviour, Decision Surface Mapping (DSM) proves itself promising when applying kNN on the reduced set. A randomized, double-blind user study was conducted to evaluate the respective methods (kNN and kNN with DSM-reduction) against Ridge Regression (RR) and RR with Random Fourier Features (RR-RFF). The kNN-based methods performed significantly better (p<0.0005) than the regression techniques. Between DSM-kNN and kNN, there was no statistically significant difference (significance level 0.05). This is remarkable in consideration of only one sample per class in the reduced set, thus yielding a reduction rate of over 99% while preserving success rate. The same behaviour could be confirmed in an extended user study. With k=1, which turned out to be an excellent choice, the runtime complexity of both kNN (in every prediction step) as well as DSM-kNN (in the training phase) becomes linear concerning the number of original samples, favouring dependable wearable prosthesis applications

Computational intelligence techniques for maximum energy efficiency of cogeneration processes based on internal combustion engines

153 p.El objeto de la tesis consiste en desarrollar estrategias de modelado y optimización del rendimiento energético de plantas de cogeneración basadas en motores de combustión interna (MCI), mediante el uso de las últimas tecnologías de inteligencia computacional. Con esta finalidad se cuenta con datos reales de una planta de cogeneración de energía, propiedad de la compañía EnergyWorks, situada en la localidad de Monzón (provincia de Huesca). La tesis se realiza en el marco de trabajo conjunto del Grupo de Diseño en Electrónica Digital (GDED) de la Universidad del País Vasco UPV/EHU y la empresa Optimitive S.L., empresa dedicada al software avanzado para la mejora en tiempo real de procesos industriale

Novel Computationally Intelligent Machine Learning Algorithms for Data Mining and Knowledge Discovery

This thesis addresses three major issues in data mining regarding feature subset selection in large dimensionality domains, plausible reconstruction of incomplete data in cross-sectional applications, and forecasting univariate time series. For the automated selection of an optimal subset of features in real time, we present an improved hybrid algorithm: SAGA. SAGA combines the ability to avoid being trapped in local minima of Simulated Annealing with the very high convergence rate of the crossover operator of Genetic Algorithms, the strong local search ability of greedy algorithms and the high computational efficiency of generalized regression neural networks (GRNN). For imputing missing values and forecasting univariate time series, we propose a homogeneous neural network ensemble. The proposed ensemble consists of a committee of Generalized Regression Neural Networks (GRNNs) trained on different subsets of features generated by SAGA and the predictions of base classifiers are combined by a fusion rule. This approach makes it possible to discover all important interrelations between the values of the target variable and the input features. The proposed ensemble scheme has two innovative features which make it stand out amongst ensemble learning algorithms: (1) the ensemble makeup is optimized automatically by SAGA; and (2) GRNN is used for both base classifiers and the top level combiner classifier. Because of GRNN, the proposed ensemble is a dynamic weighting scheme. This is in contrast to the existing ensemble approaches which belong to the simple voting and static weighting strategy. The basic idea of the dynamic weighting procedure is to give a higher reliability weight to those scenarios that are similar to the new ones. The simulation results demonstrate the validity of the proposed ensemble model