Non-Direct Encoding Method Based on Cellular Automata to Design Neural Network Architectures

Architecture design is a fundamental step in the successful application of Feed forward Neural Networks. In most cases a large number of neural networks architectures suitable to solve a problem exist and the architecture design is, unfortunately, still a human expert’s job. It depends heavily on the expert and on a tedious trial-and-error process. In the last years, many works have been focused on automatic resolution of the design of neural network architectures. Most of the methods are based on evolutionary computation paradigms. Some of the designed methods are based on direct representations of the parameters of the network. These representations do not allow scalability; thus, for representing large architectures very large structures are required. More interesting alternatives are represented by indirect schemes. They codify a compact representation of the neural network. In this work, an indirect constructive encoding scheme is proposed. This scheme is based on cellular automata representations and is inspired by the idea that only a few seeds for the initial configuration of a cellular automaton can produce a wide variety of feed forward neural networks architectures. The cellular approach is experimentally validated in different domains and compared with a direct codification scheme.Publicad

Non-Direct Encoding Method Based on Cellular Automata to Design Neural Network Architectures

Architecture design is a fundamental step in the successful application of Feed forward Neural Networks. In most cases a large number of neural networks architectures suitable to solve a problem exist and the architecture design is, unfortunately, still a human expert's job. It depends heavily on the expert and on a tedious trial-and-error process. In the last years, many works have been focused on automatic resolution of the design of neural network architectures. Most of the methods are based on evolutionary computation paradigms. Some of the designed methods are based on direct representations of the parameters of the network. These representations do not allow scalability; thus, for representing large architectures very large structures are required. More interesting alternatives are represented by indirect schemes. They codify a compact representation of the neural network. In this work, an indirect constructive encoding scheme is proposed. This scheme is based on cellular automata representations and is inspired by the idea that only a~few seeds for the initial configuration of a cellular automaton can produce a wide variety of feed forward neural networks architectures. The cellular approach is experimentally validated in different domains and compared with a direct codification scheme

Automatic design of artificial neural networks to forecast time series

Actas de: III Simposio de Inteligencia Computacional, SICO 2010, Valencia, 8-10 septiembre, 2010In this work an approach to design Artificial Neural Networks (ANN) to forecast Time Series is tackled. The approach is an automatic method that is carried out by an Evolutionary Algorithm (as a search algorithm) to design ANN. A key issue for these kinds of approaches is what information is included into the chromosome that represents an ANN There are two principal ideas about this question: first, the chromosome contains information about parameters of the topology, architecture, learning parameters, etc. of the ANN. The results using a parameter Encoding Scheme to design ANN for a Time Series Competition are shownPublicad

Design of artificial neural networks based on genetic algorithms to forecast time series

In this work an initial approach to design Artificial Neural Networks to forecast time series is tackle, and the automatic process to design is carried out by a Genetic Algorithm. A key issue for these kinds of approaches is what information is included in the chromosome that represents an Artificial Neural Network. There are two principal ideas about this question: first, the chromosome contains information about parameters of the topology, architecture, learning parameters, etc. of the Artificial Neural Network, i.e. Direct Encoding Scheme; second, the chromosome contains the necessary information so that a constructive method gives rise to an Artificial Neural Network topology (or architecture), i.e. Indirect Encoding Scheme. The results for a Direct Encoding Scheme (in order to compare with Indirect Encoding Schemes developed in future works) to design Artificial Neural Networks for NN3 Forecasting Time Series Competition are shown

ADANN: Automatic Design of Artificial Neural Networks

Proceeding of: Genetic and Evolutionary Computation Conference, GECCO-08. July 12-16, 2008, Atlanta, Georgia, USA.In this work an improvement of an initial approach to design Artificial Neural Networks to forecast Time Series is tackled, and the automatic process to design Artificial Neural Networks is carried out by a Genetic Algorithm. A key issue for these kinds of approaches is what information is included in the chromosome that represents an Artificial Neural Network. In this approach new information will be included into the chromosome so it will be possible to compare these results with those obtained in a previous approach. There are two principal ideas to take into account: first, the chromosome contains information about parameters of the topology, architecture, learning parameters, etc. of the Artificial Neural Network, i.e. Direct Encoding Scheme; second, the chromosome contains the necessary information so that a constructive method gives rise to an Artificial Neural Network topology (or architecture), i.e. Indirect Encoding Scheme. The results for a Direct Encoding Scheme (in order to compare with Indirect Encoding Schemes developed in future works) to design Artificial Neural Networks to forecast Time Series are shown.The research reported here has been supported by the Ministry of Education and Science under project TRA2007-67374-C02-02

Diseño. análisis y evaluación de conjuntos de clasificadores basados en redes de neuronas

Una de las áreas de investigación que, dentro del marco del Aprendizaje Automático, más atención ha recibido durante las últimas décadas ha sido el diseño de conjuntos de clasificadores. Bajo este denominador se engloban un gran número de algoritmos cuyo objetivo es la construcción de un clasificador robusto haciendo uso de clasificadores más simples denominados clasificadores base. Aunque el uso de los conjuntos de clasificadores se puede argumentar desde diversas perspectivas, la justificación más evidente se encuentra en el comportamiento humano. Antes de tomar una decisión importante es habitual pedir opinión a varios expertos para así tener mayor certeza de que la opción elegida es la más adecuada. Diversos estudios han demostrado que el éxito de cualquier conjunto de clasificadores viene determinado por la precisión y la diversidad de los clasificadores que lo integran. En otras palabras, para que un conjunto de clasificadores mejore la precisión de cualquiera de sus miembros se requiere que éstos sean precisos y diversos. Sin embargo, encontrar clasificadores base que, de forma simultánea, satisfagan ambos requisitos no es una tarea fácil. Por ello, en este trabajo se presentan dos nuevas arquitecturas de conjuntos de clasificadores en una de las cuales, sin obviar la diversidad, se fomenta la precisión de los clasificadores base, mientras que en la otra se fomenta la diversidad frente a la precisión. Las diferencias y la complementariedad existente entre ambas arquitecturas permitirá analizar la influencia que, en el comportamiento global del conjunto, tiene la primacía de una de estas particularidades frente a la otra. Aunque, en el mundo real, la mayor parte de los problemas de clasificación engloban a más de dos categorías, muchos de los conjuntos de clasificadores propuestos en la Bibliografía fueron originalmente concebidos para resolver problemas dicotómicos. En ocasiones, el algoritmo que rige el comportamiento de estos modelos puede extrapolarse a problemas multiclase. Sin embargo, en otros muchos casos, el problema multiclase sólo se puede resolver descomponiendo el problema original en subproblemas binarios. Además, la mayor parte de los modelos propuestos, han sido evaluados sobre dominios artificiales en los que el número de atributos con los que se describen los ejemplos es relativamente pequeño. A pesar de esta tendencia, existen un gran número de dominios reales en los que los ejemplos están descritos por cientos o incluso miles de características. La necesidad de disponer de nuevos métodos de clasificación capaces de resolver problemas reales marca uno de los objetivos de esta Tesis Doctoral. Así, las arquitecturas que se proponen en este trabajo han sido concebidas explícitamente para la resolución de problemas en los que el número de categorías es finito y superior a dos y en los que los ejemplos están descritos por un elevado número de atributos. Partiendo de estas dos singularidades, se pretende acotar, en la medida de lo posible, la complejidad y el coste computacional inherentes a la resolución de este tipo de problemas. La viabilidad de las arquitecturas propuestas se ha determinado experimentalmente. Así, el estudio realizado contempla un exhaustivo análisis en el que, sobre distintos dominios, se analiza el comportamiento de las arquitecturas propuestas y se compara con el logrado por algunos de los modelos de clasificación más referenciados en la Bibliografía. -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------The design of Ensemble of Classifiers has been one of the most active research areas in the field of Machine Learning for the last decades. In this area, many different algorithms have been proposed in order to create a more robust classifier that consists of simpler classifiers named base classifiers. Although the use of ensemble of classifiers can be justified by many different reasons, the most obvious justification is related with human decision making process. Before making a decision, it is common to ask several experts to be sure that the chosen option is the optimal. Many studies have demonstrated that the success of any ensemble of classifiers is related to the accuracy and diversity of the different base classifiers of the ensemble. In other words, an ensemble of classifiers could improve the accuracy of any of its individual members if they are accurate and diverse. However, obtaining base classifiers which satisfy both requirements simultaneously is not an easy task. For this reason, this work presents two new ensembles of classifiers: One of these ensembles prioritizes the accuracy of the base classifiers (taking also into account the diversity) and the other promotes diversity over accuracy. These ensembles are different but complement each other, so it will be possible to analyze the influence of these requirements over the global performance of the ensemble. The number of applications that require multiclass categorization is huge in the real world. However, many of the studies related to supervised learning are focused on the resolution of binary problems. Some machine learning algorithms can then be naturally extended to handle the multiclass case. For other algorithms, a direct extension to the multiclass case may be problematic. Typically, in such cases, the multiclass problem is reduced to multiple binary classification problems that can be solved separately. In addition, most of these models have been evaluated in artificial domains in which the number of features used to describe the examples is relatively small. Despite this, there are many real domains in which the examples are described by hundreds or even thousands of features. For this reason, one of the goals of this thesis is the creation of new classification methods for real world. Thus, the ensembles proposed in this work have been designed to be applicable to real domains in which each example is labeled with one of several categories and is described by a large number of features. Taking these characteristics into account, the computational complexity and cost of the proposed methods need to be reduced as much as possible. The viability of the proposed ensembles has been proved empirically. Thus, this thesis makes a comprehensive analysis in which, taking into account different domains, the performance of the proposed ensembles is analyzed and compared with other wellknown classification methods