On the automated extraction of regression knowledge from databases

The advent of inexpensive, powerful computing systems, together with the increasing amount of available data, conforms one of the greatest challenges for next-century information science. Since it is apparent that much future analysis will be done automatically, a good deal of attention has been paid recently to the implementation of ideas and/or the adaptation of systems originally developed in machine learning and other computer science areas. This interest seems to stem from both the suspicion that traditional techniques are not well-suited for large-scale automation and the success of new algorithmic concepts in difficult optimization problems. In this paper, I discuss a number of issues concerning the automated extraction of regression knowledge from databases. By regression knowledge is meant quantitative knowledge about the relationship between a vector of predictors or independent variables (x) and a scalar response or dependent variable (y). A number of difficulties found in some well-known tools are pointed out, and a flexible framework avoiding many such difficulties is described and advocated. Basic features of a new tool pursuing this direction are reviewed

Born to learn: The inspiration, progress, and future of evolved plastic artificial neural networks

Biological plastic neural networks are systems of extraordinary computational capabilities shaped by evolution, development, and lifetime learning. The interplay of these elements leads to the emergence of adaptive behavior and intelligence. Inspired by such intricate natural phenomena, Evolved Plastic Artificial Neural Networks (EPANNs) use simulated evolution in-silico to breed plastic neural networks with a large variety of dynamics, architectures, and plasticity rules: these artificial systems are composed of inputs, outputs, and plastic components that change in response to experiences in an environment. These systems may autonomously discover novel adaptive algorithms, and lead to hypotheses on the emergence of biological adaptation. EPANNs have seen considerable progress over the last two decades. Current scientific and technological advances in artificial neural networks are now setting the conditions for radically new approaches and results. In particular, the limitations of hand-designed networks could be overcome by more flexible and innovative solutions. This paper brings together a variety of inspiring ideas that define the field of EPANNs. The main methods and results are reviewed. Finally, new opportunities and developments are presented

Concurrent evolution of feature extractors and modular artificial neural networks

Artificial Neural Networks (ANNs) are commonly used in both academia and industry as a solution to challenges in the pattern recognition domain. However, there are two problems that must be addressed before an ANN can be successfully applied to a given recognition task: ANN customization and data pre-processing. First, ANNs require customization for each specific application. Although the underlying mathematics of ANNs is well understood, customization based on theoretical analysis is impractical because of the complex interrelationship between ANN behavior and the problem domain. On the other hand, an empirical approach to the task of customization can be successful with the selection of an appropriate test domain. However, this latter approach is computationally intensive, especially due to the many variables that can be adjusted within the system. Additionally, it is subject to the limitations of the selected search algorithm used to find the optimal solution. Second, data pre-processing (feature extraction) is almost always necessary in order to organize and minimize the input data, thereby optimizing ANN performance. Not only is it difficult to know what and how many features to extract from the data, but it is also challenging to find the right balance between the computational requirements for the preprocessing algorithm versus the ANN itself. Furthermore, the task of developing an appropriate pre-processing algorithm usually requires expert knowledge of the problem domain, which may not always be available. This paper contends that the concurrent evolution of ANNs and data pre-processors allows the design of highly accurate recognition networks without the need for expert knowledge in the application domain. To this end, a novel method for evolving customized ANNs with correlated feature extractors was designed and tested. This method involves the use of concurrent evolutionary processes (CEPs) as a mechanism to search the space of recognition networks. In a series of controlled experiments the CEP was applied to the digit recognition domain to show that the efficacy of this method is in-line with results seen in other digit recognition research, but without the need for expert knowledge in image processing techniques for digit recognition

Automatic synthesis of fuzzy systems: An evolutionary overview with a genetic programming perspective

Studies in Evolutionary Fuzzy Systems (EFSs) began in the 90s and have experienced a fast development since then, with applications to areas such as pattern recognition, curve‐fitting and regression, forecasting and control. An EFS results from the combination of a Fuzzy Inference System (FIS) with an Evolutionary Algorithm (EA). This relationship can be established for multiple purposes: fine‐tuning of FIS's parameters, selection of fuzzy rules, learning a rule base or membership functions from scratch, and so forth. Each facet of this relationship creates a strand in the literature, as membership function fine‐tuning, fuzzy rule‐based learning, and so forth and the purpose here is to outline some of what has been done in each aspect. Special focus is given to Genetic Programming‐based EFSs by providing a taxonomy of the main architectures available, as well as by pointing out the gaps that still prevail in the literature. The concluding remarks address some further topics of current research and trends, such as interpretability analysis, multiobjective optimization, and synthesis of a FIS through Evolving methods

Self-Organized Specialization and Controlled Emergence in Organic Computing Systems

In this chapter we studied a first approach to generate suitable rule sets for solving classification problems on systems of autonomous, memory constrained components. It was shown that a multi agent system that uses interacting Pittsburgh-style classifier systems can evolve appropiate rule sets. The system evolves specialists for parts of the classification problem and cooperation between them. In this way the components overcome their restricted memory size and are able to solve the entire problem. It was shown that the communication topology between the components strongly influences the average number of components that a request has to pass until it is classified. It was also shown that the introduction of communication costs into the fitness function leads to a more even distribution of knowledge between the components and reduces the communication overhead without influencing the classification performance very much. If the system is used to generate rule sets to solve classification tasks on real hardware systems, communication cost in the training phase can thus lead to a better knowledge distribution and small communication cost. That is, in this way the system will be more robust against the loss of single components and longer reliable in case of limited energy resources