A Genetic Programming Approach to Designing Convolutional Neural Network Architectures

The convolutional neural network (CNN), which is one of the deep learning models, has seen much success in a variety of computer vision tasks. However, designing CNN architectures still requires expert knowledge and a lot of trial and error. In this paper, we attempt to automatically construct CNN architectures for an image classification task based on Cartesian genetic programming (CGP). In our method, we adopt highly functional modules, such as convolutional blocks and tensor concatenation, as the node functions in CGP. The CNN structure and connectivity represented by the CGP encoding method are optimized to maximize the validation accuracy. To evaluate the proposed method, we constructed a CNN architecture for the image classification task with the CIFAR-10 dataset. The experimental result shows that the proposed method can be used to automatically find the competitive CNN architecture compared with state-of-the-art models.Comment: This is the revised version of the GECCO 2017 paper. The code of our method is available at https://github.com/sg-nm/cgp-cn

A Boltzmann machine for the organization of intelligent machines

In the present technological society, there is a major need to build machines that would execute intelligent tasks operating in uncertain environments with minimum interaction with a human operator. Although some designers have built smart robots, utilizing heuristic ideas, there is no systematic approach to design such machines in an engineering manner. Recently, cross-disciplinary research from the fields of computers, systems AI and information theory has served to set the foundations of the emerging area of the design of intelligent machines. Since 1977 Saridis has been developing an approach, defined as Hierarchical Intelligent Control, designed to organize, coordinate and execute anthropomorphic tasks by a machine with minimum interaction with a human operator. This approach utilizes analytical (probabilistic) models to describe and control the various functions of the intelligent machine structured by the intuitively defined principle of Increasing Precision with Decreasing Intelligence (IPDI) (Saridis 1979). This principle, even though resembles the managerial structure of organizational systems (Levis 1988), has been derived on an analytic basis by Saridis (1988). The purpose is to derive analytically a Boltzmann machine suitable for optimal connection of nodes in a neural net (Fahlman, Hinton, Sejnowski, 1985). Then this machine will serve to search for the optimal design of the organization level of an intelligent machine. In order to accomplish this, some mathematical theory of the intelligent machines will be first outlined. Then some definitions of the variables associated with the principle, like machine intelligence, machine knowledge, and precision will be made (Saridis, Valavanis 1988). Then a procedure to establish the Boltzmann machine on an analytic basis will be presented and illustrated by an example in designing the organization level of an Intelligent Machine. A new search technique, the Modified Genetic Algorithm, is presented and proved to converge to the minimum of a cost function. Finally, simulations will show the effectiveness of a variety of search techniques for the intelligent machine

The Case for a Mixed-Initiative Collaborative Neuroevolution Approach

It is clear that the current attempts at using algorithms to create artificial neural networks have had mixed success at best when it comes to creating large networks and/or complex behavior. This should not be unexpected, as creating an artificial brain is essentially a design problem. Human design ingenuity still surpasses computational design for most tasks in most domains, including architecture, game design, and authoring literary fiction. This leads us to ask which the best way is to combine human and machine design capacities when it comes to designing artificial brains. Both of them have their strengths and weaknesses; for example, humans are much too slow to manually specify thousands of neurons, let alone the billions of neurons that go into a human brain, but on the other hand they can rely on a vast repository of common-sense understanding and design heuristics that can help them perform a much better guided search in design space than an algorithm. Therefore, in this paper we argue for a mixed-initiative approach for collaborative online brain building and present first results towards this goal.Comment: Presented at WebAL-1: Workshop on Artificial Life and the Web 2014 (arXiv:1406.2507

Improving resiliency using graph based evolutionary algorithms

Resiliency is an important characteristic of any system. It signifies the ability of a system to survive and recover from unprecedented disruptions. Various characteristics exist that indicate the level of resiliency in a system. One of these attributes is the adaptability of the system. This adaptability can be enhanced by redundancy present within the system. In the context of system design, redundancy can be achieved by having a diverse set of good designs for that particular system. Evolutionary algorithms are widely used in creating designs for engineering systems, as they perform well on discontinuous and/or high dimensional problems. One method to control the diversity of solutions within an evolutionary algorithm is the use of combinatorial graphs, or graph based evolutionary algorithms. This diversity of solutions is key factor to enhance the redundancy of a system design. In this work, the way how graph based evolutionary algorithms generate diverse solutions is investigated by examining the influence of representation and mutation. This allows for greater understanding of the exploratory nature of each representation and how they can control the number of solution generated within a trial. The results of this research are then applied to the Travelling [sic] Salesman Problem, a known NP hard problem often used as a surrogate for logistic or network design problems. When the redundancy in system design is improved, adaptability can be achieved by placing an agent to initiate a transfer to other good solutions in the event of a disruption in network connectivity, making it possible to improve the resiliency of the system --Abstract, page iii

Metaheuristic Approaches to Solve a Complex Aircraft Performance Optimization Problem

The increasing demands for travelling comfort and reduction of carbon dioxide emissions have been considered substantially in the stage of conceptual aircraft design. However, the design of a modern aircraft is a multidisciplinary process, which requires the coordination of information from several specific disciplines, such as structures, aerodynamics, control, etc. To address this problem with adequate accuracy, the multidisciplinary analysis and optimization (MAO) method is usually applied as a systematic and robust approach to solve such complex design issues arising from industries. Since MAO method is tedious and computationally expensive, genetic programming (GP)-based metamodeling techniques incorporating MAO are proposed as an effective approach to minimize the wing stiffness of a large aircraft subject to aerodynamic, aeroelastic and stability constraints in the conceptual design phase. Based on the linear small-disturbance theory, the state-space equation is employed for stability analysis. In the process of multidisciplinary analysis, aeroelastic response simulations are performed using Nastran. To construct metamodels representing the responses of the interests with high accuracy as well as less computational burden, optimal Latin hypercube design of experiments (DoE) is applied to determine the optimized distribution of sampling points. Following that, parametric optimization is carried out on metamodels to obtain the optimal wing geometry shape, elastic axis positions and stiffness distribution, and then the solution is verified by finite element simulations. Finally, the superiority of the GP-based metamodel technique over genetic algorithm is demonstrated by multidisciplinary design optimization of a representative beam-frame wing structure in terms of accuracy and efficiency. The results also show that GP metamodel-based strategy for solving MAO problems can provide valuable insights to tailoring parameters for the effective design of a large aircraft in the conceptual phase