Optimizing Neural Architecture Search using Limited GPU Time in a Dynamic Search Space: A Gene Expression Programming Approach

Efficient identification of people and objects, segmentation of regions of interest and extraction of relevant data in images, texts, audios and videos are evolving considerably in these past years, which deep learning methods, combined with recent improvements in computational resources, contributed greatly for this achievement. Although its outstanding potential, development of efficient architectures and modules requires expert knowledge and amount of resource time available. In this paper, we propose an evolutionary-based neural architecture search approach for efficient discovery of convolutional models in a dynamic search space, within only 24 GPU hours. With its efficient search environment and phenotype representation, Gene Expression Programming is adapted for network's cell generation. Despite having limited GPU resource time and broad search space, our proposal achieved similar state-of-the-art to manually-designed convolutional networks and also NAS-generated ones, even beating similar constrained evolutionary-based NAS works. The best cells in different runs achieved stable results, with a mean error of 2.82% in CIFAR-10 dataset (which the best model achieved an error of 2.67%) and 18.83% for CIFAR-100 (best model with 18.16%). For ImageNet in the mobile setting, our best model achieved top-1 and top-5 errors of 29.51% and 10.37%, respectively. Although evolutionary-based NAS works were reported to require a considerable amount of GPU time for architecture search, our approach obtained promising results in little time, encouraging further experiments in evolutionary-based NAS, for search and network representation improvements.Comment: Accepted for presentation at the IEEE Congress on Evolutionary Computation (IEEE CEC) 202

Adaptive Evolutionary Multitasking to Solve Inter-Domain Path Computation Under Node-Defined Domain Uniqueness Constraint: New Solution Encoding Scheme

In multi-domain networks, the efficiency of path computation becomes more and more important. The Inter-Domain Path Computation under Node-defined Domain Uniqueness Constraint (IDPC-NDU) is a recently investigated problem where its objective is to determine the effective routing path between two nodes that traverses every domain at most once. IDPC-NDU is NP-Hard, so the approximation approaches are suitable to deal with this problem for large instances. Multifactorial Evolutionary Algorithm (MFEA) is an emerging research topic in the field of evolutionary computation that can efficiently tackle multiple optimization problems at the same time. This study proposed an approach based on the combination of the Adaptive Multifactorial Evolutionary Algorithm (dMFEA-II) and Dijkstra algorithm for solving IDPC-NDU. The encoding and evaluating methods based on the permutation representation are also introduced, and the new individual representation is always to produce valid solutions. The proposed algorithm is evaluated on two types of instances. Simulation results demonstrate the superior performance of the proposed algorithm in comparison with the existing algorithms in terms of the quality of the solution

Inevitable Evolutionary Temporal Elements in Neural Processing: A Study Based on Evolutionary Simulations

Recent studies have suggested that some neural computational mechanisms are based on the fine temporal structure of spiking activity. However, less effort has been devoted to investigating the evolutionary aspects of such mechanisms. In this paper we explore the issue of temporal neural computation from an evolutionary point of view, using a genetic simulation of the evolutionary development of neural systems. We evolve neural systems in an environment with selective pressure based on mate finding, and examine the temporal aspects of the evolved systems. In repeating evolutionary sessions, there was a significant increase during evolution in the mutual information between the evolved agent's temporal neural representation and the external environment. In ten different simulated evolutionary sessions, there was an increased effect of time -related neural ablations on the agents' fitness. These results suggest that in some fitness landscapes the emergence of temporal elements in neural computation is almost inevitable. Future research using similar evolutionary simulations may shed new light on various biological mechanisms

Exact computation of the expectation curves of the bit-flip mutation using landscapes theory

Chicano, F., & Alba E. (2011). Exact computation of the expectation curves of the bit-flip mutation using landscapes theory. Proceedings of 13th Annual Genetic and Evolutionary Computation Conference, Dublin, Ireland, July 12-16, 2011. pp. 2027–2034.Bit-flip mutation is a common operation when a genetic algorithm is applied to solve a problem with binary representation. We use in this paper some results of landscapes theory and Krawtchouk polynomials to exactly compute the expected value of the fitness of a mutated solution. We prove that this expectation is a polynomial in p, the probability of flipping a single bit. We analyze these polynomials and propose some applications of the obtained theoretical results.Universidad de Málaga. Campus de Excelencia Internacional Andalucía Tech. This research has been partially funded by the Spanish Ministry of Science and Innovation and FEDER under contract TIN2008-06491-C04-01 (the M∗ project) and the Andalusian Government under contract P07-TIC-03044 (DIRICOM project)

Evolutionary cellular configurations for designing feed-forward neural networks architectures

Proceeding of: 6th International Work-Conference on Artificial and Natural Neural Networks, IWANN 2001 Granada, Spain, June 13–15, 2001In the recent years, the interest to develop automatic methods to determine appropriate architectures of feed-forward neural networks has increased. Most of the methods are based on evolutionary computation paradigms. Some of the designed methods are based on direct representations of the parameters of the network. These representations do not allow scalability, so to represent large architectures, very large structures are required. An alternative more interesting are the indirect schemes. They codify a compact representation of the neural network. In this work, an indirect constructive encoding scheme is presented. This scheme is based on cellular automata representations in order to increase the scalability of the method