Optimization of Evolutionary Neural Networks Using Hybrid Learning Algorithms

Evolutionary artificial neural networks (EANNs) refer to a special class of artificial neural networks (ANNs) in which evolution is another fundamental form of adaptation in addition to learning. Evolutionary algorithms are used to adapt the connection weights, network architecture and learning algorithms according to the problem environment. Even though evolutionary algorithms are well known as efficient global search algorithms, very often they miss the best local solutions in the complex solution space. In this paper, we propose a hybrid meta-heuristic learning approach combining evolutionary learning and local search methods (using 1st and 2nd order error information) to improve the learning and faster convergence obtained using a direct evolutionary approach. The proposed technique is tested on three different chaotic time series and the test results are compared with some popular neuro-fuzzy systems and a recently developed cutting angle method of global optimization. Empirical results reveal that the proposed technique is efficient in spite of the computational complexity

Evidence of coevolution in multi-objective evolutionary algorithms

This paper demonstrates that simple yet important characteristics of coevolution can occur in evolutionary algorithms when only a few conditions are met. We find that interaction-based fitness measurements such as fitness (linear) ranking allow for a form of coevolutionary dynamics that is observed when 1) changes are made in what solutions are able to interact during the ranking process and 2) evolution takes place in a multi-objective environment. This research contributes to the study of simulated evolution in a at least two ways. First, it establishes a broader relationship between coevolution and multi-objective optimization than has been previously considered in the literature. Second, it demonstrates that the preconditions for coevolutionary behavior are weaker than previously thought. In particular, our model indicates that direct cooperation or competition between species is not required for coevolution to take place. Moreover, our experiments provide evidence that environmental perturbations can drive coevolutionary processes; a conclusion that mirrors arguments put forth in dual phase evolution theory. In the discussion, we briefly consider how our results may shed light onto this and other recent theories of evolution

SCANN: Synthesis of Compact and Accurate Neural Networks

Deep neural networks (DNNs) have become the driving force behind recent artificial intelligence (AI) research. An important problem with implementing a neural network is the design of its architecture. Typically, such an architecture is obtained manually by exploring its hyperparameter space and kept fixed during training. This approach is time-consuming and inefficient. Another issue is that modern neural networks often contain millions of parameters, whereas many applications and devices require small inference models. However, efforts to migrate DNNs to such devices typically entail a significant loss of classification accuracy. To address these challenges, we propose a two-step neural network synthesis methodology, called DR+SCANN, that combines two complementary approaches to design compact and accurate DNNs. At the core of our framework is the SCANN methodology that uses three basic architecture-changing operations, namely connection growth, neuron growth, and connection pruning, to synthesize feed-forward architectures with arbitrary structure. SCANN encapsulates three synthesis methodologies that apply a repeated grow-and-prune paradigm to three architectural starting points. DR+SCANN combines the SCANN methodology with dataset dimensionality reduction to alleviate the curse of dimensionality. We demonstrate the efficacy of SCANN and DR+SCANN on various image and non-image datasets. We evaluate SCANN on MNIST and ImageNet benchmarks. In addition, we also evaluate the efficacy of using dimensionality reduction alongside SCANN (DR+SCANN) on nine small to medium-size datasets. We also show that our synthesis methodology yields neural networks that are much better at navigating the accuracy vs. energy efficiency space. This would enable neural network-based inference even on Internet-of-Things sensors.Comment: 13 pages, 8 figure

Survival of the Fittest and Zero Sum Games

Competition for available resources is natural amongst coexisting species, and the fittest contenders dominate over the rest in evolution. The dynamics of this selection is studied using a simple linear model. It has similarities to features of quantum computation, in particular conservation laws leading to destructive interference. Compared to an altruistic scenario, competition introduces instability and eliminates the weaker species in a finite time.Comment: 6 pages, formatted according to journal style. Special Issue on Game Theory and Evolutionary Processes. (v2) Published version. Some clarifications added. Topological interpretation pointed ou

Schumpeterian economic dynamics as a quantifiable minimum model of evolution

We propose a simple quantitative model of Schumpeterian economic dynamics. New goods and services are endogenously produced through combinations of existing goods. As soon as new goods enter the market they may compete against already existing goods, in other words new products can have destructive effects on existing goods. As a result of this competition mechanism existing goods may be driven out from the market - often causing cascades of secondary defects (Schumpeterian gales of destruction). The model leads to a generic dynamics characterized by phases of relative economic stability followed by phases of massive restructuring of markets - which could be interpreted as Schumpeterian business `cycles'. Model timeseries of product diversity and productivity reproduce several stylized facts of economics timeseries on long timescales such as GDP or business failures, including non-Gaussian fat tailed distributions, volatility clustering etc. The model is phrased in an open, non-equilibrium setup which can be understood as a self organized critical system. Its diversity dynamics can be understood by the time-varying topology of the active production networks.Comment: 21 pages, 11 figure