Evolutionary synthesis of analog networks

The significant increase in the available computational power that took place in recent decades has been accompanied by a growing interest in the application of the evolutionary approach to the synthesis of many kinds of systems and, in particular, to the synthesis of systems like analog electronic circuits, neural networks, and, more generally, autonomous systems, for which no satisfying systematic and general design methodology has been found to date. Despite some interesting results in the evolutionary synthesis of these kinds of systems, the endowment of an artificial evolutionary process with the potential for an appreciable increase of complexity of the systems thus generated appears still an open issue. In this thesis the problem of the evolutionary growth of complexity is addressed taking as starting point the insights contained in the published material reporting the unfinished work done in the late 1940s and early 1950s by John von Neumann on the theory of self-reproducing automata. The evolutionary complexity-growth conditions suggested in that work are complemented here with a series of auxiliary conditions inspired by what has been discovered since then relatively to the structure of biological systems, with a particular emphasis on the workings of genetic regulatory networks seen as the most elementary, full-fledged level of organization of existing living organisms. In this perspective, the first chapter is devoted to the formulation of the problem of the evolutionary growth of complexity, going from the description of von Neumann's complexity-growth conditions to the specification of a set of auxiliary complexity-growth conditions derived from the analysis of the operation of genetic regulatory networks. This leads to the definition of a particular structure for the kind of systems that will be evolved and to the specification of the genetic representation for them. A system with the required structure — for which the name analog network is suggested — corresponds to a collection of devices whose terminals are connected by links characterized by a scalar value of interaction strength. One of the specificities of the evolutionary system defined in this thesis is the way these values of interaction strength are determined. This is done by associating with each device terminal of the evolving analog network a sequence of characters extracted from the sequences that constitute the genome representing the network, and by defining a map from pairs of sequences of characters to values of interaction strength. Whereas the first chapter gives general prescriptions for the definition of an evolutionary system endowed with the desired complexity-growth potential, the second chapter is devoted to the specification of all the details of an actual implementation of those prescriptions. In this chapter the structure of the genome and of the corresponding genetic operators are defined. A technique for the genetic encoding of the devices constituting the analog network is described, along with a way to implement the map that specifies the interaction between the devices of the evolved system, and between them and the devices constituting the external environment of the evolved system. The proposed implementation of the interaction map is based on the local alignment of sequences of characters. It is shown how the parameters defining the local alignment can be chosen, and what strategies can be adopted to prevent the proliferation of unwanted interactions. The third chapter is devoted to the application of the evolutionary system defined in the second chapter to problems aimed at assessing the suitability in an evolutionary context of the local alignment technique and to problems aimed at assessing the evolutionary potential of the complete evolutionary system when applied to the synthesis of analog networks. Finally, the fourth chapter briefly considers some further questions that are relevant to the proposed approach but could not be addressed in the context of this thesis. A series of appendixes is devoted to some complementary issues: the definition of a measure of diversity for an evolutionary population employing the genetic description introduced in this thesis; the choice of the quantizer for the values of interaction strength between the devices constituting the evolved analog network; the modifications required to use the analog electronic circuit simulator SPICE as a simulation engine for an evolutionary or an optimization process

A tutorial on (Bayesian) probability

The aim of this tutorial is to show that, when properly formulated, probability theory is simply the science of plausible reasoning, which permits us to represent exactly and to update our state of information about the world. This is nothing new, since in 1812 Laplace wrote “[L]a théorie de la probabilité n’est au fond que le bon sens réduit au calcul: elle fait apprécier avec exactitude, ce que les esprits justes sentent par une sorte d’instinct, sans qu’il puissent souvent s’en rendre compte”. Unfortunately, the historical development of the field has deprived us of half of probability theory. It is now time to bring it back from neglect in order to realize Laplace’s claim and give us an indispensable tool for our technical endeavors and scientific investigations

The Geometry of Time-Stepping

The space-time geometric structure of Maxwell’s equations is examined and a subset of them is found to define a pair of exact discrete time-stepping relations. The desirability of adopting an approach to the discretization of electromagnetic problems which exploits this fact is advocated, and the name topological time-stepping for numerical schemes complying with it is suggested. The analysis of the equations leading to this kind of time-stepping reveals that these equations are naturally written in terms of integrated field quantities associated with space-time domains. It is therefore suggested that these quantities be adopted as state variables within numerical methods. A list of supplementary prescriptions for a discretization of electromagnetic problems suiting this philosophy is given, with particular emphasis on the necessity to adopt a space-time approach in each discretization step. It is shown that some existing methods already comply with these tenets, but that this fact is not explicitly recognized and exploited. The role of the constitutive equations in this discretization philosophy is briefly analyzed. The extension of this approach to more general kinds of space-time meshes, to other sets of basic time-stepping equations and to other field theories is finally considered

Supervised Learning from the Bayesian Viewpoint: An informal overview

This is the second episode of the Bayesian saga started with the tutorial on the Bayesian probability. Its aim is showing in very informal terms how supervised learning can be interpreted from the Bayesian viewpoint. The focus is put on supervised learning of neural networks. The traditional approach to supervised neural network training is compared with the Bayesian perspective on supervised learning. A probabilistic interpretation is given to the traditional error function and to its minimization, to the phenomenon of overfitting and to the traditional countermeasures to prevent it. Finally, it is shown how the Bayesian approach solves the problem of assessing the performance of different network structures

Neuroevolution: from architectures to learning

Artificial neural networks (ANNs) are applied to many real-world problems, ranging from pattern classification to robot control. In order to design a neural network for a particular task, the choice of an architecture (including the choice of a neuron model), and the choice of a learning algorithm have to be addressed. Evolutionary search methods can provide an automatic solution to these problems. New insights in both neuroscience and evolutionary biology have led to the development of increasingly powerful neuroevolution techniques over the last decade. This paper gives an overview of the most prominent methods for evolving ANNs with a special focus on recent advances in the synthesis of learning architecture

Edge Elements and Cochain-Based Field Function Approximation

Edge Elements (EE) have received in recent times much attention from the Finite Element community. The present contribution analyzes the role played by EE in the preservation of the fundamental prop-erties of physical equations submitted to discretization. A short review of these properties is pre-sented, where it is emphasized the presence in the equations, of intrinsically discrete terms that can be represented in a most natural way using the concept of cochain. It is then shown how EE are instru-mental in the bridging of the gap represented by terms which cannot be exactly discretized. It is main-tained that the role of EE lies in their ability to provide a simple machinery to build a continuous rep-resentation (ideally, as a differential form) for a field starting from its discrete representation in terms of a cochain, an operation for which the name of cochain-based field function approximation is sug-gested. The interpolation practices of Finite Elements and Finite Volumes are considered under this light to clear away some confusion and show the way to further generalization

Method and device for the genetic representation and evolution of networks

A method for the genetic representation of a network (100), the network having one or more devices (20, 30, 70, 80), each device comprising at least one terminal (21, 22, 23; 71, 72) connected to at least one other terminal (21, 22, 23; 71, 72, 61) by a link with a value of interaction strength. The method includes associating with the devices terminal (21, 22,23; 71, 72) a first sequence of characters (121, 122, 123; 171,172), associating with the other terminal (21,22,23; 71,72, 61) a second sequence of characters (121, 122, 123; 171,172; 162), mapping at least part ofthe first sequence of characters (121, 122, 123; 171, 172) and at least part of the second sequence of characters (121, 122, 123; 171, 172; 161) to the value of interaction strength in order to determine the value of interaction strength