Reservoir Computing in Materio

Reservoir Computing first emerged as an efficient mechanism for training recurrent neural networks and later evolved into a general theoretical model for dynamical systems. By applying only a simple training mechanism many physical systems have become exploitable unconventional computers. However, at present, many of these systems require careful selection and tuning by hand to produce usable or optimal reservoir computers. In this thesis we show the first steps to applying the reservoir model as a simple computational layer to extract exploitable information from complex material substrates. We argue that many physical substrates, even systems that in their natural state might not form usable or "good" reservoirs, can be configured into working reservoirs given some stimulation. To achieve this we apply techniques from evolution in materio whereby configuration is through evolved input-output signal mappings and targeted stimuli. In preliminary experiments the combined model and configuration method is applied to carbon nanotube/polymer composites. The results show substrates can be configured and trained as reservoir computers of varying quality. It is shown that applying the reservoir model adds greater functionality and programmability to physical substrates, without sacrificing performance. Next, the weaknesses of the technique are addressed, with the creation of new high input-output hardware system and an alternative multi-substrate framework. Lastly, a substantial effort is put into characterising the quality of a substrate for reservoir computing, i.e its ability to realise many reservoirs. From this, a methodological framework is devised. Using the framework, radically different computing substrates are compared and assessed, something previously not possible. As a result, a new understanding of the relationships between substrate, tasks and properties is possible, outlining the way for future exploration and optimisation of new computing substrates

Digital control networks for virtual creatures

Robot control systems evolved with genetic algorithms traditionally take the form of floating-point neural network models. This thesis proposes that digital control systems, such as quantised neural networks and logical networks, may also be used for the task of robot control. The inspiration for this is the observation that the dynamics of discrete networks may contain cyclic attractors which generate rhythmic behaviour, and that rhythmic behaviour underlies the central pattern generators which drive lowlevel motor activity in the biological world. To investigate this a series of experiments were carried out in a simulated physically realistic 3D world. The performance of evolved controllers was evaluated on two well known control tasks—pole balancing, and locomotion of evolved morphologies. The performance of evolved digital controllers was compared to evolved floating-point neural networks. The results show that the digital implementations are competitive with floating-point designs on both of the benchmark problems. In addition, the first reported evolution from scratch of a biped walker is presented, demonstrating that when all parameters are left open to evolutionary optimisation complex behaviour can result from simple components

Using MapReduce Streaming for Distributed Life Simulation on the Cloud

Distributed software simulations are indispensable in the study of large-scale life models but often require the use of technically complex lower-level distributed computing frameworks, such as MPI. We propose to overcome the complexity challenge by applying the emerging MapReduce (MR) model to distributed life simulations and by running such simulations on the cloud. Technically, we design optimized MR streaming algorithms for discrete and continuous versions of Conway’s life according to a general MR streaming pattern. We chose life because it is simple enough as a testbed for MR’s applicability to a-life simulations and general enough to make our results applicable to various lattice-based a-life models. We implement and empirically evaluate our algorithms’ performance on Amazon’s Elastic MR cloud. Our experiments demonstrate that a single MR optimization technique called strip partitioning can reduce the execution time of continuous life simulations by 64%. To the best of our knowledge, we are the first to propose and evaluate MR streaming algorithms for lattice-based simulations. Our algorithms can serve as prototypes in the development of novel MR simulation algorithms for large-scale lattice-based a-life models.https://digitalcommons.chapman.edu/scs_books/1014/thumbnail.jp

Adaptive networks for robotics and the emergence of reward anticipatory circuits

Currently the central challenge facing evolutionary robotics is to determine how best to extend the range and complexity of behaviour supported by evolved neural systems. Implicit in the work described in this thesis is the idea that this might best be achieved through devising neural circuits (tractable to evolutionary exploration) that exhibit complementary functional characteristics. We concentrate on two problem domains; locomotion and sequence learning. For locomotion we compare the use of GasNets and other adaptive networks. For sequence learning we introduce a novel connectionist model inspired by the role of dopamine in the basal ganglia (commonly interpreted as a form of reinforcement learning). This connectionist approach relies upon a new neuron model inspired by notions of energy efficient signalling. Two reward adaptive circuit variants were investigated. These were applied respectively to two learning problems; where action sequences are required to take place in a strict order, and secondly, where action sequences are robust to intermediate arbitrary states. We conclude the thesis by proposing a formal model of functional integration, encompassing locomotion and sequence learning, extending ideas proposed by W. Ross Ashby. A general model of the adaptive replicator is presented, incoporating subsystems that are tuned to continuous variation and discrete or conditional events. Comparisons are made with Ross W. Ashby's model of ultrastability and his ideas on adaptive behaviour. This model is intended to support our assertion that, GasNets (and similar networks) and reward adaptive circuits of the type presented here, are intrinsically complementary. In conclusion we present some ideas on how the co-evolution of GasNet and reward adaptive circuits might lead us to significant improvements in the synthesis of agents capable of exhibiting complex adaptive behaviour

Evolving hierarchical visually guided neural network agents to investigate complex interactions.

A complex system is a system with a large number of interacting components without any mechanism for central control that displays self organisation. Understanding how these interactions affect the overall behaviour of a system is of great interest to science. Indeed, researchers use a wide variety of models to investigate complex systems. The problem with most models is that they disregard the hierarchical nature of complex systems: they ignore the fact that components of real world systems tend to be complex systems as well. This prevents researchers from investigating the interactions taking place between the lower and the higher levels of the model which may be crucial in order to gain a full understanding of the examined phenomena and of complex systems in general. Therefore, this thesis introduces Mosaic World, a multi-agent model for the purpose of investigating interactions (focusing on 'complex' multilevel interactions) within a hierarchical complex system, in addition to other computational and biological hypotheses. Mosaic World comprises a population of evolving neural network agents that inhabit a changing visual environment. By analysing the interactions that occur within Mosaic World, this thesis demonstrates the importance of incorporating hierarchical complexity into a model, and contributes to our understanding of hierarchical complex systems by showing how selective pressures cause differentiation across levels. Additionally, the study of multilevel interactions is used to probe several hypotheses and provides the following contributions among others: Analysis of agent evolvability as affected by the usage of different types of structural mutations in the evolutionary process. Demonstration that agents controlled by modular neural networks are fitter than agents that are controlled by non-modular neural networks the improvement in fitness occurs through specialisation of modules. Empirical support for a biological theory suggesting that colour vision evolved as a method of dealing with ambiguous stimuli

Suuremahuliste andmete kasutamine geenidevaheliste seoste leidmiseks

Väitekirja elektrooniline versioon ei sisalda publikatsioone.Geenid määravad ära, millistest RNA ja valgu molekulidest elusorganism koosneb. Ainult geenide tuvastamisest ei piisa, et aru saada kuidas organism toimib, millal ja kuidas erinevad geenide produktid avalduvad ja mida need teevad. Elusorganismi olemuse mõistmiseks ja bioloogiliste protsesside mõjutamiseks on vajalik aru saada geenide ja valkude omavahelistest seostest. Suure läbilaskevõimega tehnoloogiad võimaldavad hõlpsasti mõõta bioloogiliste protsesside erinevaid tahke. See omakorda on toonud kaasa andmemahtude üha kiireneva kasvutrendi ning vajaduse uute meetodite järele, mis aitaks toorandmeid analüüsida, andmeid omavahel kombineerida ning tulemusi visualiseerida. Samuti on kasvanud vajadus arvutuslike meetoditega katsetada, kas olemasolevad andmemudelid kirjeldavad bioloogilist uurimisobjekti piisavalt täpselt. Käesolevas uurimistöös on näidatud erinevaid bioinformaatilisi meetodeid, kuidas suuremahuliste ning eritüübiliste eksperimentaalsete andmete kombineerimist saab rakendada geenidevaheliste seoste leidmiseks. Suuremahulistele andmetele on integreerimise ja omavahel võrreldavaks tegemisega võimalik anda lisaväärtust. Töö käigus koondati kokku ja tehti avalikkusele ligipääsetavaks embrüonaalsete tüvirakkude regulatsiooni käsitlevate publikatsioonide lisafailides avaldatud info ESCDb andmebaasi näol. Neid andmeid kasutades on teadlaskonnal võimalik leida geenide vahelisi seoseid, mida eraldiseisvaid andmeid analüüsides ei ole võimalik välja selgitada. Andmebaasi kogutud info kombineerimisel arvutusliku mudeldamisega õnnestus leida käesoleva töö raames uus regulaator embrüonaalsetes tüvirakkudes — IL11. Lisaks võimaldas erinevate andmetüüpide kombineerimine leida embrüonaalsete tüvirakkude keskse regulaatori — OCT4 geeni alternatiivsed märklaudgeenide moodulid. Kasutades DNA konserveerumisinfot koos regulatoorsete motiivide analüüsiga leiti kolm uut rasvatüvirakkude diferentseerumise regulaatorvalku. Samuti käsitletakse töös automaatset grupeerimis- ja visualiseerimismetoodikat VisHiC, mis aitab esile tõsta huvitavaid geenigruppe, mida teiste meetoditega edasi uurida. Töös on näidatud erinevaid suuremahuliste andmestike integreerimise viise, mis võimaldavad leida selliseid geenidevahelisi seoseid, mida ei oleks võimalik leida kui analüüsiksime üht andmestikku korraga.In order to understand the basic principles of how organisms function, and to be able to affect the biological processes, we need to understand relationships between genes and proteins. Modern high-throughput technology enables to study different sides of biological processes in a rapid manner. This, however, has led to a steady growth of amount of data available. The need for more sophisticated methods for analysing raw data, for combining different data sources, and to visualise the results, has emerged. Additionally, computational modeling is required to test if our understanding of biological processes is supported by the available data. A variety of bioinformatics methods are used to demonstrate how to combine different type of high-throughput data for identifying relationships between genes. Furthermore, it was shown that through combining various data types from different sources adds value to already published data. In the thesis, data from publications about embryonic stem cell regulation were collected together and made available through Embryonic Stem Cell Database (ESCDb). Complementary data in the database allows researchers to find relationships between genes that would not be possible when analysing only one dataset at a time. One of the main findings of this study illustrates how using computational modelling on data from the ESCDb allowed to find a novel pluripotency regulator — IL11. Additionally, integration of different data types led to identification of alternative gene regulatory modules of core pluripotency regulator OCT4. Similarly, combination of conservation data and regulatory motif analysis led to identification of three new regulators of adipocyte differentiation. This thesis also covers innovative methodology, VisHiC, for automatic identification and visualisation of functionally related gene sets. This methodology allows to find relevant gene sets for further characterisation from large high-throughput datasets. This doctoral thesis demonstrates that integration of different high-throughput datasets enables establishing gene-gene relationships that would not be possible when looking at a single data type in isolation

A Systems Approach to Cellular Signal Transduction

Vital cellular processes such as growth, gene expression, and homeostasis depend on the correct transmission of molecular signals within and between cells. The vast complexity of these molecular signaling networks has necessitated the use of mathematical methods to model, characterize, and predict cellular responses. The work presented in this dissertation shows how computational methods were used to elucidate two clinically-relevant cellular signaling responses: (i) phosphotyrosine signaling through the epidermal growth factor receptor (EGFR), a receptor tyrosine kinase that is commonly overexpressed or structurally altered in human cancers; and (ii) phosphoinositide and calcium signaling in human platelets---the key cellular mediators of hemostasis and pathological thrombus formation. The kinetic model of EGFR-mediated signaling in wild-type and mutant cells showed how mutant forms of the receptor use an irregular pattern of tyrosine phosphorylation that preferentially activates the survival oncoprotein, Akt. By quantifying the amount of signal flow through diverging pathways downstream of the receptor, our calculations provided a mechanistic explanation for the clinical observation that therapeutic tyrosine kinase inhibitors can control tumor growth in cells bearing certain EGFR mutations. In the second major study, a kinetic model of ADP-stimulated calcium release in human platelets was used to make precise, quantitative predictions about the molecular makeup and structural properties of the platelet. Specifically, we found that the resting structure of platelets places strong restrictions on several biophysical quantities, such as the resting concentration of free inositol 1,4,5-trisphosphate, the ratio of calcium ATPase pumps to release channels, and the size of the calcium storage compartment. Notably, the model also demonstrated that the irregular calcium spiking behavior observed in single ADP-stimulated platelets is due to the extremely small cellular volume. A novel method for constructing kinetic signaling networks, based on restricting the steady-state properties of the model, is also presented. Future applications and extensions of the systems approach to signal transduction modeling are discussed in the final chapter

Evolutionary robotics in high altitude wind energy applications

Recent years have seen the development of wind energy conversion systems that can exploit the superior wind resource that exists at altitudes above current wind turbine technology. One class of these systems incorporates a flying wing tethered to the ground which drives a winch at ground level. The wings often resemble sports kites, being composed of a combination of fabric and stiffening elements. Such wings are subject to load dependent deformation which makes them particularly difficult to model and control. Here we apply the techniques of evolutionary robotics i.e. evolution of neural network controllers using genetic algorithms, to the task of controlling a steerable kite. We introduce a multibody kite simulation that is used in an evolutionary process in which the kite is subject to deformation. We demonstrate how discrete time recurrent neural networks that are evolved to maximise line tension fly the kite in repeated looping trajectories similar to those seen using other methods. We show that these controllers are robust to limited environmental variation but show poor generalisation and occasional failure even after extended evolution. We show that continuous time recurrent neural networks (CTRNNs) can be evolved that are capable of flying appropriate repeated trajectories even when the length of the flying lines are changing. We also show that CTRNNs can be evolved that stabilise kites with a wide range of physical attributes at a given position in the sky, and systematically add noise to the simulated task in order to maximise the transferability of the behaviour to a real world system. We demonstrate how the difficulty of the task must be increased during the evolutionary process to deal with this extreme variability in small increments. We describe the development of a real world testing platform on which the evolved neurocontrollers can be tested

Efficient Evolution of Neural Networks

This thesis addresses the study of evolutionary methods for the synthesis of neural network controllers. Chapter 1 introduces the research area, reviews the state of the art, discusses promising research directions, and presents the two major scientific objectives of the thesis. The first objective, which is covered in Chapter 2, is to verify the efficacy of some of the most promising neuro-evolutionary methods proposed in the literature, including two new methods that I elaborated. This has been made by designing extended version of the double-pole balancing problem, which can be used to more properly benchmark alternative algorithms, by studying the effect of critical parameters, and by conducting several series of comparative experiments. The obtained results indicate that some methods perform better with respect to all the considered criteria, i.e. performance, robustness to environmental variations and capability to scale-up to more complex problems. The second objective, which is targeted in Chapter 3, consists in the design of a new hybrid algorithm that combines evolution and learning by demonstration. The combination of these two processes is appealing since it potentially allows the adaptive agent to exploit a richer training feedback constituted by both a scalar performance objective (reinforcement signal or fitness measure) and a detailed description of a suitable behaviour (demonstration). The proposed method has been successfully evaluated on two qualitatively different robotic problems. Chapter 4 summarizes the results obtained and describes the major contributions of the thesis