A Survey of Robotics Control Based on Learning-Inspired Spiking Neural Networks

Biological intelligence processes information using impulses or spikes, which makes those living creatures able to perceive and act in the real world exceptionally well and outperform state-of-the-art robots in almost every aspect of life. To make up the deficit, emerging hardware technologies and software knowledge in the fields of neuroscience, electronics, and computer science have made it possible to design biologically realistic robots controlled by spiking neural networks (SNNs), inspired by the mechanism of brains. However, a comprehensive review on controlling robots based on SNNs is still missing. In this paper, we survey the developments of the past decade in the field of spiking neural networks for control tasks, with particular focus on the fast emerging robotics-related applications. We first highlight the primary impetuses of SNN-based robotics tasks in terms of speed, energy efficiency, and computation capabilities. We then classify those SNN-based robotic applications according to different learning rules and explicate those learning rules with their corresponding robotic applications. We also briefly present some existing platforms that offer an interaction between SNNs and robotics simulations for exploration and exploitation. Finally, we conclude our survey with a forecast of future challenges and some associated potential research topics in terms of controlling robots based on SNNs

Neuromorphic Computing for Interactive Robotics: A Systematic Review

Modelling functionalities of the brain in human-robot interaction contexts requires a real-time understanding of how each part of a robot (motors, sensors, emotions, etc.) works and how they interact all together to accomplish complex behavioural tasks while interacting with the environment. Human brains are very efficient as they process the information using event-based impulses also known as spikes, which make living creatures very efficient and able to outperform current mainstream robotic systems in almost every task that requires real-time interaction. In recent years, combined efforts by neuroscientists, biologists, computer scientists and engineers make it possible to design biologically realistic hardware and models that can endow the robots with the required human-like processing capability based on neuromorphic computing and Spiking Neural Network (SNN). However, while some attempts have been made, a comprehensive combination of neuromorphic computing and robotics is still missing. In this article, we present a systematic review of neuromorphic computing applications for socially interactive robotics.We first introduce the basic principles, models and architectures of neuromorphic computation. The remaining articles are classified according to the applications they focus on. Finally, we identify the potential research topics for fully integrated socially interactive neuromorphic robots

Intelligence artificielle et robotique bio-inspirée : modélisation de fonctions d'apprentissage par réseaux de neurones à impulsions

Cette thèse a comme objectif de permettre une avancée originale dans le domaine de l'informatique cognitive, plus précisément en robotique bio-inspirée. L'hypothèse défendue est qu'il est possible d'intégrer différentes fonctions d'apprentissage, élaborées et incarnées pour des robots virtuels et physiques, à un même paradigme de réseaux de neurones à impulsions agissant comme cerveaux-contrôleurs. La conception de règles d'apprentissage et la validation de l'hypothèse de recherche reposent sur la simulation de mécanismes cellulaires à base de plasticité synaptique et sur la reproduction de comportements adaptatifs des robots. Cette thèse par articles cible trois types d'apprentissage de complexité incrémentale : l'habituation comme forme d'apprentissage non associatif et les conditionnements classiques et opérants comme formes d'apprentissage associatif. L'analyse détaillée, de la synapse au comportement, et validée par des études expérimentales provenant d'invertébrés tels que le ver nématode Caenorhabditis elegans. Pour chacune de ces règles, un algorithme novateur a été proposé, conduisant à la publication d'un article scientifique. Ces règles d'apprentissage ont été modélisées en développant certains paramètres temporels et des circuits neuronaux précis. Incidemment, la granularité du temps des réseaux de neurones à impulsions (RNAI) est établie au niveau du simple potentiel d'action plutôt qu'au niveau du taux moyen de décharge par unité de temps, comme c'est le cas pour les réseaux de neurones artificiels traditionnels. Cette propriété des RNAI s'est avérée être un atout suffisant pour préférer leur utilisation pour des robots évoluant dans le monde réel. L'élaboration du modèle computationnel d'apprentissage pour des robots a requis de tester d'abord les hypothèses sur des simulations virtuelles. Puisqu'aucun simulateur n'avait les capacités suffisantes pour tester notre hypothèse, soit d'intégrer des RNAI, des structures de robots, et des interfaces pour l'exportation des RNAI vers des plateformes physiques et des environnements virtuels 3D suffisamment complexes, il a été nécessaire de développer, en parallèle de la thèse, un logiciel novateur (SIMCOG), permettant une étude analytique par le suivi dynamique des variables, des synapses de RNAI jusqu'aux comportements d'un ou plusieurs robots virtuels ou physiques. Finalement, outre l'intégration de plusieurs fonctions différentes d'apprentissage dans des RNAI, une autre des conclusions de ce travail suggère que des robots virtuels et physiques peuvent apprendre et s'adapter au niveau comportemental, de façon similaire aux agents naturels. Ces observations comportementales sont basées sur la simulation de mécanismes de plasticité synaptique modulés par des variables temporelles relatives aux stimuli physiques et aux activités cellulaires neuronales.\ud ______________________________________________________________________________ \ud MOTS-CLÉS DE L’AUTEUR : Intelligence artificielle, Cognition, Simulateur, Robotique bio-inspirée, Réseaux de neurones artificiels à impulsions, Apprentissage, Habituation, Conditionnement classique, Conditionnement opérant, Plasticité synaptiqu

From Biological Synapses to "Intelligent" Robots

This selective review explores biologically inspired learning as a model for intelligent robot control and sensing technology on the basis of specific examples. Hebbian synaptic learning is discussed as a functionally relevant model for machine learning and intelligence, as explained on the basis of examples from the highly plastic biological neural networks of invertebrates and vertebrates. Its potential for adaptive learning and control without supervision, the generation of functional complexity, and control architectures based on self-organization is brought forward. Learning without prior knowledge based on excitatory and inhibitory neural mechanisms accounts for the process through which survival-relevant or task-relevant representations are either reinforced or suppressed. The basic mechanisms of unsupervised biological learning drive synaptic plasticity and adaptation for behavioral success in living brains with different levels of complexity. The insights collected here point toward the Hebbian model as a choice solution for “intelligent” robotics and sensor systems. Keywords: Hebbian learning; synaptic plasticity; neural networks; self-organization; brain; reinforcement; sensory processing; robot contro

Neuromorphic Engineering Editors' Pick 2021

This collection showcases well-received spontaneous articles from the past couple of years, which have been specially handpicked by our Chief Editors, Profs. André van Schaik and Bernabé Linares-Barranco. The work presented here highlights the broad diversity of research performed across the section and aims to put a spotlight on the main areas of interest. All research presented here displays strong advances in theory, experiment, and methodology with applications to compelling problems. This collection aims to further support Frontiers’ strong community by recognizing highly deserving authors

Insect neuroethology of reinforcement learning

Historically, reinforcement learning is a branch of machine learning founded on observations of how animals learn. This involved collaboration between the fields of biology and artificial intelligence that was beneficial to both fields, creating smarter artificial agents and improving the understanding of how biological systems function. The evolution of reinforcement learning during the past few years was rapid but substantially diverged from providing insights into how biological systems work, opening a gap between reinforcement learning and biology. In an attempt to close this gap, this thesis studied the insect neuroethology of reinforcement learning, that is, the neural circuits that underlie reinforcement-learning-related behaviours in insects. The goal was to extract a biologically plausible plasticity function from insect-neuronal data, use this to explain biological findings and compare it to more standard reinforcement learning models. Consequently, a novel dopaminergic plasticity rule was developed to approximate the function of dopamine as the plasticity mechanism between neurons in the insect brain. This allowed a range of observed learning phenomena to happen in parallel, like memory depression, potentiation, recovery, and saturation. In addition, by using anatomical data of connections between neurons in the mushroom body neuropils of the insect brain, the neural incentive circuit of dopaminergic and output neurons was also explored. This, together with the dopaminergic plasticity rule, allowed for dynamic collaboration amongst parallel memory functions, such as acquisition, transfer, and forgetting. When tested on olfactory conditioning paradigms, the model reproduced the observed changes in the activity of the identified neurons in fruit flies. It also replicated the observed behaviour of the animals and it allowed for flexible behavioural control. Inspired by the visual navigation system of desert ants, the model was further challenged in the visual place recognition task. Although a relatively simple encoding of the olfactory information was sufficient to explain odour learning, a more sophisticated encoding of the visual input was required to increase the separability among the visual inputs and enable visual place recognition. Signal whitening and sparse combinatorial encoding were sufficient to boost the performance of the system in this task. The incentive circuit enabled the encoding of increasing familiarity along a known route, which dropped proportionally to the distance of the animal from that route. Finally, the proposed model was challenged in delayed reinforcement tasks, suggesting that it might take the role of an adaptive critic in the context of reinforcement learning

Brain-machine interface coupled cognitive sensory fusion with a Kohonen and reservoir computing scheme

Artificial Intelligence (AI) has been a source of great intrigue and has spawned many questions regarding the human condition and the core of what it means to be a sentient entity. The field has bifurcated into so-called “weak” and “strong” artificial intelligence. In weak artificial intelligence reside the forms of automation and data mining that we interact with on a daily basis. Strong artificial intelligence can be best defined as a “synthetic” being with cognitive abilities and the capacity for presence of mind that we would normally associate with humankind. We feel that this distinction is misguided. First, we begin with the statement that intelligence lies on a spectrum, even in artificial systems. The fact that our systems currently can be considered weak artificial intelligence does not preclude our ability to develop an understanding that can lead us to more complex behavior. In this research, we utilized neural feedback via electroencephalogram (EEG) data to develop an emotional landscape for linguistic interaction via the android's sensory fields which we consider to be part and parcel of embodied cognition. We have also given the iCub child android the instinct to babble the words it has learned. This is a skill that we leveraged for low-level linguistic acquisition in the latter part of this research, the slightly stronger artificial intelligence goal. This research is motivated by two main questions regarding intelligence: Is intelligence an emergent phenomenon? And, if so, can multi-modal sensory information and a term coined called “co-intelligence” which is a shared sensory experience via coupling EEG input, assist in the development of representations in the mind that we colloquially refer to as language? Given that it is not reasonable to program all of the activities needed to foster intelligence in artificial systems, our hope is that these types of forays will set the stage for further development of stronger artificial intelligence constructs. We have incorporated self-organizing processes - i.e. Kohonen maps, hidden Markov models for the speech, language development and emotional information via neural data - to help lay the substrate for emergence. Next, homage is given to the central and unique role played in intellectual study by language. We have also developed rudimentary associative memory for the iCub that is derived from the aforementioned sensory input that was collected. We formalized this process only as needed, but that is based on the assumption that mind, brain and language can be represented using the mathematics and logic of the day without contradiction. We have some reservations regarding this statement, but unfortunately a proof is a task beyond the scope of this Ph.D. Finally, this data from the coupling of the EEG and the other sensory modes of embodied cognition is used to interact with a reservoir computing recurrent neural network in an attempt to produce simple language interaction, e.g. babbling, from the child android

Using MapReduce Streaming for Distributed Life Simulation on the Cloud

Distributed software simulations are indispensable in the study of large-scale life models but often require the use of technically complex lower-level distributed computing frameworks, such as MPI. We propose to overcome the complexity challenge by applying the emerging MapReduce (MR) model to distributed life simulations and by running such simulations on the cloud. Technically, we design optimized MR streaming algorithms for discrete and continuous versions of Conway’s life according to a general MR streaming pattern. We chose life because it is simple enough as a testbed for MR’s applicability to a-life simulations and general enough to make our results applicable to various lattice-based a-life models. We implement and empirically evaluate our algorithms’ performance on Amazon’s Elastic MR cloud. Our experiments demonstrate that a single MR optimization technique called strip partitioning can reduce the execution time of continuous life simulations by 64%. To the best of our knowledge, we are the first to propose and evaluate MR streaming algorithms for lattice-based simulations. Our algorithms can serve as prototypes in the development of novel MR simulation algorithms for large-scale lattice-based a-life models.https://digitalcommons.chapman.edu/scs_books/1014/thumbnail.jp