A neuromorphic approach to auditory pattern recognition in cricket phonotaxis

Rost T, Ramachandran H, Nawrot MP, Chicca E. A neuromorphic approach to auditory pattern recognition in cricket phonotaxis. In: 2013 European Conference on Circuit Theory and Design (ECCTD). IEEE; 2013: 1-4.Developing neuromorphic computing paradigms that mimic nervous system function is an emerging field of research with high potential for technical applications. In the present study we take inspiration from the cricket auditory system and propose a biologically plausible neural network architecture that can explain how acoustic pattern recognition is achieved in the cricket central brain. Our circuit model combines two key features of neural processing dynamics: Spike Frequency Adaptation (SFA) and synaptic short term plasticity. We developed and extensively tested the model function in software simulations. Furthermore, the feasibility of an analogue VLSI implementation is demonstrated using a multi-neuron chip comprising Integrate-and-Fire (IF) neurons and adaptive synapses

Phonotaxis in crickets and robots

Journal ArticleOver the past decade, we have built and tested several robot models to investigate a particular biological behavior, the sound localizing (phonotaxis) ability of the cricket. This work has had several purposes. One is to develop robotic technology, such as novel sensors and control systems, by copying biology. However, the primary motivation is the "reverse"-to use the technology to develop understanding of biological systems-in particular, how neural circuits control sensorimotor behavior. This is effectively a new methodology for biological modeling, discussed in Webb (to appear). In this chapter, the aim is to provide a summary of the problems addressed and the key results to date. (More-detailed presentations of the biological background and the implementations can be found in Lund, Webb, and Hallam, 1997, 1998; Webb and Scutt, 2000; and Webb and Harrison, 2000.

Short term plasticity. A neuromorphic perspective

Ramachandran H. Short term plasticity. A neuromorphic perspective. Bielefeld: Universität Bielefeld; 2018

The importance of space and time in neuromorphic cognitive agents

Artificial neural networks and computational neuroscience models have made tremendous progress, allowing computers to achieve impressive results in artificial intelligence (AI) applications, such as image recognition, natural language processing, or autonomous driving. Despite this remarkable progress, biological neural systems consume orders of magnitude less energy than today's artificial neural networks and are much more agile and adaptive. This efficiency and adaptivity gap is partially explained by the computing substrate of biological neural processing systems that is fundamentally different from the way today's computers are built. Biological systems use in-memory computing elements operating in a massively parallel way rather than time-multiplexed computing units that are reused in a sequential fashion. Moreover, activity of biological neurons follows continuous-time dynamics in real, physical time, instead of operating on discrete temporal cycles abstracted away from real-time. Here, we present neuromorphic processing devices that emulate the biological style of processing by using parallel instances of mixed-signal analog/digital circuits that operate in real time. We argue that this approach brings significant advantages in efficiency of computation. We show examples of embodied neuromorphic agents that use such devices to interact with the environment and exhibit autonomous learning

Predicting voluntary movements from motor cortical activity with neuromorphic hardware

This document is the Accepted Manuscript version of the following article: A. Lungu, A. Riehle, M. P. Nawrot and M. Schmuker, "Predicting voluntary movements from motor cortical activity with neuromorphic hardware," in IBM Journal of Research and Development, Vol. 61, no. 2/3, pp. 5:1-5:12, March-May 1 2017. The version of record is available online at doi: 10.1147/JRD.2017.2656063. © 2017 by International Business Machines Corporation. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.Neurons in the mammalian motor cortices encode physical parameters of voluntary movements during planning and execution of a motor task. Brain-machine interfaces can decode limb movements from the activity of these neurons in real time. The future goal is to control prosthetic devices in severely paralyzed patients or to restore communication if the ability to speak or make gestures is lost. Here, we implemented a spiking neural network that decodes movement intentions from individual neuronal activity recorded in the motor cortex of a monkey. The network runs on neuromorphic hardware and performs its computations in a purely spike-based fashion. It incorporates an insect-brain-inspired, three-layer architecture with 176 neurons. Cortical signals are filtered using lateral inhibition, and the network is trained in a supervised fashion to predict two opposing directions of the monkey’s arm reaching movement before the movement is carried out. Our network operates on the actual spikes that have been emitted by motor cortical neurons, without the need to construct intermediate non-spiking representations. Using a pseudo-population of 12 manually-selected neurons, it reliably predicts the movement direction with an accuracy of 89.32 % on unseen data after only 100 training trials. Our results provide a proof of concept for the first-time use of a neuromorphic device for decoding movement intentions.Peer reviewe

Biomimetic models of visual navigation - active sensing for embodied intelligence

Insects have developed small scale search behaviours to pursue navigation relevant stimuli more effectively. These often resemble a variation of Zig-Zagging, steering periodically to the left and right, therefore increasing the sampling. In this context we investigate the role of a homologous insect brain structure, the Lateral Accessory Lobe (LAL), which has been described as a pre-motor centre but received limited attention so far. Following a synthesis of the literature on the LAL we developed a steering framework, which proposes that with lateralised stimuli as input, the LAL can initiate a Zig-Zagging behaviour if the input is too weak, meaning unreliable, and targeted steering behaviours if the input is strong, thus reliable. Based on this framework we model a Spiking Neural Network (SNN) investigating a sensory modulated Central Pattern Generator (CPG) as a possible neural mechanism enabling adaptive search behaviours. We investigated the parameter space of the model to discover both the range of possible behaviours as well as which parameter combinations lead to the previously described behaviour. We found that no parameter combination accounts for the majority of observed behaviours. Furthermore, changing the computational noise levels does not lead to break-down of this behaviour. We conclude, that this neural architecture is robust to generate an adaptable Zig-Zagging behaviour. Additionally, we developed a more comprehensive network to explore the functions of known neuron-types with regard to motor control. To investigate how this steering framework might work for view based navigation, we investigated how lateralised sensory input can be used for snapshot navigation. We used a 3D-reconstruction from a LiDAR-scanned field-site (“Antworld”) to generate realistic visual stimuli. Instead of using the entire panorama, we subdivided this into two Fields of View for snapshot generation and the later image comparisons. The difference of image familiarity from both sides was subtracted to initiate a steering response into the most familiar direction. We found that a bigger Field of View alongside non-forward facing memories generated the most correct steering responses towards the snapshot direction. This demonstrates that the LAL-inspired steering framework can be functional for a complex sensori-motor task that had previously not been implicated in LAL functionality. Finally, we modelled how bilateral sensory information and a SNN model of the LAL behave in a snapshot navigation setup using Antworld. We compared the original snapshot navigation model using a panoramic Field of View with several combinations of the Core-Network and bilateral vision models: using a bilateral view, a bilateral view with the SNN, a panoramic view with SNN and other standard movement behaviours. We confirmed the findings of preliminary work, in an abstract setup, that had shown that a bilateral view combined with a SNN performs best to recover and approach navigation relevant locations. Also introducing models based on the steering framework into this visually complex environment improved the performance of agents performing snapshot navigation

Towards a bionic bat: A biomimetic investigation of active sensing, Doppler-shift estimation, and ear morphology design for mobile robots.

Institute of Perception, Action and BehaviourSo-called CF-FM bats are highly mobile creatures who emit long calls in which much of the energy is concentrated in a single frequency. These bats face sensor interpretation problems very similar to those of mobile robots provided with ultrasonic sensors, while navigating in cluttered environments. This dissertation presents biologically inspired engineering on the use of narrowband Sonar in mobile robotics. It replicates, using robotics as a modelling medium, how CF-FM bats process and use the constant frequency part of their emitted call for several tasks, aiming to improve the design and use of narrowband ultrasonic sensors for mobile robot navigation. The experimental platform for the work is RoBat, the biomimetic sonarhead designed by Peremans and Hallam, mounted on a commercial mobile platform as part of the work reported in this dissertation. System integration, including signal processing capabilities inspired by the bat’s auditory system and closed loop control of both sonarhead and mobile base movements, was designed and implemented. The result is a versatile tool for studying the relationship between environmental features, their acoustic correlates and the cues computable from them, in the context of both static, and dynamic real-time closed loop, behaviour. Two models of the signal processing performed by the bat’s cochlea were implemented, based on sets of bandpass filters followed by full-wave rectification and low-pass filtering. One filterbank uses Butterworth filters whose centre frequencies vary linearly across the set. The alternative filterbank uses gammatone filters, with centre frequencies varying non-linearly across the set. Two methods of estimating Doppler-shift from the return echoes after cochlear signal processing were implemented. The first was a simple energy-weighted average of filter centre frequencies. The second was a novel neural network-based technique. Each method was tested with each of the cochlear models, and evaluated in the context of several dynamic tasks in which RoBat was moved at different velocities towards stationary echo sources such as walls and posts. Overall, the performance of the linear filterbank was more consistent than the gammatone. The same applies to the ANN, with consistently better noise performance than the weighted average. The effect of multiple reflectors contained in a single echo was also analysed in terms of error in Doppler-shift estimation assuming a single wider reflector. Inspired by the Doppler-shift compensation and obstacle avoidance behaviours found in CF-FM bats, a Doppler-based controller suitable for collision detection and convoy navigation in robots was devised and implemented in RoBat. The performance of the controller is satisfactory despite low Doppler-shift resolution caused by lower velocity of the robot when compared to real bats. Barshan’s and Kuc’s 2D object localisation method was implemented and adapted to the geometry of RoBat’s sonarhead. Different TOF estimation methods were tested, the parabola fitting being the most accurate. Arc scanning, the ear movement technique to recover elevation cues proposed by Walker, and tested in simulation by her, Peremans and Hallam, was here implemented on RoBat, and integrated with Barshan’s and Kuc’s method in a preliminary narrowband 3D tracker. Finally, joint work with Kim, K¨ampchen and Hallam on designing optimal reflector surfaces inspired by the CF-FM bat’s large pinnae is presented. Genetic algorithms are used for improving the current echolocating capabilities of the sonarhead for both arc scanning and IID behaviours. Multiple reflectors around the transducer using a simple ray light-like model of sound propagation are evolved. Results show phase cancellation problems and the need of a more complete model of wave propagation. Inspired by a physical model of sound diffraction and reflections in the human concha a new model is devised and used to evolve pinnae surfaces made of finite elements. Some interesting paraboloid shapes are obtained, improving performance significantly with respect to the bare transducer

Peripersonal Space in the Humanoid Robot iCub

Developing behaviours for interaction with objects close to the body is a primary goal for any organism to survive in the world. Being able to develop such behaviours will be an essential feature in autonomous humanoid robots in order to improve their integration into human environments. Adaptable spatial abilities will make robots safer and improve their social skills, human-robot and robot-robot collaboration abilities. This work investigated how a humanoid robot can explore and create action-based representations of its peripersonal space, the region immediately surrounding the body where reaching is possible without location displacement. It presents three empirical studies based on peripersonal space findings from psychology, neuroscience and robotics. The experiments used a visual perception system based on active-vision and biologically inspired neural networks. The first study investigated the contribution of binocular vision in a reaching task. Results indicated the signal from vergence is a useful embodied depth estimation cue in the peripersonal space in humanoid robots. The second study explored the influence of morphology and postural experience on confidence levels in reaching assessment. Results showed that a decrease of confidence when assessing targets located farther from the body, possibly in accordance to errors in depth estimation from vergence for longer distances. Additionally, it was found that a proprioceptive arm-length signal extends the robot’s peripersonal space. The last experiment modelled development of the reaching skill by implementing motor synergies that progressively unlock degrees of freedom in the arm. The model was advantageous when compared to one that included no developmental stages. The contribution to knowledge of this work is extending the research on biologically-inspired methods for building robots, presenting new ways to further investigate the robotic properties involved in the dynamical adaptation to body and sensing characteristics, vision-based action, morphology and confidence levels in reaching assessment.CONACyT, Mexico (National Council of Science and Technology

Autopoietic-extended architecture: can buildings think?

To incorporate bioremedial functions into the performance of buildings and to balance generative architecture's dominant focus on computational programming and digital fabrication, this thesis first hybridizes theories of autopoiesis into extended cognition in order to research biological domains that include synthetic biology and biocomputation. Under the rubric of living technology I survey multidisciplinary fields to gather perspective for student design of bioremedial and/or metabolic components in generative architecture where generative not only denotes the use of computation but also includes biochemical, biomechanical, and metabolic functions. I trace computation and digital simulations back to Alan Turing's early 1950s Morphogenetic drawings, reaction-diffusion algorithms, and pioneering artificial intelligence (AI) in order to establish generative architecture's point of origin. I ask provocatively: Can buildings think? as a question echoing Turing's own "Can machines think?" Thereafter, I anticipate not only future bioperformative materials but also theories capable of underpinning strains of metabolic intelligences made possible via AI, synthetic biology, and living technology. I do not imply that metabolic architectural intelligence will be like human cognition. I suggest, rather, that new research and pedagogies involving the intelligence of bacteria, plants, synthetic biology, and algorithms define approaches that generative architecture should take in order to source new forms of autonomous life that will be deployable as corrective environmental interfaces. I call the research protocol autopoietic-extended design, theorizing it as an operating system (OS), a research methodology, and an app schematic for design studios and distance learning that makes use of in-field, e-, and m-learning technologies. A quest of this complexity requires scaffolding for coordinating theory-driven teaching with practice-oriented learning. Accordingly, I fuse Maturana and Varela's biological autopoiesis and its definitions of minimal biological life with Andy Clark's hypothesis of extended cognition and its cognition-to-environment linkages. I articulate a generative design strategy and student research method explained via architectural history interpreted from Louis Sullivan's 1924 pedagogical drawing system, Le Corbusier's Modernist pronouncements, and Greg Lynn's Animate Form. Thus, autopoietic-extended design organizes thinking about the generation of ideas for design prior to computational production and fabrication, necessitating a fresh relationship between nature/science/technology and design cognition. To systematize such a program requires the avoidance of simple binaries (mind/body, mind/nature) as well as the stationing of tool making, technology, and architecture within the ream of nature. Hence, I argue, in relation to extended phenotypes, plant-neurobiology, and recent genetic research: Consequently, autopoietic-extended design advances design protocols grounded in morphology, anatomy, cognition, biology, and technology in order to appropriate metabolic and intelligent properties for sensory/response duty in buildings. At m-learning levels smartphones, social media, and design apps source data from nature for students to mediate on-site research by extending 3D pedagogical reach into new university design programs. I intend the creation of a dialectical investigation of animal/human architecture and computational history augmented by theory relevant to current algorithmic design and fablab production. The autopoietic-extended design dialectic sets out ways to articulate opposition/differences outside the Cartesian either/or philosophy in order to prototype metabolic architecture, while dialectically maintaining: Buildings can think