Neuromorphic Sensory Integration for Combining Sound Source Localization and Collision Avoidance

Animals combine various sensory cues with previously acquired knowledge to safely travel towards a target destination. In close analogy to biological systems, we propose a neuromorphic system which decides, based on auditory and visual input, how to reach a sound source without collisions. The development of this sensory integration system, which identifies the shortest possible path, is a key achievement towards autonomous robotics. The proposed neuromorphic system comprises two event based sensors (the eDVS for vision and the NAS for audition) and the SpiNNaker processor. Open loop experiments were performed to evaluate the system performances. In the presence of acoustic stimulation alone, the heading direction points to the direction of the sound source with a Pearson correlation coefficient of 0.89. When visual input is introduced into the network the heading direction always points at the direction of null optical flow closest to the sound source. Hence, the sensory integration network is able to find the shortest path to the sound source while avoiding obstacles. This work shows that a simple, task dependent mapping of sensory information can lead to highly complex and robust decisions.Ministerio de Economía y Competitividad TEC2016-77785-

Finding the Gap:Neuromorphic Motion Vision in Cluttered Environments

Many animals meander in environments and avoid collisions. How the underlying neuronal machinery can yield robust behaviour in a variety of environments remains unclear. In the fly brain, motion-sensitive neurons indicate the presence of nearby objects and directional cues are integrated within an area known as the central complex. Such neuronal machinery, in contrast with the traditional stream-based approach to signal processing, uses an event-based approach, with events occurring when changes are sensed by the animal. Contrary to von Neumann computing architectures, event-based neuromorphic hardware is designed to process information in an asynchronous and distributed manner. Inspired by the fly brain, we model, for the first time, a neuromorphic closed-loop system mimicking essential behaviours observed in flying insects, such as meandering in clutter and gap crossing, which are highly relevant for autonomous vehicles. We implemented our system both in software and on neuromorphic hardware. While moving through an environment, our agent perceives changes in its surroundings and uses this information for collision avoidance. The agent's manoeuvres result from a closed action-perception loop implementing probabilistic decision-making processes. This loop-closure is thought to have driven the development of neural circuitry in biological agents since the Cambrian explosion. In the fundamental quest to understand neural computation in artificial agents, we come closer to understanding and modelling biological intelligence by closing the loop also in neuromorphic systems. As a closed-loop system, our system deepens our understanding of processing in neural networks and computations in biological and artificial systems. With these investigations, we aim to set the foundations for neuromorphic intelligence in the future, moving towards leveraging the full potential of neuromorphic systems.Comment: 7 main pages with two figures, including appendix 26 pages with 14 figure

Neuromorphic Sensory Integration for Combining Sound Source Localization and Collision Avoidance

Schoepe T, Gutierrez-Galan D, Dominguez-Morales JP, Jimenez-Fernandez A, Linares-Barranco A, Chicca E. Neuromorphic Sensory Integration for Combining Sound Source Localization and Collision Avoidance. 2019 IEEE Biomedical Circuits and Systems Conference (BioCAS). 2019:1-4

Live Demonstration: Neuromorphic Sensory Integration for Combining Sound Source Localization and Collision Avoidance

Schoepe T, Gutierrez-Galan D, Dominguez-Morales JP, Jimenez-Fernandez A, Linares-Barranco A, Chicca E. Live Demonstration: Neuromorphic Sensory Integration for Combining Sound Source Localization and Collision Avoidance. Presented at the 2020 IEEE International Symposium on Circuits & Systems, Seville, Spain

Live Demonstration: Neuromorphic Sensory Integration for Combining Sound Source Localization and Collision Avoidance

The brain is able to solve complex tasks in real time by combining different sensory cues with previously acquired knowledge. Inspired by the brain, we designed a neuromorphicdemonstrator which combines auditory and visual input to find an obstacle free direction closest to the sound source. The system consists of two event-based sensors (the eDVS for vision and the NAS for audition) mounted onto a pan-tilt unit and a spiking neural network implemented on the SpiNNaker platform. By combining the different sensory information, the demonstrator is able to point at a sound source direction while avoiding obstacles in real tim