Bat azimuthal echolocation using interaural level differences: modeling and implementation by a VLSI-based hardware system

Bats have long fascinated both scientists and engineers due to their superb ability to use echolocation to fly with speed and agility through complex natural environments in complete darkness. This dissertation presents a neuromorphic VLSI circuit model of bat azimuthal echolocation. Interaural level differences (ILDs) are the cues for bat azimuthal echolocation and are also the primary cues used by other mammals to localize high frequency sounds. The fact that neurons in bats respond to short echoes by one or two spikes strongly suggests that the conventionally used firing rate is an unlikely code. The operation of first spike latency in ILD computation and transformation is investigated in a network of spiking neurons linking the lateral superior olive (LSO), dorsal nucleus of the lateral lemniscus (DNLL), and inferior colliculus (IC). The results of the investigation suggest that spatially distributed first spike latencies can serve as a fast code for azimuth that can be ``read-out'' by ascending stages. With the hardware echolocation model that uses spike timing representation, we study how multiple echoes can affect bat echolocation and demonstrate that the response to multiple sounds is not a simple linear addition of the response to single sounds. By developing functional models of the bat echolocation system, we can study the efficient implementation demonstrated by nature. For example, variations among analog VLSI circuit units due to the unavoidable transistor mismatch - traditionally thought of as a hurdle to overcome - have been found beneficial in generating the desired diversity of response that is similar to their neural counterparts. This work advocates the use and design of summating and exponentially decaying synapses. A compact and easily controllable synapse circuit has found an application in achieving a linear temporal spike summation by operating with a very short time constants. It has also been applied in modeling a nonlinear intensity-latency trading by working with a long synaptic time constant. We propose a new synapse circuit model that is compatible with those used in computational models and implementable by CMOS transistors operating in the subthreshold region

NEUROMORPHIC VLSI REALIZATION OF THE HIPPOCAMPAL FORMATION AND THE LATERAL SUPERIOR OLIVE

In this work, the focus is on realizing the function of the hippocampal formation (HF) and the lateral superior olive (LSO) in electronic circuits. The first major contribution of this dissertation is to realize the function of the HF in silicon. This was based on the GRIDSmap model and the Bayesian integration. For this, two novel circuits were designed and integrated with others. The first circuit was that of a Bayesian integration synapse which can perform Bayesian integration at the single neuron level. The second circuit was that of a velocity integrator which is so compact that it can enable integration of the entire system on a single chip compared to its predecessors which would have needed 27 chips! However, since the computational neuroscience models of the hippocampal place cells do not explain all the characteristics observed empirically, a novel model for the place cells, based on the sensori-motor integration of inputs is proposed. This is the second major contribution of this thesis. The third major contribution is to demonstrate a VLSI system which can perform azimuthal localization based on population response of the LSO. This system was based on the Reed and Blum's model of the LSO. For this, a novel circuit of a second order synapse and that of a conductance neuron was designed and integrated with other circuits. This synapse circuit can produce an output current whose peak is delayed and is proportional to the number of inputs it receives. The HF is thought to aid in spatial navigation and the LSO is thought to be involved in azimuthal localization of sounds both of which are useful for autonomous robotic spatial navigation. Hence, silicon realization of these two will be useful in robotics which is an area of interest for the neuromorphic engineers

Neural architecture for echo suppression during sound source localization based on spiking neural cell models

Zusammenfassung Diese Arbeit untersucht die biologischen Ursachen des psycho-akustischen Präzedenz Effektes, der Menschen in die Lage versetzt, akustische Echos während der Lokalisation von Schallquellen zu unterdrücken. Sie enthält ein Modell zur Echo-Unterdrückung während der Schallquellenlokalisation, welches in technischen Systemen zur Mensch-Maschine Interaktion eingesetzt werden kann. Die Grundlagen dieses Modells wurden aus eigenen elektrophysiologischen Experimenten an der Mongolischen Wüstenrennmaus gewonnen. Die dabei erstmalig an der Wüstenrennmaus erzielten Ergebnisse, zeigen ein besonderes Verhalten spezifischer Zellen im Dorsalen Kern des Lateral Lemniscus, einer dedizierten Region des auditorischen Hirnstammes. Die dort sichtbare Langzeithemmung scheint die Grundlage für die Echounterdrückung in höheren auditorischen Zentren zu sein. Das entwickelte Model war in der Lage dieses Verhalten nachzubilden, und legt die Vermutung nahe, dass eine starke und zeitlich präzise Hyperpolarisation der zugrundeliegende physiologische Mechanismus dieses Verhaltens ist. Die entwickelte Neuronale Modellarchitektur modelliert das Innenohr und fünf wesentliche Kerne des auditorischen Hirnstammes in ihrer Verbindungsstruktur und internen Dynamik. Sie stellt einen neuen Typus neuronaler Modellierung dar, der als Spike-Interaktionsmodell (SIM) bezeichnet wird. SIM nutzen die präzise räumlich-zeitliche Interaktion einzelner Aktionspotentiale (Spikes) für die Kodierung und Verarbeitung neuronaler Informationen. Die Basis dafür bilden Integrate-and-Fire Neuronenmodelle sowie Hebb'sche Synapsen, welche um speziell entwickelte dynamische Kernfunktionen erweitert wurden. Das Modell ist in der Lage, Zeitdifferenzen von 10 mykrosekunden zu detektieren und basiert auf den Prinzipien der zeitlichen und räumlichen Koinzidenz sowie der präzisen lokalen Inhibition. Es besteht ausschließlich aus Elementen einer eigens entwickelten Neuronalen Basisbibliothek (NBL) die speziell für die Modellierung verschiedenster Spike- Interaktionsmodelle entworfen wurde. Diese Bibliothek erweitert die kommerziell verfügbare dynamische Simulationsumgebung von MATLAB/SIMULINK um verschiedene Modelle von Neuronen und Synapsen, welche die intrinsischen dynamischen Eigenschaften von Nervenzellen nachbilden. Die Nutzung dieser Bibliothek versetzt sowohl den Ingenieur als auch den Biologen in die Lage, eigene, biologisch plausible, Modelle der neuronalen Informationsverarbeitung ohne detaillierte Programmierkenntnisse zu entwickeln. Die grafische Oberfläche ermöglicht strukturelle sowie parametrische Modifikationen und ist in der Lage, den Zeitverlauf mikroskopischer Zellpotentiale aber auch makroskopischer Spikemuster während und nach der Simulation darzustellen. Zwei grundlegende Elemente der Neuronalen Basisbibliothek wurden zur Implementierung als spezielle analog-digitale Schaltungen vorbereitet. Erste Silizium Implementierungen durch das Team des DFG Graduiertenkollegs GRK 164 konnten die Möglichkeit einer vollparallelen on line Verarbeitung von Schallsignalen nachweisen. Durch Zuhilfenahme des im GRK entwickelten automatisierten Layout Generators wird es möglich, spezielle Prozessoren zur Anwendung biologischer Verarbeitungsprinzipien in technischen Systemen zu entwickeln. Diese Prozessoren unterscheiden sich grundlegend von den klassischen von Neumann Prozessoren indem sie räumlich und zeitlich verteilte Spikemuster, anstatt sequentieller binärer Werte zur Informationsrepräsentation nutzen. Sie erweitern das digitale Kodierungsprinzip durch die Dimensionen des Raumes (2 dimensionale Nachbarschaft) der Zeit (Frequenz, Phase und Amplitude) sowie der zeitlichen Dynamik analoger Potentialverläufe. Diese Dissertation besteht aus sieben Kapiteln, welche den verschiedenen Bereichen der Computational Neuroscience gewidmet sind. Kapitel 1 beschreibt die Motivation dieser Arbeit welche aus der Absicht rühren, biologische Prinzipien der Schallverarbeitung zu erforschen und für technische Systeme während der Interaktion mit dem Menschen nutzbar zu machen. Zusätzlich werden fünf Gründe für die Nutzung von Spike-Interaktionsmodellen angeführt sowie deren neuartiger Charakter beschrieben. Kapitel 2 führt die biologischen Prinzipien der Schallquellenlokalisation und den psychoakustischen Präzedenz Effekt ein. Aktuelle Hypothesen zur Entstehung dieses Effektes werden anhand ausgewählter experimenteller Ergebnisse verschiedener Forschungsgruppen diskutiert. Kapitel 3 beschreibt die entwickelte Neuronale Basisbibliothek und führt die einzelnen neuronalen Simulationselemente ein. Es erklärt die zugrundeliegenden mathematischen Funktionen der dynamischen Komponenten und beschreibt deren generelle Einsetzbarkeit zur dynamischen Simulation spikebasierter Neuronaler Netzwerke. Kapitel 4 enthält ein speziell entworfenes Modell des auditorischen Hirnstammes beginnend mit den Filterkaskaden zur Simulation des Innenohres, sich fortsetzend über mehr als 200 Zellen und 400 Synapsen in 5 auditorischen Kernen bis zum Richtungssensor im Bereich des auditorischen Mittelhirns. Es stellt die verwendeten Strukturen und Parameter vor und enthält grundlegende Hinweise zur Nutzung der Simulationsumgebung. Kapitel 5 besteht aus drei Abschnitten, wobei der erste Abschnitt die Experimentalbedingungen und Ergebnisse der eigens durchgeführten Tierversuche beschreibt. Der zweite Abschnitt stellt die Ergebnisse von 104 Modellversuchen zur Simulationen psycho-akustischer Effekte dar, welche u.a. die Fähigkeit des Modells zur Nachbildung des Präzedenz Effektes testen. Schließlich beschreibt der letzte Abschnitt die Ergebnisse der 54 unter realen Umweltbedingungen durchgeführten Experimente. Dabei kamen Signale zur Anwendung, welche in normalen sowie besonders stark verhallten Räumen aufgezeichnet wurden. Kapitel 6 vergleicht diese Ergebnisse mit anderen biologisch motivierten und technischen Verfahren zur Echounterdrückung und Schallquellenlokalisation und führt den aktuellen Status der Hardwareimplementierung ein. Kapitel 7 enthält schließlich eine kurze Zusammenfassung und einen Ausblick auf weitere Forschungsobjekte und geplante Aktivitäten. Diese Arbeit möchte zur Entwicklung der Computational Neuroscience beitragen, indem sie versucht, in einem speziellen Anwendungsfeld die Lücke zwischen biologischen Erkenntnissen, rechentechnischen Modellen und Hardware Engineering zu schließen. Sie empfiehlt ein neues räumlich-zeitliches Paradigma der dynamischen Informationsverarbeitung zur Erschließung biologischer Prinzipien der Informationsverarbeitung für technische Anwendungen.This thesis investigates the biological background of the psycho-acoustical precedence effect, enabling humans to suppress echoes during the localization of sound sources. It provides a technically feasible and biologically plausible model for sound source localization under echoic conditions, ready to be used by technical systems during man-machine interactions. The model is based upon own electro-physiological experiments in the mongolian gerbil. The first time in gerbils obtained results reveal a special behavior of specific cells of the dorsal nucleus of the lateral lemniscus (DNLL) - a distinct region in the auditory brainstem. The explored persistent inhibition effect of these cells seems to account for the base of echo suppression at higher auditory centers. The developed model proved capable to duplicate this behavior and suggests, that a strong and timely precise hyperpolarization is the basic mechanism behind this cell behavior. The developed neural architecture models the inner ear as well as five major nuclei of the auditory brainstem in their connectivity and intrinsic dynamics. It represents a new type of neural modeling described as Spike Interaction Models (SIM). SIM use the precise spatio-temporal interaction of single spike events for coding and processing of neural information. Their basic elements are Integrate-and-Fire Neurons and Hebbian synapses, which have been extended by specially designed dynamic transfer functions. The model is capable to detect time differences as small as 10 mircrosecondes and employs the principles of coincidence detection and precise local inhibition for auditory processing. It consists exclusively of elements of a specifically designed Neural Base Library (NBL), which has been developed for multi purpose modeling of Spike Interaction Models. This library extends the commercially available dynamic simulation environment of MATLAB/SIMULINK by different models of neurons and synapses simulating the intrinsic dynamic properties of neural cells. The usage of this library enables engineers as well as biologists to design their own, biologically plausible models of neural information processing without the need for detailed programming skills. Its graphical interface provides access to structural as well as parametric changes and is capable to display the time course of microscopic cell parameters as well as macroscopic firing pattern during simulations and thereafter. Two basic elements of the Neural Base Library have been prepared for implementation by specialized mixed analog-digital circuitry. First silicon implementations were realized by the team of the DFG Graduiertenkolleg GRK 164 and proved the possibility of fully parallel on line processing of sounds. By using the automated layout processor under development in the Graduiertenkolleg, it will be possible to design specific processors in order to apply theprinciples of distributed biological information processing to technical systems. These processors differ from classical von Neumann processors by the use of spatio temporal spike pattern instead of sequential binary values. They will extend the digital coding principle by the dimensions of space (spatial neighborhood), time (frequency, phase and amplitude) as well as the dynamics of analog potentials and introduce a new type of information processing. This thesis consists of seven chapters, dedicated to the different areas of computational neuroscience. Chapter 1: provides the motivation of this study arising from the attempt to investigate the biological principles of sound processing and make them available to technical systems interacting with humans under real world conditions. Furthermore, five reasons to use spike interaction models are given and their novel characteristics are discussed. Chapter 2: introduces the biological principles of sound source localization and the precedence effect. Current hypothesis on echo suppression and the underlying principles of the precedence effect are discussed by reference to a small selection of physiological and psycho-acoustical experiments. Chapter 3: describes the developed neural base library and introduces each of the designed neural simulation elements. It also explains the developed mathematical functions of the dynamic compartments and describes their general usage for dynamic simulation of spiking neural networks. Chapter 4: introduces the developed specific model of the auditory brainstem, starting from the filtering cascade in the inner ear via more than 200 cells and 400 synapses in five auditory regions up to the directional sensor at the level of the auditory midbrain. It displays the employed parameter sets and contains basic hints for the set up and configuration of the simulation environment. Chapter 5: consists of three sections, whereas the first one describes the set up and results of the own electro-physiological experiments. The second describes the results of 104 model simulations, performed to test the models ability to duplicate psycho-acoustical effects like the precedence effect. Finally, the last section of this chapter contains the results of 54 real world experiments using natural sound signals, recorded under normal as well as highly reverberating conditions. Chapter 6: compares the achieved results to other biologically motivated and technical models for echo suppression and sound source localization and introduces the current status of silicon implementation. Chapter 7: finally provides a short summary and an outlook toward future research subjects and areas of investigation. This thesis aims to contribute to the field of computational neuroscience by bridging the gap between biological investigation, computational modeling and silicon engineering in a specific field of application. It suggests a new spatio-temporal paradigm of information processing in order to access the capabilities of biological systems for technical applications

ORIENTING IN 3D SPACE: BEHAVIORAL AND NEUROPHYSIOLOGICAL STUDIES IN BIG BROWN BATS

In their natural environment, animals engage in a wide range of behavioral tasks that require them to orient to stimuli in three-dimensional space, such as navigating around obstacles, reaching for objects and escaping from predators. Echolocating bats, for example, have evolved a high-resolution 3D acoustic orienting system that allows them to localize and track small moving targets in azimuth, elevation and range. The bat’s active control over the features of its echolocation signals contributes directly to the information represented in its sonar receiver, and its adaptive adjustments in sonar signal design provide a window into the acoustic features that are important for different behavioral tasks. When bats inspect sonar objects and require accurate 3D localization of targets, they produce sonar sound groups (SSGs), which are clusters of sonar calls produced at short intervals and flanked by long interval calls. SSGs are hypothesized to enhance the bat’s range resolution, but this hypothesis has not been directly tested. We first, in Chapter 2, provide a comprehensive comparison of SSG production of bats flying in the field and in the lab under different environmental conditions. Further, in Chapter 3, we devise an experiment to specifically compare SSG production under conditions when target motion is predictable and unpredictable, with the latter mimicking natural conditions where bats chase erratically moving prey. Data from both of these studies are consistent with the hypothesis that SSGs improve the bat’s spatio-temporal resolution of target range, and provide a behavioral foundation for the analysis and interpretation of neural recording data in chapters 4 and 6. The complex orienting behaviors exhibited by animals can be understood as a feedback loop between sensing and action. A primary brain structure involved in sensorimotor integration is the midbrain superior colliculus (SC). The SC is a widely studied brain region and has been implicated in species-specific orienting behaviors. However, most studies of the SC have investigated its functional organization using synthetic 2D (azimuth and elevation) stimuli in restrained animals, leaving gaps in our knowledge of how 3D space (azimuth, elevation and distance) is represented in the CNS. In contrast, the representation of stimulus distance in the auditory systems of bats has been widely studied. Almost all of these studies have been conducted in passively listening bats, thus severing the loop between sensing and action and leaving gaps in our knowledge regarding how target distance is represented in the auditory system of actively echolocating bats. In chapters 4, 5 and 6, we attempt to fill gaps in our knowledge by recording from the SC of free flying echolocating bats engaged in a naturalistic navigation task where bats produce SSGs. In chapter 4, we provide a framework to compute time-of-arrival and direction of the instantaneous echo stimuli received at the bats ears. In chapters 5 and 6, we provide an algorithm to classify neural activity in the SC as sensory, sensorimotor and premotor and then compute spatial receptive fields of SC neurons. Our results show that neurons in the SC of the free-flying echolocating bat respond selectively to stimulus azimuth, elevation and range. Importantly, we find that SC neuron response profiles are modulated by the bat’s behavioral state, indicated by the production of SSG. Broadly, we use both behavior and electrophysiology to understand the action-perception loop that supports spatial orientation by echolocation. We believe that the results and methodological advances presented here will open doors to further studies of sensorimotor integration in freely behaving animals

Spiking neural network model of sound localisation using the interaural intensity difference

In this paper, a spiking neural network (SNN) architecture to simulate the sound localization ability of the mammalian auditory pathways using the interaural intensity difference cue is presented. The lateral superior olive was the inspiration for the architecture, which required the integration of an auditory periphery (cochlea) model and a model of the medial nucleus of the trapezoid body. The SNN uses leaky integrateand-fire excitatory and inhibitory spiking neurons, facilitating synapses and receptive fields. Experimentally derived headrelated transfer function (HRTF) acoustical data from adult domestic cats were employed to train and validate the localization ability of the architecture, training used the supervised learning algorithm called the remote supervision method to determine the azimuthal angles. The experimental results demonstrate that the architecture performs best when it is localizing high-frequency sound data in agreement with the biology, and also shows a high degree of robustness when the HRTF acoustical data is corrupted by noise

A spatial contrast retina with on-chip calibration for neuromorphic spike-based AER vision systems

We present a 32 32 pixels contrast retina microchip that provides its output as an address event representation (AER) stream. Spatial contrast is computed as the ratio between pixel photocurrent and a local average between neighboring pixels obtained with a diffuser network. This current-based computation produces an important amount of mismatch between neighboring pixels, because the currents can be as low as a few pico-amperes. Consequently, a compact calibration circuitry has been included to trimm each pixel. Measurements show a reduction in mismatch standard deviation from 57% to 6.6% (indoor light). The paper describes the design of the pixel with its spatial contrast computation and calibration sections. About one third of pixel area is used for a 5-bit calibration circuit. Area of pixel is 58 m 56 m, while its current consumption is about 20 nA at 1-kHz event rate. Extensive experimental results are provided for a prototype fabricated in a standard 0.35- m CMOS process.This work was supported by Spanish Research Grants TIC2003-08164-C03-01 (SAMANTA), TEC2006-11730-C03-01 (SAMANTA-II), and EU grant IST-2001-34124 (CAVIAR). JCS was supported by the I3P program of the Spanish Research Council. RSG was supported by a national grant from the Spanish Ministry of Education and Science.Peer reviewe

A VLSI model of the bat dorsal nucleus of the lateral lemniscus for azimuthal echolocation

Abstract — The dorsal nucleus of the lateral lemniscus (DNLL) is a distinct group of auditory cells that play a strategic role in azimuthal echolocation in the bat. Dominated by EI-type cells that receive excitation from the contralateral ear and inhibition from the ipsilateral ear, the DNLL processes interaural level difference (ILD) information by integrating inputs from lower brainstem areas and projecting its outputs to the midbrain. In this paper, we propose a two layer recurrent spiking neural network model that simulates ILD processing by the DNLL, and present a VLSI implementation using the address-event representation (AER) protocol. We demonstrate, using this neuromorphic VLSI-based hardware system, that long-lasting inhibition in the DNLL can alter its spatial selectivity to multiple sounds (objects). I