Biologically inspired analog IC for visual collision detection

Journal ArticleWe have designed and tested a single-chip analog VLSI sensor that detects imminent collisions by measuring radially expanding optic flow. The design of the chip is based on a model proposed to explain leg-extension behavior in flies during landing approaches. We evaluated a detailed version of this model in simulation using a library of 50 test movies taken through a fisheye lens. The algorithm was evaluated on its ability to distinguish movies ending in collisions from movies in which no collision occurred. This biologically inspired algorithm is capable of 94% correct performance in this task using an ultra-low-resolution (132-pixel) image as input. A new elementary motion detector (EMD) circuit was developed to measure optic flow on a CMOS focal-plane sensor. This EMD circuit models the bandpass nature of large monopolar cells (LMCs) immediately postsynaptic to photoreceptors in the fly visual system as well as a saturating multiplication operation proposed for Reichart-type motion detectors. A 16 x 16 array of two-dimensional motion detectors was fabricated in a standard 0.5µm CMOS process. The chip consumes 140 µW of power from a 5 V supply. With the addition of wide-angle optics, the sensor is able to detect collisions 100-400 ms before impact in complex, real-world scenes. Index Terms-CMOS imager, collision detection, Gilbert multiplier, insect vision, neuromorphic systems, optic flow, smart sensor

Low-power analog VLSI visual collision detector

Journal ArticleWe have designed and tested a single-chip analog VLSI sensor that detects imminent collisions by measuring radially expansive optic flow. The design of the chip is based on a model proposed to explain leg-extension behavior in flies during landing approaches. A new elementary motion detector (EMD) circuit was developed to measure optic flow. This EMD circuit models the bandpass nature of large monopolar cells (LMCs) immediately postsynaptic to photoreceptors in the fly visual system. A 16 × 16 array of 2-D motion detectors was fabricated on a 2.24 mm × 2.24 mm die in a standard 0.5-μm CMOS process. The chip consumes 140 μW of power from a 5 V supply. With the addition of wide-angle optics, the sensor is able to detect collisions around 500 ms before impact in complex, real-world scenes

A bio-inspired computational model for motion detection

Tese de Doutoramento (Programa Doutoral em Engenharia Biomédica)Last years have witnessed a considerable interest in research dedicated to show that solutions to challenges in autonomous robot navigation can be found by taking inspiration from biology. Despite their small size and relatively simple nervous systems, insects have evolved vision systems able to perform the computations required for a safe navigation in dynamic and unstructured environments, by using simple, elegant and computationally efficient strategies. Thus, invertebrate neuroscience provides engineers with many neural circuit diagrams that can potentially be used to solve complicated engineering control problems. One major and yet unsolved problem encountered by visually guided robotic platforms is collision avoidance in complex, dynamic and inconstant light environments. In this dissertation, the main aim is to draw inspiration from recent and future findings on insect’s collision avoidance in dynamic environments and on visual strategies of light adaptation applied by diurnal insects, to develop a computationally efficient model for robotic control, able to work even in adverse light conditions. We first present a comparative analysis of three leading collision avoidance models based on a neural pathway responsible for signing collisions, the Lobula Giant Movement Detector/Desceding Contralateral Movement Detector (LGMD/DCMD), found in the locust visual system. Models are described, simulated and results are compared with biological data from literature. Due to the lack of information related to the way this collision detection neuron deals with dynamic environments, new visual stimuli were developed. Locusts Lo- custa Migratoria were stimulated with computer-generated discs that traveled along a combination of non-colliding and colliding trajectories, placed over a static and two distinct moving backgrounds, while simultaneously recording the DCMD activity extracellularly. Based on these results, an innovative model was developed. This model was tested in specially designed computer simulations, replicating the same visual conditions used for the biological recordings. The proposed model is shown to be sufficient to give rise to experimentally observed neural insect responses. Using a different approach, and based on recent findings, we present a direct approach to estimate potential collisions through a sequential computation of the image’s power spectra. This approach has been implemented in a real robotic platform, showing that distant dependent variations on image statistics are likely to be functional significant. Maintaining the collision detection performance at lower light levels is not a trivial task. Nevertheless, some insect visual systems have developed several strategies to help them to optimize visual performance over a wide range of light intensities. In this dissertation we address the neural adaptation mechanisms responsible to improve light capture on a day active insect, the bumblebee Bombus Terrestris. Behavioral analyses enabled us to investigate and infer about the spatial and temporal neural summation extent applied by those insects to improve image reliability at the different light levels. As future work, the collision avoidance model may be coupled with a bio-inspired light adaptation mechanism and used for robotic autonomous navigation.Os últimos anos têm testemunhado um aumento progressivo da investigação dedicada a demonstrar que possíveis soluções, para problemas existentes na navegação autónoma de robôs, podem ser encontradas buscando inspiração na biologia. Apesar do reduzido tamanho e da simplicidade do seu sistema nervoso, os insectos possuem sistemas de visão capazes de realizar os cálculos necessários para uma navegação segura em ambientes dinâmicos e não estruturados, por meio de estratégias simples, elegantes e computacionalmente eficientes. Assim, a área da neurociência que se debruça sobre o estudo dos invertebrados fornece, à area da engenharia, uma vasta gama de diagramas de circuitos neurais, que podem ser usados como base para a resolução de problemas complexos. Um atual e notável problema, cujas plataformas robóticas baseadas em sistemas de visão estão sujeitas, é o problema de deteção de colisões em ambientes complexos, dinâmicos e de intensidade luminosa variável. Assim, o objetivo principal do trabalho aqui apresentado é o de procurar inspiração em recentes e futuras descobertas relacionadas com os mecanismos que possibilitam a deteção de colisões em ambientes dinâmicos, bem como nas estratégias visuais de adaptação à luz, aplicadas por insectos diurnos. Numa primeira abordagem é feita uma análise comparativa dos três principais modelos, propostos na literatura, de deteção de colisões, que têm por base o funcionamento dos neurónios Lobular Gigante Detector de Movimento/ Detector de Movimento Descendente Contralateral (LGMD / DCMD), que fazem parte do sistema visual do gafanhoto. Os modelos são descritos, simulados e os resultados são comparados com os dados biológicos existentes, descritos na literatura. Devido à falta de informação relacionada com a forma como estes neurónios detectores de colisões lidam com ambientes dinâmicos, foram desenvolvidos novos estímulos visuais. A estimulação de gafanhotos Locusta Migratoria foi realizada usando-se estímulos controlados, gerados por computador, efectuando diferentes combinações de trajectórias de não-colisão e colisão, colocados sobre um fundo estático e dois fundos dinâmicos. extracelulares do neurónio DCMD. Com base nos resultados obtidos foi possível desenvolver um modelo inovador. Este foi testado sob estímulos visuais desenvolvidos computacionalmente, recriando as mesmas condições visuais usadas aquando dos registos neuronais biológicos. O modelo proposto mostrou ser capaz de reproduzir os resultados neuronais dos gafanhotos, experimentalmente obtidos. Usando uma abordagem diferente, e com base em descobertas recentes, apresentamos uma metodologia mais direta, que possibilita estimar possíveis colisões através de cálculos sequenciais dos espetros de potência das imagens captadas. Esta abordagem foi implementada numa plataforma robótica real, mostrando que, variações estatísticas nas imagens captadas, são susceptíveis de serem funcionalmente significativas. Manter o desempenho da deteção de colisões, em níveis de luz reduzida, não é uma tarefa trivial. No entanto, alguns sistemas visuais de insectos desenvolveram estratégias de forma a optimizar o seu desempenho visual numa larga gama de intensidades luminosas. Nesta dissertação, os mecanismos de adaptação neuronais, responsáveis pela melhoraria de captação de luz num inseto diurno, a abelha Bombus Terrestris, serviram como uma base de estudo. Adaptando análises comportamentais, foi-nos permitido investigar e inferir acerca da extensão dos somatórios neuronais, espaciais e temporais, aplicados por estes insetos, por forma a melhorar a qualidade das imagens captadas a diferentes níveis de luz. Como trabalho futuro, o modelo de deteção de colisões deverá ser acoplado com um mecanismo de adaptação à luz, sendo ambos bio-inspirados, e que possam ser utilizados na navegação robótica autónoma

Processing of sky compass cues and wide-field motion in the central complex of the desert locust (Schistocerca gregaria)

1. Polarization-sensitive neurons of the locust central complex show azimuthdependent responses to unpolarized light spots. This suggests that direct sunlight supports the sky polarization compass in this brain area. / 2. In the brain of the desert locust, neurons sensitive to the plane of celestial polarization are arranged like a compass in the slices of the central complex. These neurons, in addition, code for the horizontal direction of an unpolarized light cue possibly representing the sun. We show here that horizontal directions are, in addition to E-vector orientations from dorsal direction, represented in a compass-like manner across the slices of the central complex. However, both compasses are not linked to each other but seem to interact in a cell specific nonlinear way. Our study confirms the role of the central complex in signaling heading directions signaling and shows that different cues are employed for this task. / 3. Visual cues are essential for animal navigation and spatial orientation. Many insects rely on celestial cues for spatial orientation, including the sky polarization pattern. In desert locusts neurons encoding the plane of polarized light (E-vector) are located in the central complex (CX), a group of midline-spanning neuropils. Several types of CX neuron signalling heading direction represent zenithal Evectors in a topographic manner across the slices of the CX and, likely, act as an internal sky compass. Because animals experience optic flow stimulation during flight, we asked whether progressive wide-field motion affects the responses of CX neurons to polarized light. In most neurons, progressive motion disadapted the response to the preferred E-vector (i.e. the E-vector eliciting strongest firing), whereas the response to the anti-preferred E-vector remained comparatively unaffected. This suggests context-dependent gain modulation in sky compass signalling. Three types of compass neuron were responsive to motion simulating body rotation around the yaw axis. Depending on arborization domains in the CX and rotation direction these neurons were strongly excited or inhibited. As proposed for Drosophila, they may be involved in shifting compass signal activity across the slices of the CX as the animal turns enabling it to keep track of its heading

Processing of sky compass cues and wide-field motion in the central complex of the desert locust (Schistocerca gregaria)

1. Polarization-sensitive neurons of the locust central complex show azimuthdependent responses to unpolarized light spots. This suggests that direct sunlight supports the sky polarization compass in this brain area. / 2. In the brain of the desert locust, neurons sensitive to the plane of celestial polarization are arranged like a compass in the slices of the central complex. These neurons, in addition, code for the horizontal direction of an unpolarized light cue possibly representing the sun. We show here that horizontal directions are, in addition to E-vector orientations from dorsal direction, represented in a compass-like manner across the slices of the central complex. However, both compasses are not linked to each other but seem to interact in a cell specific nonlinear way. Our study confirms the role of the central complex in signaling heading directions signaling and shows that different cues are employed for this task. / 3. Visual cues are essential for animal navigation and spatial orientation. Many insects rely on celestial cues for spatial orientation, including the sky polarization pattern. In desert locusts neurons encoding the plane of polarized light (E-vector) are located in the central complex (CX), a group of midline-spanning neuropils. Several types of CX neuron signalling heading direction represent zenithal Evectors in a topographic manner across the slices of the CX and, likely, act as an internal sky compass. Because animals experience optic flow stimulation during flight, we asked whether progressive wide-field motion affects the responses of CX neurons to polarized light. In most neurons, progressive motion disadapted the response to the preferred E-vector (i.e. the E-vector eliciting strongest firing), whereas the response to the anti-preferred E-vector remained comparatively unaffected. This suggests context-dependent gain modulation in sky compass signalling. Three types of compass neuron were responsive to motion simulating body rotation around the yaw axis. Depending on arborization domains in the CX and rotation direction these neurons were strongly excited or inhibited. As proposed for Drosophila, they may be involved in shifting compass signal activity across the slices of the CX as the animal turns enabling it to keep track of its heading

Computation of Object Approach by a Wide-Field, Motion-Sensitive Neuron

The lobula giant motion detector (LGMD) in the locust visual system is a wide-field, motion-sensitive neuron that responds vigorously to objects approaching the animal on a collision course. We investigated the computation performed by LGMD when it responds to approaching objects by recording the activity of its postsynaptic target, the descending contralateral motion detector (DCMD). In each animal, peak DCMD activity occurred a fixed delay δ (15 ≤ δ ≤ 35 msec) after the approaching object had reached a specific angular threshold θthres on the retina (15° ≤ θthres ≤ 40°). θthres was independent of the size or velocity of the approaching object. This angular threshold computation was quite accurate: the error of LGMD and DCMD in estimating θthres(3.1–11.9°) corresponds to the angular separation between two and six ommatidia at each edge of the expanding object on the locust retina. It was also resistant to large amplitude changes in background luminosity, contrast, and body temperature. Using several experimentally derived assumptions, the firing rate of LGMD and DCMD could be shown to depend on the product ψ(t − δ) · e^(−αθ(t−δ0)), where θ(t) is the angular size subtended by the object during approach, ψ(t) is the angular edge velocity of the object and the constant, and α is related to the angular threshold size [α = 1/tan(θthres^(/2))]. Because LGMD appears to receive distinct input projections, respectively motion- and size-sensitive, this result suggests that a multiplication operation is implemented by LGMD. Thus, LGMD might be an ideal model to investigate the biophysical implementation of a multiplication operation by single neurons

Signal transformation at the input and output of the Drosophila visual system

A key function of the nervous system is to sample data from the external world, generate internal signals, and transform them into meaningful information that can be used to trigger behaviour. In order to gain insight into the underlying mechanism for signal transformation, the visual system has been extensively studied: partly owing to the stimulus being reliably presentable, and the anatomy being well described. The Drosophila visual system is one such system, with the added advantage of genetic tractability. In this thesis, I studied the filtering property of visual neurons at two levels, biophysical and circuit levels. The first study looks at signal transformation at the biophysical level, at the input of the visual system, in photoreceptors. Voltage-gated potassium channels counteract the depolarization caused by opening of light sensitive channels, and the heterogeneous properties of their kinetics can fine-tune the photoreceptor’s frequency response to fulfill the animal’s ecological requirements. Shaker (Kv1) and Shab (Kv2) have been identified as fast and slow inactivating components of the photoreceptor’s outward currents, however a current with intermediate kinetics (IKf) has not been molecularly identified, but had been postulated to be Shal (Kv4). I focused on characterizing this current using whole-cell patch clamp in wild type and mutants, and using antibodies for Shal. My results from whole-cell patch clamp indicated that IKf in adult R1-6 cells are not Shal, from their voltage dependence and insensitivity to a Kv4 blocker. This calls for alternative molecular basis for IKf, which is likely to be a slow inactivating component of Shaker, or a combination of its many splice variants. The second study looks at signal transformation at the circuit level, at the output end, in the third optic neuropil, lobula. Visual projection neurons project from the lobula to the central brain, and have been proposed to carry behaviourally relevant visual features to higher brain regions. It was recently shown that optogenetic activation of individual visual projection neuron types could induce distinct behaviours such as takeoff and backward walking, linking these visual neurons to specific behavioural programs downstream. Using in vivo two-photon calcium imaging, I recorded visually evoked calcium responses from three of these cell types. Cell types that showed induced takeoff and backward walking preferentially responded to dark looming stimuli or fragmented expanding local features, suggesting their role in behaviours triggered by object approach. To explore how this visual information is transformed in the downstream circuit, we identified several candidate neurons that receive input from this cell type by anatomical overlap, and then validated their connections using optogenetic activation and calcium imaging. One downstream cell-type that projects bilaterally had very similar response properties to its upstream partner, whereas another cell-type that projects ipsilaterally seemed to filter out some information from its upstream partner. This is one of the first studies that functionally characterizes lobula visual projection neurons and their downstream partners in Drosophila, and their response properties agree with the general idea that visual information becomes increasingly selective as it is sent to higher brain regions.HHMI-Janelia graduate scholarshi

Novelty detection and context dependent processing of sky-compass cues in the brain of the desert locust Schistocerca gregaria

NERVOUS SYSTEMS facilitate purposeful interactions between animals and their environment, based on the perceptual powers, cognition and higher motor control. Through goal-directed behavior, the animal aims to increase its advantage and minimize risk. For instance, the migratory desert locust should profit from being fast in finding a fresh habitat, thus minimizing the investment of bodily resources in locomotion as well as the risk of starvation or capture by a predator en route. Efficient solutions to this and similar tasks – be it finding your way to work, the daily foraging of worker bees or the seasonal long-range migration of monarch butterflies - strongly depend on spatial orientation in local or global frames of reference. Local settings may include visual landmarks at stable positions that can be mapped onto egocentric space and learned for orientation, e.g. to remember a short route to a source of benefit (e.g. food) that is distant or visually less salient than the landmarks. Compass signals can mediate orientation to a global reference-frame (allothetic orienation), e.g. for locomotion in a particular compass direction or to merely ensure motion along a straight line. Whilst spatial orientation is a prerequisite of doing the planned in such tasks, animal survival in general depends on the ability to adequately respond to the unexpected, i.e. to unpredicted events such as the approach of a predator or mate. The process of identifying relevant events in the outside world that are not predictable from preceding events is termed novelty detection. Yet, the definition of ‘novelty’ is highly contextual: depending on the current situation and goal, some changes may be irrelevant and remain ´undetected´. The present thesis describes neuronal representations of a compass stimulus, correlates of novelty detection and interactions between the two in the minute brain of an insect, the migratory desert locust Schistocerca gregaria. Experiments were carried out in tethered locusts with legs and wings removed. More precisely, adult male subjects in the gregarious phase (see phase theory, Uvarov 1966) that migrates in swarms across territories in North Africa and the Middle East were used. The author performed electrophysiological recordings from single neurons in the locust brain, while either the compass stimulus (Chapter I) or events in the visual scenery (Chapter II) or combinations of both (Chapter III) were being presented to the animal. Injections of a tracer through the recording electrode, visualized by means of fluorescent-dye coupling, allowed the allocation of cellular morphologies to previously described types of neuron or the characterization of novel cell types, respectively. Recordings were focused on cells of the central complex, a higher integration area in the insect brain that was shown to be involved in the visually mediated control of goal-directed locomotion. Experiments delivered insights into how representations of the compass cue are modulated in a manner suited for their integration in the control of goal-directed locomotion. In particular, an interaction between compass-signaling and novelty detection was found, corresponding to a process in which input in one sensory domain (object vision) modulates the processing of concurrent input to a different exteroceptive sensory system (compass sense). In addition to deepening the understanding of the compass network in the locust brain, the results reveal fundamental parallels to higher context-dependent processing of sensory information by the vertebrate cortex, both with respect to spatial cues and novelty detection