A modified model for the Lobula Giant Movement Detector and its FPGA implementation

The Lobula Giant Movement Detector (LGMD) is a wide-field visual neuron located in the Lobula layer of the Locust nervous system. The LGMD increases its firing rate in response to both the velocity of an approaching object and the proximity of this object. It has been found that it can respond to looming stimuli very quickly and trigger avoidance reactions. It has been successfully applied in visual collision avoidance systems for vehicles and robots. This paper introduces a modified neural model for LGMD that provides additional depth direction information for the movement. The proposed model retains the simplicity of the previous model by adding only a few new cells. It has been simplified and implemented on a Field Programmable Gate Array (FPGA), taking advantage of the inherent parallelism exhibited by the LGMD, and tested on real-time video streams. Experimental results demonstrate the effectiveness as a fast motion detector

Local learning algorithms for stochastic spiking neural networks

This dissertation focuses on the development of machine learning algorithms for spiking neural networks, with an emphasis on local three-factor learning rules that are in keeping with the constraints imposed by current neuromorphic hardware. Spiking neural networks (SNNs) are an alternative to artificial neural networks (ANNs) that follow a similar graphical structure but use a processing paradigm more closely modeled after the biological brain in an effort to harness its low power processing capability. SNNs use an event based processing scheme which leads to significant power savings when implemented in dedicated neuromorphic hardware such as Intel’s Loihi chip. This work is distinguished by the consideration of stochastic SNNs based on spiking neurons that employ a stochastic spiking process, implementing generalized linear models (GLM) rather than deterministic thresholded spiking. In this framework, the spiking signals are random variables which may be sampled from a distribution defined by the neurons. The spiking signals may be observed or latent variables, with neurons whose outputs are observed termed visible neurons and otherwise termed hidden neurons. This choice provides a strong mathematical basis for maximum likelihood optimization of the network parameters via stochastic gradient descent, avoiding the issue of gradient backpropagation through the discontinuity created by the spiking process. Three machine learning algorithms are developed for stochastic SNNs with a focus on power efficiency, learning efficiency and model adaptability; characteristics that are valuable in resource constrained settings. They are studied in the context of applications where low power learning on the edge is key. All of the learning rules that are derived include only local variables along with a global learning signal, making these algorithms tractable to implementation in current neuromorphic hardware. First, a stochastic SNN that includes only visible neurons, the simplest case for probabilistic optimization, is considered. A policy gradient reinforcement learning (RL) algorithm is developed in which the stochastic SNN defines the policy, or state-action distribution, of an RL agent. Action choices are sampled directly from the policy by interpreting the outputs of the read-out neurons using a first to spike decision rule. This study highlights the power efficiency of the SNN in terms of spike frequency. Next, an online meta-learning framework is proposed with the goal of progressively improving the learning efficiency of an SNN over a stream of tasks. In this setting, SNNs including both hidden and visible neurons are considered, posing a more complex maximum likelihood learning problem that is solved using a variational learning method. The meta-learning rule yields a hyperparameter initialization for SNN models that supports fast adaptation of the model to individualized data on edge devices. Finally, moving away from the supervised learning paradigm, a hybrid adver-sarial training framework for SNNs, termed SpikeGAN, is developed. Rather than optimize for the likelihood of target spike patterns at the SNN outputs, the training is mediated by an auxiliary discriminator that provides a measure of how similar the spiking data is to a target distribution. Because no direct spiking patterns are given, the SNNs considered in adversarial learning include only hidden neurons. A Bayesian adaptation of the SpikeGAN learning rule is developed to broaden the range of temporal data that a single SpikeGAN can estimate. Additionally, the online meta-learning rule is extended to include meta-learning for SpikeGAN, to enable efficient generation of data from sequential data distributions

Attention and Prediction-Guided Motion Detection for Low-Contrast Small Moving Targets

Small target motion detection within complex natural environments is an extremely challenging task for autonomous robots. Surprisingly, the visual systems of insects have evolved to be highly efficient in detecting mates and tracking prey, even though targets occupy as small as a few degrees of their visual fields. The excellent sensitivity to small target motion relies on a class of specialized neurons called small target motion detectors (STMDs). However, existing STMD-based models are heavily dependent on visual contrast and perform poorly in complex natural environments where small targets generally exhibit extremely low contrast against neighbouring backgrounds. In this paper, we develop an attention and prediction guided visual system to overcome this limitation. The developed visual system comprises three main subsystems, namely, an attention module, an STMD-based neural network, and a prediction module. The attention module searches for potential small targets in the predicted areas of the input image and enhances their contrast against complex background. The STMD-based neural network receives the contrast-enhanced image and discriminates small moving targets from background false positives. The prediction module foresees future positions of the detected targets and generates a prediction map for the attention module. The three subsystems are connected in a recurrent architecture allowing information to be processed sequentially to activate specific areas for small target detection. Extensive experiments on synthetic and real-world datasets demonstrate the effectiveness and superiority of the proposed visual system for detecting small, low-contrast moving targets against complex natural environments

Biomimetic set up for chemosensor-based machine olfaction

The thesis falls into the field of machine olfaction and accompanying experimental set up for chemical gas sensing. Perhaps more than any other sensory modality, chemical sensing faces with major technical and conceptual challenges: low specificity, slow response time, long term instability, power consumption, portability, coding capacity and robustness. There is an important trend of the last decade pushing artificial olfaction to mimic the biological olfaction system of insects and mammalians. The designers of machine olfaction devices take inspiration from the biological olfactory system, because animals effortlessly accomplish some of the unsolved problems in machine olfaction. In a remarkable example of an olfactory guided behavior, male moths navigate over large distances in order to locate calling females by detecting pheromone signals both rapidly and robustly. The biomimetic chemical sensing aims to identify the key blocks in the olfactory pathways at all levels from the olfactory receptors to the central nervous system, and simulate to some extent the operation of these blocks, that would allow to approach the sensing performance known in biological olfactory system of animals. New technical requirements arise to the hardware and software equipment used in such machine olfaction experiments. This work explores the bioinspired approach to machine olfaction in depth on the technological side. At the hardware level, the embedded computer is assembled, being the core part of the experimental set up. The embedded computer is interfaced with two main biomimetic modules designed by the collaborators: a large-scale sensor array for emulation of the population of the olfactory receptors, and a mobile robotic platform for autonomous experiments for guiding olfactory behaviour. At the software level, the software development kit is designed to host the neuromorphic models of the collaborators for processing the sensory inputs as in the olfactory pathway. Virtualization of the set up was one of the key engineering solutions in the development. Being a device, the set up is transformed to a virtual system for running data simulations, where the software environment is essentially the same, and the real sensors are replaced by the virtual sensors coming from especially designed data simulation tool. The proposed abstraction of the set up results in an ecosystem containing both the models of the olfactory system and the virtual array. This ecosystem can loaded from the developed system image on any personal computer. In addition to the engineering products released in the course of thesis, the scientific results have been published in three journal articles, two book chapters and conference proceedings. The main results on validation of the set up under the scenario of robotic odour localization are reported in the book chapters. The series of three journal articles covers the work on the data simulation tool for machine olfaction: the novel model of drift, the models to simulate the sensor array data based on the reference data set, and the parametrized simulated data and benchmarks proposed for the first time in machine olfaction. This thesis ends up with a solid foundation for the research in biomimetic simulations and algorithms on machine olfaction. The results achieved in the thesis are expected to give rise to new bioinspired applications in machine olfaction, which could have a significant impact in the biomedical engineering research area.Esta tesis se enmarca en el campo de bioingeneria, mas particularmente en la configuración de un sistema experimental de sensores de gases químicos. Quizás más que en cualquier otra modalidad de sensores, los sensores químicos representan un conjunto de retos técnicos y conceptuales ya que deben lidiar con problemas como su baja especificidad, su respuesta temporal lenta, su inestabilidad a largo plazo, su alto consumo enérgético, su portabilidad, así como la necesidad de un sistema de datos y código robusto. En la última década, se ha observado una clara tendencia por parte de los sistemas de machine olfaction hacia la imitación del sistema de olfato biológico de insectos y mamíferos. Los diseñadores de estos sistemas se inspiran del sistema olfativo biológico, ya que los animales cumplen, sin apenas esfuerzo, algunos de los escenarios no resueltos en machine olfaction. Por ejemplo, las polillas machos recorren largas distancias para localizar las polillas hembra, detectando sus feromonas de forma rápida y robusta. La detección biomimética de gases químicos tiene como objetivo identificar los elementos fundamentales de la vía olfativa a todos los niveles, desde los receptores olfativos hasta el sistema nervioso central, y simular, en cierta medida, el funcionamiento de estos bloques, lo que permitiría acercar el rendimiento de la detección al rendimiento de los sistemas olfativos conociodos de los animales. Esto conlleva nuevos requisitos técnicos a nivel de equipamiento tanto hardware como software utilizado en este tipo de experimentos de machine olfaction. Este trabajo propone un enfoque bioinspirado para la ¿machine olfaction¿, explorando a fondo la parte tecnológica. A nivel hardware, un ordenador embedido se ha ensamblado, siendo ésta la parte más importante de la configuración experimental. Este ordenador integrado está interconectado con dos módulos principales biomiméticos diseñados por los colaboradores: una matriz de sensores a gran escala y una plataforma móvil robotizada para experimentos autónomos. A nivel software, el kit de desarrollo software se ha diseñado para recoger los modelos neuromórficos de los colaboradores para el procesamiento de las entradas sensoriales como en la vía olfativa biológica. La virtualización del sistema fue una de las soluciones ingenieriles clave de su desarrollo. Al ser un dispositivo, el sistema se ha transformado en un sistema virtual para la realización de simulaciones de datos, donde el entorno de software es esencialmente el mismo, y donde los sensores reales se sustituyen por sensores virtuales procedentes de una herramienta de simulación de datos especialmente diseñada. La propuesta de abstracción del sistema resulta en un ecosistema que contiene tanto los modelos del sistema olfativo como la matriz virtual . Este ecosistema se puede cargar en cualquier ordenador personal como una imagen del sistema desarrollado. Además de los productos de ingeniería entregados en esta tesis, los resultados científicos se han publicado en tres artículos en revistas, dos capítulos de libros y los proceedings de dos conferencias internacionales. Los principales resultados en la validación del sistema en el escenario de la localización robótica de olores se presentan en los capítulos del libro. Los tres artículos de revistas abarcan el trabajo en la herramienta de simulación de datos para machine olfaction: el novedoso modelo de drift, los modelos para simular la matriz de sensores basado en el conjunto de datos de referencia, y la parametrización de los datos simulados y los benchmarks propuestos por primera vez en machine olfaction. Esta tesis ofrece una base sólida para la investigación en simulaciones biomiméticas y en algoritmos en machine olfaction. Los resultados obtenidos en la tesis pretenden dar lugar a nuevas aplicaciones bioinspiradas en machine olfaction, lo que podría tener un significativo impacto en el área de investigación en ingeniería biomédic

NeuroEditor: a tool to edit and visualize neuronal morphologies

The digital extraction of detailed neuronal morphologies from microscopy data is an essential step in the study of neurons. Ever since Cajal’s work, the acquisition and analysis of neuron anatomy has yielded invaluable insight into the nervous system, which has led to our present understanding of many structural and functional aspects of the brain and the nervous system, well beyond the anatomical perspective. Obtaining detailed anatomical data, though, is not a simple task. Despite recent progress, acquiring neuron details still involves using labor-intensive, error prone methods that facilitate the introduction of inaccuracies and mistakes. In consequence, getting reliable morphological tracings usually needs the completion of post-processing steps that require user intervention to ensure the extracted data accuracy. Within this framework, this paper presents NeuroEditor, a new software tool for visualization, editing and correction of previously reconstructed neuronal tracings. This tool has been developed specifically for alleviating the burden associated with the acquisition of detailed morphologies. NeuroEditor offers a set of algorithms that can automatically detect the presence of potential errors in tracings. The tool facilitates users to explore an error with a simple mouse click so that it can be corrected manually or, where applicable, automatically. In some cases, this tool can also propose a set of actions to automatically correct a particular type of error. Additionally, this tool allows users to visualize and compare the original and modified tracings, also providing a 3D mesh that approximates the neuronal membrane. The approximation of this mesh is computed and recomputed on-the-fly, reflecting any instantaneous changes during the tracing process. Moreover, NeuroEditor can be easily extended by users, who can program their own algorithms in Python and run them within the tool. Last, this paper includes an example showing how users can easily define a customized workflow by applying a sequence of editing operations. The edited morphology can then be stored, together with the corresponding 3D mesh that approximates the neuronal membrane

Cognitive Algorithms and digitized Tissue – based Diagnosis

Aims: To analyze the nature and impact of cognitive algorithms and programming on digitized tissue – based diagnosis. Definitions: Digitized tissue – based diagnosis includes all computerized tissue investigations that contribute to the most appropriate description and forecast of the actual patient’s disease [1]. Cognitive algorithms are programs that encompass machine learning, reasoning, and human – computer interaction [2]. Theoretical considerations: Digitized blood data, objective clinical findings, microscopic, gross, radiological images and gene alterations are analyzed by specialized image analysis methods, and transferred in numbers and vectors. These are analyzed by statistical procedures. They include higher order statistics such as multivariate analysis, neural networks and ‘black box’ strategies, for example ‘deep learning’ or ‘Watson’ approaches. These algorithms can be applied at different cognitive ‘levels’, to reach a digital decision for different procedures which should assist the patient’s health condition. These levels can be grouped in self learning, self promoting, self targeting, and self exploring algorithms. Each of them requires a memory and neighbourhood condition. Self targeting and exploring algorithms are circumscribed mechanisms with singularities and repair procedures. They develop self recognition.   Consecutives: Medical doctors including pathologists are commonly not trained to understand the basic principles and workflow of applied or potential future procedures. At present, basic medical data only serve for simple cognitive algorithms. Most of the investigations focus on ‘deep learning’ procedures. The applied learning and decision algorithms might be modified and themselves be used for ‘next order cognitive algorithms’. Such systems will develop their own strategies, and become independent from potential human interactions. The basic strategy of such IT systems is described herein. Perspectives: Medical doctors including pathologists should be aware about the abilities to enhance their work by supporting tools. In some case the users may not be able to fully understand these tools. Furthermore, these tools will probably become self learning, and, therefore, seem to propose the daily workflow probably without any medical control or even interaction

Vision-based guidance and control of a hovering vehicle in unknown environments

Thesis (S.M.)--Massachusetts Institute of Technology, Dept. of Mechanical Engineering, 2008.Includes bibliographical references (leaves 115-122).This thesis presents a methodology, architecture, hardware implementation, and results of a system capable of controlling and guiding a hovering vehicle in unknown environments, emphasizing cluttered indoor spaces. Six-axis inertial data and a low-resolution onboard camera yield sufficient information for image processing, Kalman filtering, and novel mapping algorithms to generate a, high-performance estimate of vehicle motion, as well as an accurate three-dimensional map of the environment. This combination of mapping and localization enables a quadrotor vehicle to autonomously navigate cluttered, unknown environments safely. Communication limitations are considered, and a hybrid control architecture is presented to demonstrate the feasibility of combining separated proactive offboard and reactive onboard planners simultaneously, including a detailed presentation of a novel reactive obstacle avoidance algorithm and preliminary results integrating the MIT Darpa Urban Challenge planner for high-level control. The RAVEN testbed is successfully employed as a prototyping facility for rapid development of these algorithms using emulated inertial data and offboard processing as a precursor to embedded development. An analysis of computational demand and a comparison of the emulated inertial system to an embedded sensor package demonstrates the feasibility of porting the onboard algorithms to an embedded autopilot. Finally, flight results using only the single camera and emulated inertial data for closed-loop trajectory following, environment mapping, and obstacle avoidance are presented and discussed.by Spencer Greg Ahrens.S.M

A bio-inspired computational model for motion detection

Tese de Doutoramento (Programa Doutoral em Engenharia Biomédica)Last years have witnessed a considerable interest in research dedicated to show that solutions to challenges in autonomous robot navigation can be found by taking inspiration from biology. Despite their small size and relatively simple nervous systems, insects have evolved vision systems able to perform the computations required for a safe navigation in dynamic and unstructured environments, by using simple, elegant and computationally efficient strategies. Thus, invertebrate neuroscience provides engineers with many neural circuit diagrams that can potentially be used to solve complicated engineering control problems. One major and yet unsolved problem encountered by visually guided robotic platforms is collision avoidance in complex, dynamic and inconstant light environments. In this dissertation, the main aim is to draw inspiration from recent and future findings on insect’s collision avoidance in dynamic environments and on visual strategies of light adaptation applied by diurnal insects, to develop a computationally efficient model for robotic control, able to work even in adverse light conditions. We first present a comparative analysis of three leading collision avoidance models based on a neural pathway responsible for signing collisions, the Lobula Giant Movement Detector/Desceding Contralateral Movement Detector (LGMD/DCMD), found in the locust visual system. Models are described, simulated and results are compared with biological data from literature. Due to the lack of information related to the way this collision detection neuron deals with dynamic environments, new visual stimuli were developed. Locusts Lo- custa Migratoria were stimulated with computer-generated discs that traveled along a combination of non-colliding and colliding trajectories, placed over a static and two distinct moving backgrounds, while simultaneously recording the DCMD activity extracellularly. Based on these results, an innovative model was developed. This model was tested in specially designed computer simulations, replicating the same visual conditions used for the biological recordings. The proposed model is shown to be sufficient to give rise to experimentally observed neural insect responses. Using a different approach, and based on recent findings, we present a direct approach to estimate potential collisions through a sequential computation of the image’s power spectra. This approach has been implemented in a real robotic platform, showing that distant dependent variations on image statistics are likely to be functional significant. Maintaining the collision detection performance at lower light levels is not a trivial task. Nevertheless, some insect visual systems have developed several strategies to help them to optimize visual performance over a wide range of light intensities. In this dissertation we address the neural adaptation mechanisms responsible to improve light capture on a day active insect, the bumblebee Bombus Terrestris. Behavioral analyses enabled us to investigate and infer about the spatial and temporal neural summation extent applied by those insects to improve image reliability at the different light levels. As future work, the collision avoidance model may be coupled with a bio-inspired light adaptation mechanism and used for robotic autonomous navigation.Os últimos anos têm testemunhado um aumento progressivo da investigação dedicada a demonstrar que possíveis soluções, para problemas existentes na navegação autónoma de robôs, podem ser encontradas buscando inspiração na biologia. Apesar do reduzido tamanho e da simplicidade do seu sistema nervoso, os insectos possuem sistemas de visão capazes de realizar os cálculos necessários para uma navegação segura em ambientes dinâmicos e não estruturados, por meio de estratégias simples, elegantes e computacionalmente eficientes. Assim, a área da neurociência que se debruça sobre o estudo dos invertebrados fornece, à area da engenharia, uma vasta gama de diagramas de circuitos neurais, que podem ser usados como base para a resolução de problemas complexos. Um atual e notável problema, cujas plataformas robóticas baseadas em sistemas de visão estão sujeitas, é o problema de deteção de colisões em ambientes complexos, dinâmicos e de intensidade luminosa variável. Assim, o objetivo principal do trabalho aqui apresentado é o de procurar inspiração em recentes e futuras descobertas relacionadas com os mecanismos que possibilitam a deteção de colisões em ambientes dinâmicos, bem como nas estratégias visuais de adaptação à luz, aplicadas por insectos diurnos. Numa primeira abordagem é feita uma análise comparativa dos três principais modelos, propostos na literatura, de deteção de colisões, que têm por base o funcionamento dos neurónios Lobular Gigante Detector de Movimento/ Detector de Movimento Descendente Contralateral (LGMD / DCMD), que fazem parte do sistema visual do gafanhoto. Os modelos são descritos, simulados e os resultados são comparados com os dados biológicos existentes, descritos na literatura. Devido à falta de informação relacionada com a forma como estes neurónios detectores de colisões lidam com ambientes dinâmicos, foram desenvolvidos novos estímulos visuais. A estimulação de gafanhotos Locusta Migratoria foi realizada usando-se estímulos controlados, gerados por computador, efectuando diferentes combinações de trajectórias de não-colisão e colisão, colocados sobre um fundo estático e dois fundos dinâmicos. extracelulares do neurónio DCMD. Com base nos resultados obtidos foi possível desenvolver um modelo inovador. Este foi testado sob estímulos visuais desenvolvidos computacionalmente, recriando as mesmas condições visuais usadas aquando dos registos neuronais biológicos. O modelo proposto mostrou ser capaz de reproduzir os resultados neuronais dos gafanhotos, experimentalmente obtidos. Usando uma abordagem diferente, e com base em descobertas recentes, apresentamos uma metodologia mais direta, que possibilita estimar possíveis colisões através de cálculos sequenciais dos espetros de potência das imagens captadas. Esta abordagem foi implementada numa plataforma robótica real, mostrando que, variações estatísticas nas imagens captadas, são susceptíveis de serem funcionalmente significativas. Manter o desempenho da deteção de colisões, em níveis de luz reduzida, não é uma tarefa trivial. No entanto, alguns sistemas visuais de insectos desenvolveram estratégias de forma a optimizar o seu desempenho visual numa larga gama de intensidades luminosas. Nesta dissertação, os mecanismos de adaptação neuronais, responsáveis pela melhoraria de captação de luz num inseto diurno, a abelha Bombus Terrestris, serviram como uma base de estudo. Adaptando análises comportamentais, foi-nos permitido investigar e inferir acerca da extensão dos somatórios neuronais, espaciais e temporais, aplicados por estes insetos, por forma a melhorar a qualidade das imagens captadas a diferentes níveis de luz. Como trabalho futuro, o modelo de deteção de colisões deverá ser acoplado com um mecanismo de adaptação à luz, sendo ambos bio-inspirados, e que possam ser utilizados na navegação robótica autónoma

2019 EURēCA Abstract Book

Listing of student participant abstracts