TOPOLOGICAL PROPERTIES OF A NETWORK OF SPIKING NEURONS IN FACE IMAGE RECOGNITION

We introduce a novel system for recognition of partially occluded and rotated images. The system is based on a hierarchical network of integrate-and-fire spiking neurons with random synaptic connections and a novel organization process. The network generates integrated output sequences that are used for image classification. The network performed satisfactorily given appropriate topology, i.e. the number of neurons and synaptic connections, which corresponded to the size of input images. Comparison of Synaptic Plasticity Activity Rule (SAPR) and Spike Timing Dependant Plasticity (STDP) rules, used to update connections between the neurons, indicated that the SAPR gave better results and thus was used throughout. Test results showed that the network performed better than Support Vector Machines. We also introduced a stopping criterion based on entropy, which significantly shortened the iterative process while only slightly affecting classification performance

Sincronización de modelos matemáticos de neuronas mediante acoplamientos basados en umbrales

"En el área de sistemas dinámicos, el sistema de Hindmarsh–Rose (HR) es un modelo matemático bien conocido de la actividad eléctrica neuronal. Dicho modelo diseñado a partir de una generalización del modelo de Hodgkin-Huxley (HH) y de FitzHugh-Nagumo (FHN) representa mediante un conjunto de ecuaciones matemáticas el potencial de membrana y las corrientes de iones que ocurren de forma bastante aproximada al comportamiento real de las neuronas activas. Dado que las neuronas forman una amplia y compleja red de transmisión de información y se comunican con otras neuronas por medio de conexiones entre ellas, la sincronización de su actividad no ha sido analizada del todo y resulta un factor fundamental en su estudio y comprensión. Por lo cual, en este trabajo se aborda la sincronización de la dinámica caótica de los modelos matemáticos de dos neuronas de HR a través de un acoplamiento basado en niveles o umbrales de activación previamente definidos. El tipo de acoplamiento utilizado es unidireccional y se realiza mediante una señal subamortiguada adaptable, la cual se activa en cada evento de cruce entre la trayectoria de una neurona asignada como maestra atravesando un valor umbral definido mediante un plano de Poincaré. Se presenta un nuevo sistema acoplado de neuronas maestro-esclavo basado en modelos matemáticos. El tipo de sincronización entre los sistemas se detecta mediante análisis matemáticos y simulaciones numéricas considerando el estudio de la estabilidad de los puntos de equilibrio, el enfoque del sistema auxiliar y los exponentes de Lyapunov para determinar la dinámica caótica que resulta de dichos sistemas. ""In the area of dynamical systems, the Hindmarsh-Rose (HR) system is a well-known mathematical model of neuronal electrical activity. This model designed from a generalization of the models of Hodgkin-Huxley (HH) and FitzHugh-Nagumo (FHN) represents by a set of mathematical equations the membrane potential and the ion currents that occur in a highly approximate way to the actual behavior of active neurons. Since the neurons form a wide and complex network of information transmission and communicate with other neurons through connections between them, the synchronization of their activity is a fundamental factor. Therefore, this work deals with the synchronization of the chaotic dynamics of two Neurons of HR through a link based on previously defined levels or thresholds. The type of coupling used is unidirectional and is made by an adaptive subdampened signal, which is activated in each crossover event between the trajectory of a neuron assigned as a master through a threshold value located by a plane of Poincaré. A new coupled system of master-slave neurons is presented based on mathematical models. The type of synchronization between the systems is detected by mathematical analysis and numerical simulations considering the study of the stability of the equilibrium points, the approach of the auxiliary system and the exponents of Lyapunov to determine the Chaotic dynamics that result from these systems.

Low-power neuromorphic sensor fusion for elderly care

Smart wearable systems have become a necessary part of our daily life with applications ranging from entertainment to healthcare. In the wearable healthcare domain, the development of wearable fall recognition bracelets based on embedded systems is getting considerable attention in the market. However, in embedded low-power scenarios, the sensor’s signal processing has propelled more challenges for the machine learning algorithm. Traditional machine learning method has a huge number of calculations on the data classification, and it is difficult to implement real-time signal processing in low-power embedded systems. In an embedded system, ensuring data classification in a low-power and real-time processing to fuse a variety of sensor signals is a huge challenge. This requires the introduction of neuromorphic computing with software and hardware co-design concept of the system. This thesis is aimed to review various neuromorphic computing algorithms, research hardware circuits feasibility, and then integrate captured sensor data to realise data classification applications. In addition, it has explored a human being benchmark dataset, which is following defined different levels to design the activities classification task. In this study, firstly the data classification algorithm is applied to human movement sensors to validate the neuromorphic computing on human activity recognition tasks. Secondly, a data fusion framework has been presented, it implements multiple-sensing signals to help neuromorphic computing achieve sensor fusion results and improve classification accuracy. Thirdly, an analog circuits module design to carry out a neural network algorithm to achieve low power and real-time processing hardware has been proposed. It shows a hardware/software co-design system to combine the above work. By adopting the multi-sensing signals on the embedded system, the designed software-based feature extraction method will help to fuse various sensors data as an input to help neuromorphic computing hardware. Finally, the results show that the classification accuracy of neuromorphic computing data fusion framework is higher than that of traditional machine learning and deep neural network, which can reach 98.9% accuracy. Moreover, this framework can flexibly combine acquisition hardware signals and is not limited to single sensor data, and can use multi-sensing information to help the algorithm obtain better stability

Linear and nonlinear approaches to unravel dynamics and connectivity in neuronal cultures

[eng] In the present thesis, we propose to explore neuronal circuits at the mesoscale, an approach in which one monitors small populations of few thousand neurons and concentrates in the emergence of collective behavior. In our case, we carried out such an exploration both experimentally and numerically, and by adopting an analysis perspective centered on time series analysis and dynamical systems. Experimentally, we used neuronal cultures and prepared more than 200 of them, which were monitored using fluorescence calcium imaging. By adjusting the experimental conditions, we could set two basic arrangements of neurons, namely homogeneous and aggregated. In the experiments, we carried out two major explorations, namely development and disintegration. In the former we investigated changes in network behavior as it matured; in the latter we applied a drug that reduced neuronal interconnectivity. All the subsequent analyses and modeling along the thesis are based on these experimental data. Numerically, the thesis comprised two aspects. The first one was oriented towards a simulation of neuronal connectivity and dynamics. The second one was oriented towards the development of linear and nonlinear analysis tools to unravel dynamic and connectivity aspects of the measured experimental networks. For the first aspect, we developed a sophisticated software package to simulate single neuronal dynamics using a quadratic integrate–and–fire model with adaptation and depression. This model was plug into a synthetic graph in which the nodes of the network are neurons, and the edges connections. The graph was created using spatial embedding and realistic biology. We carried out hundreds of simulations in which we tuned the density of neurons, their spatial arrangement and the characteristics of the fluorescence signal. As a key result, we observed that homogeneous networks required a substantial number of neurons to fire and exhibit collective dynamics, and that the presence of aggregation significantly reduced the number of required neurons. For the second aspect, data analysis, we analyzed experiments and simulations to tackle three major aspects: network dynamics reconstruction using linear descriptions, dynamics reconstruction using nonlinear descriptors, and the assessment of neuronal connectivity from solely activity data. For the linear study, we analyzed all experiments using the power spectrum density (PSD), and observed that it was sufficiently good to describe the development of the network or its disintegration. PSD also allowed us to distinguish between healthy and unhealthy networks, and revealed dynamical heterogeneities across the network. For the nonlinear study, we used techniques in the context of recurrence plots. We first characterized the embedding dimension m and the time delay δ for each experiment, built the respective recurrence plots, and extracted key information of the dynamics of the system through different descriptors. Experimental results were contrasted with numerical simulations. After analyzing about 400 time series, we concluded that the degree of dynamical complexity in neuronal cultures changes both during development and disintegration. We also observed that the healthier the culture, the higher its dynamic complexity. Finally, for the reconstruction study, we first used numerical simulations to determine the best measure of ‘statistical interdependence’ among any two neurons, and took Generalized Transfer Entropy. We then analyzed the experimental data. We concluded that young cultures have a weak connectivity that increases along maturation. Aggregation increases average connectivity, and more interesting, also the assortativity, i.e. the tendency of highly connected nodes to connect with other highly connected node. In turn, this assortativity may delineates important aspects of the dynamics of the network. Overall, the results show that spatial arrangement and neuronal dynamics are able to shape a very rich repertoire of dynamical states of varying complexity.[cat] L’habilitat dels teixits neuronals de processar i transmetre informació de forma eficient depèn de les propietats dinàmiques intrínseques de les neurones i de la connectivitat entre elles. La present tesi proposa explorar diferents tècniques experimentals i de simulació per analitzar la dinàmica i connectivitat de xarxes neuronals corticals de rata embrionària. Experimentalment, la gravació de l’activitat espontània d’una població de neurones en cultiu, mitjançant una càmera ràpida i tècniques de fluorescència, possibilita el seguiment de forma controlada de l’activitat individual de cada neurona, així com la modificació de la seva connectivitat. En conjunt, aquestes eines permeten estudiar el comportament col.lectiu emergent de la població neuronal. Amb l’objectiu de simular els patrons observats en el laboratori, hem implementat un model mètric aleatori de creixement neuronal per simular la xarxa física de connexions entre neurones, i un model quadràtic d’integració i dispar amb adaptació i depressió per modelar l’ampli espectre de dinàmiques neuronals amb un cost computacional reduït. Hem caracteritzat la dinàmica global i individual de les neurones i l’hem correlacionat amb la seva estructura subjacent mitjançant tècniques lineals i no–lineals de series temporals. L’anàlisi espectral ens ha possibilitat la descripció del desenvolupament i els canvis en connectivitat en els cultius, així com la diferenciació entre cultius sans dels patològics. La reconstrucció de la dinàmica subjacent mitjançant mètodes d’incrustació i l’ús de gràfics de recurrència ens ha permès detectar diferents transicions dinàmiques amb el corresponent guany o pèrdua de la complexitat i riquesa dinàmica del cultiu durant els diferents estudis experimentals. Finalment, a fi de reconstruir la connectivitat interna hem testejat, mitjançant simulacions, diferents quantificadors per mesurar la dependència estadística entre neurona i neurona, seleccionant finalment el mètode de transferència d’entropia gereralitzada. Seguidament, hem procedit a caracteritzar les xarxes amb diferents paràmetres. Malgrat presentar certs tres de xarxes tipus ‘petit món’, els nostres cultius mostren una distribució de grau ‘exponencial’ o ‘esbiaixada’ per, respectivament, cultius joves i madurs. Addicionalment, hem observat que les xarxes homogènies presenten la propietat de disassortativitat, mentre que xarxes amb un creixent nivell d’agregació espaial presenten assortativitat. Aquesta propietat impacta fortament en la transmissió, resistència i sincronització de la xarxa

Arquitectura escalable SIMD con conectividad jerárquica y reconfigurable para la emulación de SNN

A biological neural system consists of millions of highly integrated neurons with multiple dynamic functions operating in coordination with each other. Its structural organization is characterized by highly hierarchical assemblies. These assemblies are distinguished by locally dense and globally ispersed connections communicated by spikes traveling through the axon to the target neuron. In the last century, approaching the biological complexity of the cortex by means of hardware architectures has continued to be a challenge still unattainable. This is not only due to the massively parallel processing with support for the communication between neurons in large-scale networks, but also for the need of mechanisms that allow the evolution of the neural network efficiently. In this context, this thesis contributes to the development of an architecture called HEENS (Hardware Emulator of Evolved Neural System), which supports inter-chip connectivity with a ring topology between a Master Chip (MC) controlling one or more Neuromorphic Chips (NCs). The MC is implemented in a PSoC device that integrates a CPU ARM Dual Core together with programmable logic. The ARM is responsible for setting up the communication ring and executing the software application that controls the data configuration transmission from the algorithm and the neural parameters to all NCs in the network. Besides, the MC is in charge of activating the evolution mode of the network, as well as managing the dispatching of reconfiguration data to any of the nodes during the execution. Each NC, in turn, consists of a configurable 2D array of Processing Elements (PEs) with a SIMD-like processing scheme implemented on a Kintex7 FPGA. NCs are SNN multiprocessors that support the execution of any neural algorithm based on spikes. A set of custom instructions was designed specifically for this architecture. The NCs support a hierarchical scheme of local and global spikes to mimic the brain structural configuration. Local spikes establish inter-neuronal connectivity within a single chip and the global ones allow inter-modular communication between different chips. The NCs have fixed hub neurons that process local and global spikes, thus allowing inter-modular and intra-modular connectivity. This definition of local and global spikes allows the development of multi-level hierarchical architectures inspired by the brain topologies, and offers excellent scalability. The spike propagation through the multi-chip network is supported by an Aurora / AER-SRT protocol stack. The Aurora protocol encapsulates and de-capsulates the packets transmitted through a high-speed serial link that communicates the platform, while the Synchronous Address Event representation (AER-SRT) protocol manages the data (address events) and controls packets that allow synchronization of the operation of the neural network. Each event encapsulates the address neuron that fires a spike as result of the neural algorithm execution. The definition of local and global synaptic topology is implemented using on-chip RAM blocks, which reduces the combinational logic requirements and, in addition to allowing the dynamic connectivity configuration, permits the development of evolutionary applications by supporting the on-line reconfiguration of both the neural algorithm or the neural and synaptic parameters. HEENS also supports axon programmable delays, which incorporates dynamic features to the network.Un sistema neuronal biológico consiste de millones de neuronas altamente integradas con múltiples funciones dinámicas operando en coordinación entre sí. Su organización estructural se caracteriza por contener agrupaciones altamente jerárquicas. Dichas agrupaciones se distinguen por conexiones localmente densas y globalmente dispersas comunicadas a través de pulsos transitorios (spikes) que viajan por el axón hasta la neurona destino. En el último siglo, aproximarse a la complejidad biológica del cortex mediante arquitecturas de hardware continúa siendo un desafío todavía inalcanzable. Esto se debe, no sólo al masivo procesamiento paralelo con soporte para la comunicación entre neuronas en redes de gran escala, sinó también a la necesidad de mecanismos que permitan la evolución de la red neuronal de forma eficiente. En este marco, esta tesis contribuye al desarrollo de una arquitectura denominada HEENS (Emulador de Hardware para Sistemas Neuronales Evolutivos, Hardware Emulator of Evolved Neural System) que soporta conectividad inter-chip con una topología de anillo entre un chip que actúa de master (MC) y uno o más Chips Neuromórficos (NCs). El MC está implementado en un dispositivo PSoC que integra un CPU ARM Dual Core junto con lógica programable. El ARM se encarga de configurar el anillo de comunicación y de ejecutar la aplicación de software que controla el envío de información de configuración del algoritmo y los parámetros neuronales a todos los NCs de la red. Además, el MC es el encargado de activar el modo de evolución de la red, así como de gestionar el envío de datos de reconfiguración a cualquiera de los nodos durante la ejecución. Cada NC a su vez, está compuesto por un arreglo 2D parametrizable de Elementos de Procesamiento (Processing Elements, PEs) con un esquema de procesamiento tipo SIMD implementado sobre una FPGA Kintex7. Los NCs son multiprocesadores SNN que soportan la ejecución de cualquier algoritmo neuronal basado en spikes. Se cuenta con un set de instrucciones personalizadas diseñadas específicamente para esta arquitectura. Imitando la configuración estructural del cerebro los NC soportan un esquema jerárquico con spikes locales y globales. Los spikes locales establecen la conectividad inter-neuronal dentro de un mismo chip, y los globales la comunicación inter-modular entre diferentes chips. Los NC cuentan con neuronas fijas tipo hub que procesan spikes locales y globales que permiten la conectividad inter e intra modulos. La definición de spikes locales y globales permite desarrollar arquitecturas jerárquicas multi-nivel que se inspiran en las topologías del cerebro y ofrecen una escalabilidad excelente. La propagación de spikes a través de la red multi-chip es soportada por una pila de protocolos Aurora/AER-SRT. El protocolo Aurora encapsula y desencapsula los paquetes transmitidos a través del enlace serial de alta velocidad que comunica la plataforma. Mientras que el protocolo Síncrono de Representación de Eventos de Dirección (AER-SRT) gestiona los datos (eventos de dirección) y los paquetes de control que permiten sincronizar la operación de la red neuronal. Cada evento encapsula la dirección de la neurona que genera un spike como resultado del procesamiento del algoritmo neuronal. La definición de topología sináptica local y global es implementada usando bloques de memoria RAM on-chip, lo que reduce los requerimientos de lógica combinacional y, además de facilitar la configuración del conexionado sin modificar el hardware, permite el desarrollo de aplicaciones evolutivas al soportar la reconfiguración on-line tanto del algoritmo neuronal como de los parámetros neuronales y sinápticos. HEENS también admite retardos programables de axón, lo cual incorpora características dinámicas a la red

18th IEEE Workshop on Nonlinear Dynamics of Electronic Systems: Proceedings

Proceedings of the 18th IEEE Workshop on Nonlinear Dynamics of Electronic Systems, which took place in Dresden, Germany, 26 – 28 May 2010.:Welcome Address ........................ Page I Table of Contents ........................ Page III Symposium Committees .............. Page IV Special Thanks ............................. Page V Conference program (incl. page numbers of papers) ................... Page VI Conference papers Invited talks ................................ Page 1 Regular Papers ........................... Page 14 Wednesday, May 26th, 2010 ......... Page 15 Thursday, May 27th, 2010 .......... Page 110 Friday, May 28th, 2010 ............... Page 210 Author index ............................... Page XII