Pattern Recognition Using Spiking Neural Networks

Deep learning believed to be a promising approach for solving specific problems in the field of artificial intelligence whenever a large amount of data and computation is available. However, tasks that require immediate yet robust decisions in the presence of small data are not suited for such an approach. The superior performance of the human brain in specific tasks like pattern recognition in comparison to traditional neural networks convinced neuroscientists to introduce a biologically plausible model of the neuron, which is known as spiking neurons. In opposition to conventional neuron, spiking neurons use a short electrical pulse known as a spike to transfer the information. The complexity and dynamic of these neurons allow them to perform complex computational tasks. However, training a spiking neural network does not follow the rule of conventional ANN, and we need to devise new methods of training that are compatible with the unsupervised nature of these networks. This thesis aims to investigate the unsupervised approaches of training spiking networks using spike time-dependent plasticity (STDP) and assess their performance on real-world machine learning applications like handwritten digit recognition

Improving the bees algorithm for complex optimisation problems

An improved swarm-based optimisation algorithm from the Bees Algorithm family for solving complex optimisation problems is proposed. Like other Bees Algorithms, the algorithm performs a form of exploitative local search combined with random exploratory global search. This thesis details the development and optimisation of this algorithm and demonstrates its robustness. The development includes a new method of tuning the Bees Algorithm called Meta Bees Algorithm and the functionality of the proposed method is compared to the standard Bees Algorithm and to a range of state-of-the-art optimisation algorithms. A new fitness evaluation method has been developed to enable the Bees Algorithm to solve a stochastic optimisation problem. The new modified Bees Algorithm was tested on the optimisation of parameter values for the Ant Colony Optimisation algorithm when solving Travelling Salesman Problems. Finally, the Bees Algorithm has been adapted and employed to solve complex combinatorial problems. The algorithm has been combined with two neighbourhood operators to solve such problems. The performance of the proposed Bees Algorithm has been tested on a number of travelling salesman problems, including two problems on printed circuit board assembly machine sequencing

Arquitectura escalable SIMD con conectividad jerárquica y reconfigurable para la emulación de SNN

A biological neural system consists of millions of highly integrated neurons with multiple dynamic functions operating in coordination with each other. Its structural organization is characterized by highly hierarchical assemblies. These assemblies are distinguished by locally dense and globally ispersed connections communicated by spikes traveling through the axon to the target neuron. In the last century, approaching the biological complexity of the cortex by means of hardware architectures has continued to be a challenge still unattainable. This is not only due to the massively parallel processing with support for the communication between neurons in large-scale networks, but also for the need of mechanisms that allow the evolution of the neural network efficiently. In this context, this thesis contributes to the development of an architecture called HEENS (Hardware Emulator of Evolved Neural System), which supports inter-chip connectivity with a ring topology between a Master Chip (MC) controlling one or more Neuromorphic Chips (NCs). The MC is implemented in a PSoC device that integrates a CPU ARM Dual Core together with programmable logic. The ARM is responsible for setting up the communication ring and executing the software application that controls the data configuration transmission from the algorithm and the neural parameters to all NCs in the network. Besides, the MC is in charge of activating the evolution mode of the network, as well as managing the dispatching of reconfiguration data to any of the nodes during the execution. Each NC, in turn, consists of a configurable 2D array of Processing Elements (PEs) with a SIMD-like processing scheme implemented on a Kintex7 FPGA. NCs are SNN multiprocessors that support the execution of any neural algorithm based on spikes. A set of custom instructions was designed specifically for this architecture. The NCs support a hierarchical scheme of local and global spikes to mimic the brain structural configuration. Local spikes establish inter-neuronal connectivity within a single chip and the global ones allow inter-modular communication between different chips. The NCs have fixed hub neurons that process local and global spikes, thus allowing inter-modular and intra-modular connectivity. This definition of local and global spikes allows the development of multi-level hierarchical architectures inspired by the brain topologies, and offers excellent scalability. The spike propagation through the multi-chip network is supported by an Aurora / AER-SRT protocol stack. The Aurora protocol encapsulates and de-capsulates the packets transmitted through a high-speed serial link that communicates the platform, while the Synchronous Address Event representation (AER-SRT) protocol manages the data (address events) and controls packets that allow synchronization of the operation of the neural network. Each event encapsulates the address neuron that fires a spike as result of the neural algorithm execution. The definition of local and global synaptic topology is implemented using on-chip RAM blocks, which reduces the combinational logic requirements and, in addition to allowing the dynamic connectivity configuration, permits the development of evolutionary applications by supporting the on-line reconfiguration of both the neural algorithm or the neural and synaptic parameters. HEENS also supports axon programmable delays, which incorporates dynamic features to the network.Un sistema neuronal biológico consiste de millones de neuronas altamente integradas con múltiples funciones dinámicas operando en coordinación entre sí. Su organización estructural se caracteriza por contener agrupaciones altamente jerárquicas. Dichas agrupaciones se distinguen por conexiones localmente densas y globalmente dispersas comunicadas a través de pulsos transitorios (spikes) que viajan por el axón hasta la neurona destino. En el último siglo, aproximarse a la complejidad biológica del cortex mediante arquitecturas de hardware continúa siendo un desafío todavía inalcanzable. Esto se debe, no sólo al masivo procesamiento paralelo con soporte para la comunicación entre neuronas en redes de gran escala, sinó también a la necesidad de mecanismos que permitan la evolución de la red neuronal de forma eficiente. En este marco, esta tesis contribuye al desarrollo de una arquitectura denominada HEENS (Emulador de Hardware para Sistemas Neuronales Evolutivos, Hardware Emulator of Evolved Neural System) que soporta conectividad inter-chip con una topología de anillo entre un chip que actúa de master (MC) y uno o más Chips Neuromórficos (NCs). El MC está implementado en un dispositivo PSoC que integra un CPU ARM Dual Core junto con lógica programable. El ARM se encarga de configurar el anillo de comunicación y de ejecutar la aplicación de software que controla el envío de información de configuración del algoritmo y los parámetros neuronales a todos los NCs de la red. Además, el MC es el encargado de activar el modo de evolución de la red, así como de gestionar el envío de datos de reconfiguración a cualquiera de los nodos durante la ejecución. Cada NC a su vez, está compuesto por un arreglo 2D parametrizable de Elementos de Procesamiento (Processing Elements, PEs) con un esquema de procesamiento tipo SIMD implementado sobre una FPGA Kintex7. Los NCs son multiprocesadores SNN que soportan la ejecución de cualquier algoritmo neuronal basado en spikes. Se cuenta con un set de instrucciones personalizadas diseñadas específicamente para esta arquitectura. Imitando la configuración estructural del cerebro los NC soportan un esquema jerárquico con spikes locales y globales. Los spikes locales establecen la conectividad inter-neuronal dentro de un mismo chip, y los globales la comunicación inter-modular entre diferentes chips. Los NC cuentan con neuronas fijas tipo hub que procesan spikes locales y globales que permiten la conectividad inter e intra modulos. La definición de spikes locales y globales permite desarrollar arquitecturas jerárquicas multi-nivel que se inspiran en las topologías del cerebro y ofrecen una escalabilidad excelente. La propagación de spikes a través de la red multi-chip es soportada por una pila de protocolos Aurora/AER-SRT. El protocolo Aurora encapsula y desencapsula los paquetes transmitidos a través del enlace serial de alta velocidad que comunica la plataforma. Mientras que el protocolo Síncrono de Representación de Eventos de Dirección (AER-SRT) gestiona los datos (eventos de dirección) y los paquetes de control que permiten sincronizar la operación de la red neuronal. Cada evento encapsula la dirección de la neurona que genera un spike como resultado del procesamiento del algoritmo neuronal. La definición de topología sináptica local y global es implementada usando bloques de memoria RAM on-chip, lo que reduce los requerimientos de lógica combinacional y, además de facilitar la configuración del conexionado sin modificar el hardware, permite el desarrollo de aplicaciones evolutivas al soportar la reconfiguración on-line tanto del algoritmo neuronal como de los parámetros neuronales y sinápticos. HEENS también admite retardos programables de axón, lo cual incorpora características dinámicas a la red

Evolution of Spiking Neural Networks for Temporal Pattern Recognition and Animat Control

I extended an artificial life platform called GReaNs (the name stands for Gene Regulatory evolving artificial Networks) to explore the evolutionary abilities of biologically inspired Spiking Neural Network (SNN) model. The encoding of SNNs in GReaNs was inspired by the encoding of gene regulatory networks. As proof-of-principle, I used GReaNs to evolve SNNs to obtain a network with an output neuron which generates a predefined spike train in response to a specific input. Temporal pattern recognition was one of the main tasks during my studies. It is widely believed that nervous systems of biological organisms use temporal patterns of inputs to encode information. The learning technique used for temporal pattern recognition is not clear yet. I studied the ability to evolve spiking networks with different numbers of interneurons in the absence and the presence of noise to recognize predefined temporal patterns of inputs. Results showed, that in the presence of noise, it was possible to evolve successful networks. However, the networks with only one interneuron were not robust to noise. The foraging behaviour of many small animals depends mainly on their olfactory system. I explored whether it was possible to evolve SNNs able to control an agent to find food particles on 2-dimensional maps. Using ring rate encoding to encode the sensory information in the olfactory input neurons, I managed to obtain SNNs able to control an agent that could detect the position of the food particles and move toward it. Furthermore, I did unsuccessful attempts to use GReaNs to evolve an SNN able to control an agent able to collect sound sources from one type out of several sound types. Each sound type is represented as a pattern of different frequencies. In order to use the computational power of neuromorphic hardware, I integrated GReaNs with the SpiNNaker hardware system. Only the simulation part was carried out using SpiNNaker, but the rest steps of the genetic algorithm were done with GReaNs

Pattern recognition with spiking neural networks and the ROLLS low-power online learning neuromorphic processor

Online monitoring applications requiring advanced pattern recognition capabilities implemented in resource-constrained wireless sensor systems are challenging to construct using standard digital computers. An interesting alternative solution is to use a low-power neuromorphic processor like the ROLLS, with subthreshold mixed analog/digital circuits and online learning capabilities that approximate the behavior of real neurons and synapses. This requires that the monitoring algorithm is implemented with spiking neural networks, which in principle are efficient computational models for tasks such as pattern recognition. In this work, I investigate how spiking neural networks can be used as a pre-processing and feature learning system in a condition monitoring application where the vibration of a machine with healthy and faulty rolling-element bearings is considered. Pattern recognition with spiking neural networks is investigated using simulations with Brian -- a Python-based open source toolbox -- and an implementation is developed for the ROLLS neuromorphic processor. I analyze the learned feature-response properties of individual neurons. When pre-processing the input signals with a neuromorphic cochlea known as the AER-EAR system, the ROLLS chip learns to classify the resulting spike patterns with a training error of less than 1 %, at a combined power consumption of approximately 30 mW. Thus, the neuromorphic hardware system can potentially be realized in a resource-constrained wireless sensor for online monitoring applications.However, further work is needed for testing and cross validation of the feature learning and pattern recognition networks.