Error-backpropagation in temporally encoded networks of spiking neurons

For a network of spiking neurons that encodes information in the timing of individual spike-times, we derive a supervised learning rule, emph{SpikeProp, akin to traditional error-backpropagation and show how to overcome the discontinuities introduced by thresholding. With this algorithm, we demonstrate how networks of spiking neurons with biologically reasonable action potentials can perform complex non-linear classification in fast temporal coding just as well as rate-coded networks. We perform experiments for the classical XOR-problem, when posed in a temporal setting, as well as for a number of other benchmark datasets. Comparing the (implicit) number of spiking neurons required for the encoding of the interpolated XOR problem, it is demonstrated that temporal coding requires significantly less neurons than instantaneous rate-coding

Brain-inspired artificial intelligence: spiking neural networks as a biologically plausible learning model

openLe reti neurali artificiali (ANNs) nascono come modello di machine learning con lo scopo di simulare il modo in cui i neuroni biologici comunicano, per poter riprodurre, seppur in maniera semplificata, la capacità di elaborazione di informazioni del cervello umano. Negli ultimi decenni, e soprattutto negli ultimi anni grazie all’avvento del deep learning, le reti neurali hanno trovato sempre più applicazione in vari settori, tra cui riconoscimento di immagini e riconoscimento vocale, diagnosi mediche, previsioni sull’andamento del mercato, supporto a processi decisionali e analisi del rischio. Tuttavia, il modello presenta problemi legato al consumo di energia, richiesta di quantità di dati, e costi computazionali alti. Queste problematiche oggi sono attenuate da un particolare tipo di reti neurali: le spiking neural networks (SNNs). Le SNNs rappresentano la terza generazione delle reti neurali artificiali, e sono ispirate, in maniera più forte rispetto alle deep neural networks, al meccanismo di funzionamento dei neuroni biologici. La comunicazione tra i nodi, i neuroni, avviene quindi tramite la trasmissione di segnali discreti, e permette di elaborare anche la componente temporale degli stimoli. L’obiettivo di questo elaborato è presentare le caratteristiche principali delle SNNs confrontandole con le reti biologiche, e fornire una panoramica sui settori di applicazione e e i recenti sviluppi di tale modello

Temporal Spike Sequence Learning via Backpropagation for Deep Spiking Neural Networks

Spiking neural networks (SNNs) are well suited for spatio-temporal learning and implementations on energy-efficient event-driven neuromorphic processors. However, existing SNN error backpropagation (BP) methods lack proper handling of spiking discontinuities and suffer from low performance compared with the BP methods for traditional artificial neural networks. In addition, a large number of time steps are typically required to achieve decent performance, leading to high latency and rendering spike-based computation unscalable to deep architectures. We present a novel Temporal Spike Sequence Learning Backpropagation (TSSL-BP) method for training deep SNNs, which breaks down error backpropagation across two types of inter-neuron and intra-neuron dependencies and leads to improved temporal learning precision. It captures inter-neuron dependencies through presynaptic firing times by considering the all-or-none characteristics of firing activities and captures intra-neuron dependencies by handling the internal evolution of each neuronal state in time. TSSL-BP efficiently trains deep SNNs within a much shortened temporal window of a few steps while improving the accuracy for various image classification datasets including CIFAR10.Comment: Accepted for spotlight presentation of NeurIPS (Neural Information Processing System) 2020: https://proceedings.neurips.cc/paper/2020/hash/8bdb5058376143fa358981954e7626b8-Abstract.htm