614 research outputs found
Direct Learning-Based Deep Spiking Neural Networks: A Review
The spiking neural network (SNN), as a promising brain-inspired computational
model with binary spike information transmission mechanism, rich
spatially-temporal dynamics, and event-driven characteristics, has received
extensive attention. However, its intricately discontinuous spike mechanism
brings difficulty to the optimization of the deep SNN. Since the surrogate
gradient method can greatly mitigate the optimization difficulty and shows
great potential in directly training deep SNNs, a variety of direct
learning-based deep SNN works have been proposed and achieved satisfying
progress in recent years. In this paper, we present a comprehensive survey of
these direct learning-based deep SNN works, mainly categorized into accuracy
improvement methods, efficiency improvement methods, and temporal dynamics
utilization methods. In addition, we also divide these categorizations into
finer granularities further to better organize and introduce them. Finally, the
challenges and trends that may be faced in future research are prospected.Comment: Accepted by Frontiers in Neuroscienc
Spiking PointNet: Spiking Neural Networks for Point Clouds
Recently, Spiking Neural Networks (SNNs), enjoying extreme energy efficiency,
have drawn much research attention on 2D visual recognition and shown gradually
increasing application potential. However, it still remains underexplored
whether SNNs can be generalized to 3D recognition. To this end, we present
Spiking PointNet in the paper, the first spiking neural model for efficient
deep learning on point clouds. We discover that the two huge obstacles limiting
the application of SNNs in point clouds are: the intrinsic optimization
obstacle of SNNs that impedes the training of a big spiking model with large
time steps, and the expensive memory and computation cost of PointNet that
makes training a big spiking point model unrealistic. To solve the problems
simultaneously, we present a trained-less but learning-more paradigm for
Spiking PointNet with theoretical justifications and in-depth experimental
analysis. In specific, our Spiking PointNet is trained with only a single time
step but can obtain better performance with multiple time steps inference,
compared to the one trained directly with multiple time steps. We conduct
various experiments on ModelNet10, ModelNet40 to demonstrate the effectiveness
of Spiking PointNet. Notably, our Spiking PointNet even can outperform its ANN
counterpart, which is rare in the SNN field thus providing a potential research
direction for the following work. Moreover, Spiking PointNet shows impressive
speedup and storage saving in the training phase.Comment: Accepted by NeurIP
Ternary Spike: Learning Ternary Spikes for Spiking Neural Networks
The Spiking Neural Network (SNN), as one of the biologically inspired neural
network infrastructures, has drawn increasing attention recently. It adopts
binary spike activations to transmit information, thus the multiplications of
activations and weights can be substituted by additions, which brings high
energy efficiency. However, in the paper, we theoretically and experimentally
prove that the binary spike activation map cannot carry enough information,
thus causing information loss and resulting in accuracy decreasing. To handle
the problem, we propose a ternary spike neuron to transmit information. The
ternary spike neuron can also enjoy the event-driven and multiplication-free
operation advantages of the binary spike neuron but will boost the information
capacity. Furthermore, we also embed a trainable factor in the ternary spike
neuron to learn the suitable spike amplitude, thus our SNN will adopt different
spike amplitudes along layers, which can better suit the phenomenon that the
membrane potential distributions are different along layers. To retain the
efficiency of the vanilla ternary spike, the trainable ternary spike SNN will
be converted to a standard one again via a re-parameterization technique in the
inference. Extensive experiments with several popular network structures over
static and dynamic datasets show that the ternary spike can consistently
outperform state-of-the-art methods. Our code is open-sourced at
https://github.com/yfguo91/Ternary-Spike.Comment: Accepted by AAAI202
Learning Spiking Neural Network from Easy to Hard task
Starting with small and simple concepts, and gradually introducing complex
and difficult concepts is the natural process of human learning. Spiking Neural
Networks (SNNs) aim to mimic the way humans process information, but current
SNNs models treat all samples equally, which does not align with the principles
of human learning and overlooks the biological plausibility of SNNs. To address
this, we propose a CL-SNN model that introduces Curriculum Learning(CL) into
SNNs, making SNNs learn more like humans and providing higher biological
interpretability. CL is a training strategy that advocates presenting easier
data to models before gradually introducing more challenging data, mimicking
the human learning process. We use a confidence-aware loss to measure and
process the samples with different difficulty levels. By learning the
confidence of different samples, the model reduces the contribution of
difficult samples to parameter optimization automatically. We conducted
experiments on static image datasets MNIST, Fashion-MNIST, CIFAR10, and
neuromorphic datasets N-MNIST, CIFAR10-DVS, DVS-Gesture. The results are
promising. To our best knowledge, this is the first proposal to enhance the
biologically plausibility of SNNs by introducing CL
Optimising Event-Driven Spiking Neural Network with Regularisation and Cutoff
Spiking neural networks (SNNs), a variant of artificial neural networks
(ANNs) with the benefit of energy efficiency, have achieved the accuracy close
to its ANN counterparts, on benchmark datasets such as CIFAR10/100 and
ImageNet. However, comparing with frame-based input (e.g., images), event-based
inputs from e.g., Dynamic Vision Sensor (DVS) can make a better use of SNNs
thanks to the SNNs' asynchronous working mechanism. In this paper, we
strengthen the marriage between SNNs and event-based inputs with a proposal to
consider anytime optimal inference SNNs, or AOI-SNNs, which can terminate
anytime during the inference to achieve optimal inference result. Two novel
optimisation techniques are presented to achieve AOI-SNNs: a regularisation and
a cutoff. The regularisation enables the training and construction of SNNs with
optimised performance, and the cutoff technique optimises the inference of SNNs
on event-driven inputs. We conduct an extensive set of experiments on multiple
benchmark event-based datasets, including CIFAR10-DVS, N-Caltech101 and DVS128
Gesture. The experimental results demonstrate that our techniques are superior
to the state-of-the-art with respect to the accuracy and latency
Unleashing the Potential of Spiking Neural Networks for Sequential Modeling with Contextual Embedding
The human brain exhibits remarkable abilities in integrating temporally
distant sensory inputs for decision-making. However, existing brain-inspired
spiking neural networks (SNNs) have struggled to match their biological
counterpart in modeling long-term temporal relationships. To address this
problem, this paper presents a novel Contextual Embedding Leaky
Integrate-and-Fire (CE-LIF) spiking neuron model. Specifically, the CE-LIF
model incorporates a meticulously designed contextual embedding component into
the adaptive neuronal firing threshold, thereby enhancing the memory storage of
spiking neurons and facilitating effective sequential modeling. Additionally,
theoretical analysis is provided to elucidate how the CE-LIF model enables
long-term temporal credit assignment. Remarkably, when compared to
state-of-the-art recurrent SNNs, feedforward SNNs comprising the proposed
CE-LIF neurons demonstrate superior performance across extensive sequential
modeling tasks in terms of classification accuracy, network convergence speed,
and memory capacity
Event-Driven Learning for Spiking Neural Networks
Brain-inspired spiking neural networks (SNNs) have gained prominence in the
field of neuromorphic computing owing to their low energy consumption during
feedforward inference on neuromorphic hardware. However, it remains an open
challenge how to effectively benefit from the sparse event-driven property of
SNNs to minimize backpropagation learning costs. In this paper, we conduct a
comprehensive examination of the existing event-driven learning algorithms,
reveal their limitations, and propose novel solutions to overcome them.
Specifically, we introduce two novel event-driven learning methods: the
spike-timing-dependent event-driven (STD-ED) and membrane-potential-dependent
event-driven (MPD-ED) algorithms. These proposed algorithms leverage precise
neuronal spike timing and membrane potential, respectively, for effective
learning. The two methods are extensively evaluated on static and neuromorphic
datasets to confirm their superior performance. They outperform existing
event-driven counterparts by up to 2.51% for STD-ED and 6.79% for MPD-ED on the
CIFAR-100 dataset. In addition, we theoretically and experimentally validate
the energy efficiency of our methods on neuromorphic hardware. On-chip learning
experiments achieved a remarkable 30-fold reduction in energy consumption over
time-step-based surrogate gradient methods. The demonstrated efficiency and
efficacy of the proposed event-driven learning methods emphasize their
potential to significantly advance the fields of neuromorphic computing,
offering promising avenues for energy-efficiency applications
Sharing Leaky-Integrate-and-Fire Neurons for Memory-Efficient Spiking Neural Networks
Spiking Neural Networks (SNNs) have gained increasing attention as
energy-efficient neural networks owing to their binary and asynchronous
computation. However, their non-linear activation, that is
Leaky-Integrate-and-Fire (LIF) neuron, requires additional memory to store a
membrane voltage to capture the temporal dynamics of spikes. Although the
required memory cost for LIF neurons significantly increases as the input
dimension goes larger, a technique to reduce memory for LIF neurons has not
been explored so far. To address this, we propose a simple and effective
solution, EfficientLIF-Net, which shares the LIF neurons across different
layers and channels. Our EfficientLIF-Net achieves comparable accuracy with the
standard SNNs while bringing up to ~4.3X forward memory efficiency and ~21.9X
backward memory efficiency for LIF neurons. We conduct experiments on various
datasets including CIFAR10, CIFAR100, TinyImageNet, ImageNet-100, and
N-Caltech101. Furthermore, we show that our approach also offers advantages on
Human Activity Recognition (HAR) datasets, which heavily rely on temporal
information
LC-TTFS: Towards Lossless Network Conversion for Spiking Neural Networks with TTFS Coding
The biological neurons use precise spike times, in addition to the spike
firing rate, to communicate with each other. The time-to-first-spike (TTFS)
coding is inspired by such biological observation. However, there is a lack of
effective solutions for training TTFS-based spiking neural network (SNN). In
this paper, we put forward a simple yet effective network conversion algorithm,
which is referred to as LC-TTFS, by addressing two main problems that hinder an
effective conversion from a high-performance artificial neural network (ANN) to
a TTFS-based SNN. We show that our algorithm can achieve a near-perfect mapping
between the activation values of an ANN and the spike times of an SNN on a
number of challenging AI tasks, including image classification, image
reconstruction, and speech enhancement. With TTFS coding, we can achieve up to
orders of magnitude saving in computation over ANN and other rate-based SNNs.
The study, therefore, paves the way for deploying ultra-low-power TTFS-based
SNNs on power-constrained edge computing platforms
- …