14 research outputs found
Solving Machine Learning Problems with Biological Principles
Spiking neural networks (SNNs) have been proposed both as models of cortical computation and as candidates for solving problems in machine learning. While increasing recent works have improved their performances in benchmark discriminative tasks, most of them learn by surrogates of backpropagation where biological features such as spikes are regarded more as defects than merits. In this thesis, we explore the enerative abilities of SNNs with built-in biological mechanisms. When sampling from high-dimensional multimodal distributions, models based on general Markov chain Monte Carlo methods often have the mixing problem that the sampler is easy to get trapped in local minima. Inspired from traditional annealing or tempering approaches, we demonstrate that increasing the rate of background Poisson noise in an SNN can flatten the energy landscape and facilitate mixing of the system. In addition, we show that with synaptic short-term plasticity (STP) the SNN can achieve more efficient mixing by local modulation of active
attractors and eventually outperforming traditional benchmark models. We reveal diverse sampling statistics of SNNs induced by STP and finally study its implementation
on conventional machine learning methods. Our work thereby highlights important computational consequences of biological features that might otherwise appear as artifacts
of evolution
Spiking neurons with short-term synaptic plasticity form superior generative networks
Spiking networks that perform probabilistic inference have been proposed both
as models of cortical computation and as candidates for solving problems in
machine learning. However, the evidence for spike-based computation being in
any way superior to non-spiking alternatives remains scarce. We propose that
short-term plasticity can provide spiking networks with distinct computational
advantages compared to their classical counterparts. In this work, we use
networks of leaky integrate-and-fire neurons that are trained to perform both
discriminative and generative tasks in their forward and backward information
processing paths, respectively. During training, the energy landscape
associated with their dynamics becomes highly diverse, with deep attractor
basins separated by high barriers. Classical algorithms solve this problem by
employing various tempering techniques, which are both computationally
demanding and require global state updates. We demonstrate how similar results
can be achieved in spiking networks endowed with local short-term synaptic
plasticity. Additionally, we discuss how these networks can even outperform
tempering-based approaches when the training data is imbalanced. We thereby
show how biologically inspired, local, spike-triggered synaptic dynamics based
simply on a limited pool of synaptic resources can allow spiking networks to
outperform their non-spiking relatives.Comment: corrected typo in abstrac
Efficient Deep Spiking Multi-Layer Perceptrons with Multiplication-Free Inference
Advancements in adapting deep convolution architectures for Spiking Neural
Networks (SNNs) have significantly enhanced image classification performance
and reduced computational burdens. However, the inability of
Multiplication-Free Inference (MFI) to harmonize with attention and transformer
mechanisms, which are critical to superior performance on high-resolution
vision tasks, imposes limitations on these gains. To address this, our research
explores a new pathway, drawing inspiration from the progress made in
Multi-Layer Perceptrons (MLPs). We propose an innovative spiking MLP
architecture that uses batch normalization to retain MFI compatibility and
introduces a spiking patch encoding layer to reinforce local feature extraction
capabilities. As a result, we establish an efficient multi-stage spiking MLP
network that effectively blends global receptive fields with local feature
extraction for comprehensive spike-based computation. Without relying on
pre-training or sophisticated SNN training techniques, our network secures a
top-1 accuracy of 66.39% on the ImageNet-1K dataset, surpassing the directly
trained spiking ResNet-34 by 2.67%. Furthermore, we curtail computational
costs, model capacity, and simulation steps. An expanded version of our network
challenges the performance of the spiking VGG-16 network with a 71.64% top-1
accuracy, all while operating with a model capacity 2.1 times smaller. Our
findings accentuate the potential of our deep SNN architecture in seamlessly
integrating global and local learning abilities. Interestingly, the trained
receptive field in our network mirrors the activity patterns of cortical cells.Comment: 11 pages, 6 figure
Automotive Object Detection via Learning Sparse Events by Temporal Dynamics of Spiking Neurons
Event-based sensors, with their high temporal resolution (1us) and dynamical
range (120dB), have the potential to be deployed in high-speed platforms such
as vehicles and drones. However, the highly sparse and fluctuating nature of
events poses challenges for conventional object detection techniques based on
Artificial Neural Networks (ANNs). In contrast, Spiking Neural Networks (SNNs)
are well-suited for representing event-based data due to their inherent
temporal dynamics. In particular, we demonstrate that the membrane potential
dynamics can modulate network activity upon fluctuating events and strengthen
features of sparse input. In addition, the spike-triggered adaptive threshold
can stabilize training which further improves network performance. Based on
this, we develop an efficient spiking feature pyramid network for event-based
object detection. Our proposed SNN outperforms previous SNNs and sophisticated
ANNs with attention mechanisms, achieving a mean average precision (map50) of
47.7% on the Gen1 benchmark dataset. This result significantly surpasses the
previous best SNN by 9.7% and demonstrates the potential of SNNs for
event-based vision. Our model has a concise architecture while maintaining high
accuracy and much lower computation cost as a result of sparse computation. Our
code will be publicly available
Neuro-Modulated Hebbian Learning for Fully Test-Time Adaptation
Fully test-time adaptation aims to adapt the network model based on
sequential analysis of input samples during the inference stage to address the
cross-domain performance degradation problem of deep neural networks. We take
inspiration from the biological plausibility learning where the neuron
responses are tuned based on a local synapse-change procedure and activated by
competitive lateral inhibition rules. Based on these feed-forward learning
rules, we design a soft Hebbian learning process which provides an unsupervised
and effective mechanism for online adaptation. We observe that the performance
of this feed-forward Hebbian learning for fully test-time adaptation can be
significantly improved by incorporating a feedback neuro-modulation layer. It
is able to fine-tune the neuron responses based on the external feedback
generated by the error back-propagation from the top inference layers. This
leads to our proposed neuro-modulated Hebbian learning (NHL) method for fully
test-time adaptation. With the unsupervised feed-forward soft Hebbian learning
being combined with a learned neuro-modulator to capture feedback from external
responses, the source model can be effectively adapted during the testing
process. Experimental results on benchmark datasets demonstrate that our
proposed method can significantly improve the adaptation performance of network
models and outperforms existing state-of-the-art methods.Comment: CVPR2023 accepte
Weakly-Supervised Action Localization by Hierarchically-structured Latent Attention Modeling
Weakly-supervised action localization aims to recognize and localize action
instancese in untrimmed videos with only video-level labels. Most existing
models rely on multiple instance learning(MIL), where the predictions of
unlabeled instances are supervised by classifying labeled bags. The MIL-based
methods are relatively well studied with cogent performance achieved on
classification but not on localization. Generally, they locate temporal regions
by the video-level classification but overlook the temporal variations of
feature semantics. To address this problem, we propose a novel attention-based
hierarchically-structured latent model to learn the temporal variations of
feature semantics. Specifically, our model entails two components, the first is
an unsupervised change-points detection module that detects change-points by
learning the latent representations of video features in a temporal hierarchy
based on their rates of change, and the second is an attention-based
classification model that selects the change-points of the foreground as the
boundaries. To evaluate the effectiveness of our model, we conduct extensive
experiments on two benchmark datasets, THUMOS-14 and ActivityNet-v1.3. The
experiments show that our method outperforms current state-of-the-art methods,
and even achieves comparable performance with fully-supervised methods.Comment: Accepted to ICCV 2023. arXiv admin note: text overlap with
arXiv:2203.15187, arXiv:2003.12424, arXiv:2104.02967 by other author
Accurate and Efficient Event-based Semantic Segmentation Using Adaptive Spiking Encoder-Decoder Network
Leveraging the low-power, event-driven computation and the inherent temporal
dynamics, spiking neural networks (SNNs) are potentially ideal solutions for
processing dynamic and asynchronous signals from event-based sensors. However,
due to the challenges in training and the restrictions in architectural design,
there are limited examples of competitive SNNs in the realm of event-based
dense prediction when compared to artificial neural networks (ANNs). In this
paper, we present an efficient spiking encoder-decoder network designed for
large-scale event-based semantic segmentation tasks. This is achieved by
optimizing the encoder using a hierarchical search method. To enhance learning
from dynamic event streams, we harness the inherent adaptive threshold of
spiking neurons to modulate network activation. Moreover, we introduce a
dual-path Spiking Spatially-Adaptive Modulation (SSAM) block, specifically
designed to enhance the representation of sparse events, thereby considerably
improving network performance. Our proposed network achieves a 72.57% mean
intersection over union (MIoU) on the DDD17 dataset and a 57.22% MIoU on the
recently introduced, larger DSEC-Semantic dataset. This performance surpasses
the current state-of-the-art ANNs by 4%, whilst consuming significantly less
computational resources. To the best of our knowledge, this is the first study
demonstrating SNNs outperforming ANNs in demanding event-based semantic
segmentation tasks, thereby establishing the vast potential of SNNs in the
field of event-based vision. Our source code will be made publicly accessible
Cortical oscillations implement a backbone for sampling-based computation in spiking neural networks
Brains need to deal with an uncertain world. Often, this requires visiting
multiple interpretations of the available information or multiple solutions to
an encountered problem. This gives rise to the so-called mixing problem: since
all of these "valid" states represent powerful attractors, but between
themselves can be very dissimilar, switching between such states can be
difficult. We propose that cortical oscillations can be effectively used to
overcome this challenge. By acting as an effective temperature, background
spiking activity modulates exploration. Rhythmic changes induced by cortical
oscillations can then be interpreted as a form of simulated tempering. We
provide a rigorous mathematical discussion of this link and study some of its
phenomenological implications in computer simulations. This identifies a new
computational role of cortical oscillations and connects them to various
phenomena in the brain, such as sampling-based probabilistic inference, memory
replay, multisensory cue combination and place cell flickering.Comment: 30 pages, 11 figure
Accelerated physical emulation of Bayesian inference in spiking neural networks
The massively parallel nature of biological information processing plays an
important role for its superiority to human-engineered computing devices. In
particular, it may hold the key to overcoming the von Neumann bottleneck that
limits contemporary computer architectures. Physical-model neuromorphic devices
seek to replicate not only this inherent parallelism, but also aspects of its
microscopic dynamics in analog circuits emulating neurons and synapses.
However, these machines require network models that are not only adept at
solving particular tasks, but that can also cope with the inherent
imperfections of analog substrates. We present a spiking network model that
performs Bayesian inference through sampling on the BrainScaleS neuromorphic
platform, where we use it for generative and discriminative computations on
visual data. By illustrating its functionality on this platform, we implicitly
demonstrate its robustness to various substrate-specific distortive effects, as
well as its accelerated capability for computation. These results showcase the
advantages of brain-inspired physical computation and provide important
building blocks for large-scale neuromorphic applications.Comment: This preprint has been published 2019 November 14. Please cite as:
Kungl A. F. et al. (2019) Accelerated Physical Emulation of Bayesian
Inference in Spiking Neural Networks. Front. Neurosci. 13:1201. doi:
10.3389/fnins.2019.0120
25th annual computational neuroscience meeting: CNS-2016
The same neuron may play different functional roles in the neural circuits to which it belongs. For example, neurons in the Tritonia pedal ganglia may participate in variable phases of the swim motor rhythms [1]. While such neuronal functional variability is likely to play a major role the delivery of the functionality of neural systems, it is difficult to study it in most nervous systems. We work on the pyloric rhythm network of the crustacean stomatogastric ganglion (STG) [2]. Typically network models of the STG treat neurons of the same functional type as a single model neuron (e.g. PD neurons), assuming the same conductance parameters for these neurons and implying their synchronous firing [3, 4]. However, simultaneous recording of PD neurons shows differences between the timings of spikes of these neurons. This may indicate functional variability of these neurons. Here we modelled separately the two PD neurons of the STG in a multi-neuron model of the pyloric network. Our neuron models comply with known correlations between conductance parameters of ionic currents. Our results reproduce the experimental finding of increasing spike time distance between spikes originating from the two model PD neurons during their synchronised burst phase. The PD neuron with the larger calcium conductance generates its spikes before the other PD neuron. Larger potassium conductance values in the follower neuron imply longer delays between spikes, see Fig. 17.Neuromodulators change the conductance parameters of neurons and maintain the ratios of these parameters [5]. Our results show that such changes may shift the individual contribution of two PD neurons to the PD-phase of the pyloric rhythm altering their functionality within this rhythm. Our work paves the way towards an accessible experimental and computational framework for the analysis of the mechanisms and impact of functional variability of neurons within the neural circuits to which they belong