2 research outputs found
Biologically Plausible Sequence Learning with Spiking Neural Networks
Motivated by the celebrated discrete-time model of nervous activity outlined
by McCulloch and Pitts in 1943, we propose a novel continuous-time model, the
McCulloch-Pitts network (MPN), for sequence learning in spiking neural
networks. Our model has a local learning rule, such that the synaptic weight
updates depend only on the information directly accessible by the synapse. By
exploiting asymmetry in the connections between binary neurons, we show that
MPN can be trained to robustly memorize multiple spatiotemporal patterns of
binary vectors, generalizing the ability of the symmetric Hopfield network to
memorize static spatial patterns. In addition, we demonstrate that the model
can efficiently learn sequences of binary pictures as well as generative models
for experimental neural spike-train data. Our learning rule is consistent with
spike-timing-dependent plasticity (STDP), thus providing a theoretical ground
for the systematic design of biologically inspired networks with large and
robust long-range sequence storage capacity.Comment: Accepted for publication in the Proceedings of the 34th AAAI
Conference on Artificial Intelligence (AAAI-20
Variational Probability Flow for Biologically Plausible Training of Deep Neural Networks
The quest for biologically plausible deep learning is driven, not just by the desire to explain experimentally-observed properties of biological neural networks, but also by the hope of discovering more efficient methods for training artificial networks. In this paper, we propose a new algorithm named Variational Probably Flow (VPF), an extension of minimum probability flow for training binary Deep Boltzmann Machines (DBMs). We show that weight updates in VPF are local, depending only on the states and firing rates of the adjacent neurons. Unlike contrastive divergence, there is no need for Gibbs confabulations; and unlike backpropagation, alternating feedforward and feedback phases are not required. Moreover, the learning algorithm is effective for training DBMs with intra-layer connections between the hidden nodes. Experiments with MNIST and Fashion MNIST demonstrate that VPF learns reasonable features quickly, reconstructs corrupted images more accurately, and generates samples with a high estimated log-likelihood. Lastly, we note that, interestingly, if an asymmetric version of VPF exists, the weight updates directly explain experimental results in Spike-Timing-Dependent Plasticity (STDP)