5 research outputs found
Updates of Equilibrium Prop Match Gradients of Backprop Through Time in an RNN with Static Input
Equilibrium Propagation (EP) is a biologically inspired learning algorithm
for convergent recurrent neural networks, i.e. RNNs that are fed by a static
input x and settle to a steady state. Training convergent RNNs consists in
adjusting the weights until the steady state of output neurons coincides with a
target y. Convergent RNNs can also be trained with the more conventional
Backpropagation Through Time (BPTT) algorithm. In its original formulation EP
was described in the case of real-time neuronal dynamics, which is
computationally costly. In this work, we introduce a discrete-time version of
EP with simplified equations and with reduced simulation time, bringing EP
closer to practical machine learning tasks. We first prove theoretically, as
well as numerically that the neural and weight updates of EP, computed by
forward-time dynamics, are step-by-step equal to the ones obtained by BPTT,
with gradients computed backward in time. The equality is strict when the
transition function of the dynamics derives from a primitive function and the
steady state is maintained long enough. We then show for more standard
discrete-time neural network dynamics that the same property is approximately
respected and we subsequently demonstrate training with EP with equivalent
performance to BPTT. In particular, we define the first convolutional
architecture trained with EP achieving ~ 1% test error on MNIST, which is the
lowest error reported with EP. These results can guide the development of deep
neural networks trained with EP
Contrastive Similarity Matching for Supervised Learning
We propose a novel biologically-plausible solution to the credit assignment
problem motivated by observations in the ventral visual pathway and trained
deep neural networks. In both, representations of objects in the same category
become progressively more similar, while objects belonging to different
categories become less similar. We use this observation to motivate a
layer-specific learning goal in a deep network: each layer aims to learn a
representational similarity matrix that interpolates between previous and later
layers. We formulate this idea using a contrastive similarity matching
objective function and derive from it deep neural networks with feedforward,
lateral, and feedback connections, and neurons that exhibit
biologically-plausible Hebbian and anti-Hebbian plasticity. Contrastive
similarity matching can be interpreted as an energy-based learning algorithm,
but with significant differences from others in how a contrastive function is
constructed
Scaling Equilibrium Propagation to Deep ConvNets by Drastically Reducing its Gradient Estimator Bias
Equilibrium Propagation (EP) is a biologically-inspired algorithm for
convergent RNNs with a local learning rule that comes with strong theoretical
guarantees. The parameter updates of the neural network during the credit
assignment phase have been shown mathematically to approach the gradients
provided by Backpropagation Through Time (BPTT) when the network is
infinitesimally nudged toward its target. In practice, however, training a
network with the gradient estimates provided by EP does not scale to visual
tasks harder than MNIST. In this work, we show that a bias in the gradient
estimate of EP, inherent in the use of finite nudging, is responsible for this
phenomenon and that cancelling it allows training deep ConvNets by EP. We show
that this bias can be greatly reduced by using symmetric nudging (a positive
nudging and a negative one). We also generalize previous EP equations to the
case of cross-entropy loss (by opposition to squared error). As a result of
these advances, we are able to achieve a test error of 11.7% on CIFAR-10 by EP,
which approaches the one achieved by BPTT and provides a major improvement with
respect to the standard EP approach with same-sign nudging that gives 86% test
error. We also apply these techniques to train an architecture with asymmetric
forward and backward connections, yielding a 13.2% test error. These results
highlight EP as a compelling biologically-plausible approach to compute error
gradients in deep neural networks
Equilibrium Propagation with Continual Weight Updates
Equilibrium Propagation (EP) is a learning algorithm that bridges Machine
Learning and Neuroscience, by computing gradients closely matching those of
Backpropagation Through Time (BPTT), but with a learning rule local in space.
Given an input and associated target , EP proceeds in two phases: in the
first phase neurons evolve freely towards a first steady state; in the second
phase output neurons are nudged towards until they reach a second steady
state. However, in existing implementations of EP, the learning rule is not
local in time: the weight update is performed after the dynamics of the second
phase have converged and requires information of the first phase that is no
longer available physically. In this work, we propose a version of EP named
Continual Equilibrium Propagation (C-EP) where neuron and synapse dynamics
occur simultaneously throughout the second phase, so that the weight update
becomes local in time. Such a learning rule local both in space and time opens
the possibility of an extremely energy efficient hardware implementation of EP.
We prove theoretically that, provided the learning rates are sufficiently
small, at each time step of the second phase the dynamics of neurons and
synapses follow the gradients of the loss given by BPTT (Theorem 1). We
demonstrate training with C-EP on MNIST and generalize C-EP to neural networks
where neurons are connected by asymmetric connections. We show through
experiments that the more the network updates follows the gradients of BPTT,
the best it performs in terms of training. These results bring EP a step closer
to biology by better complying with hardware constraints while maintaining its
intimate link with backpropagation
Training End-to-End Analog Neural Networks with Equilibrium Propagation
We introduce a principled method to train end-to-end analog neural networks
by stochastic gradient descent. In these analog neural networks, the weights to
be adjusted are implemented by the conductances of programmable resistive
devices such as memristors [Chua, 1971], and the nonlinear transfer functions
(or `activation functions') are implemented by nonlinear components such as
diodes. We show mathematically that a class of analog neural networks (called
nonlinear resistive networks) are energy-based models: they possess an energy
function as a consequence of Kirchhoff's laws governing electrical circuits.
This property enables us to train them using the Equilibrium Propagation
framework [Scellier and Bengio, 2017]. Our update rule for each conductance,
which is local and relies solely on the voltage drop across the corresponding
resistor, is shown to compute the gradient of the loss function. Our numerical
simulations, which use the SPICE-based Spectre simulation framework to simulate
the dynamics of electrical circuits, demonstrate training on the MNIST
classification task, performing comparably or better than equivalent-size
software-based neural networks. Our work can guide the development of a new
generation of ultra-fast, compact and low-power neural networks supporting
on-chip learning