27 research outputs found
Pattern representation and recognition with accelerated analog neuromorphic systems
Despite being originally inspired by the central nervous system, artificial
neural networks have diverged from their biological archetypes as they have
been remodeled to fit particular tasks. In this paper, we review several
possibilites to reverse map these architectures to biologically more realistic
spiking networks with the aim of emulating them on fast, low-power neuromorphic
hardware. Since many of these devices employ analog components, which cannot be
perfectly controlled, finding ways to compensate for the resulting effects
represents a key challenge. Here, we discuss three different strategies to
address this problem: the addition of auxiliary network components for
stabilizing activity, the utilization of inherently robust architectures and a
training method for hardware-emulated networks that functions without perfect
knowledge of the system's dynamics and parameters. For all three scenarios, we
corroborate our theoretical considerations with experimental results on
accelerated analog neuromorphic platforms.Comment: accepted at ISCAS 201
Neuromorphic Hardware In The Loop: Training a Deep Spiking Network on the BrainScaleS Wafer-Scale System
Emulating spiking neural networks on analog neuromorphic hardware offers
several advantages over simulating them on conventional computers, particularly
in terms of speed and energy consumption. However, this usually comes at the
cost of reduced control over the dynamics of the emulated networks. In this
paper, we demonstrate how iterative training of a hardware-emulated network can
compensate for anomalies induced by the analog substrate. We first convert a
deep neural network trained in software to a spiking network on the BrainScaleS
wafer-scale neuromorphic system, thereby enabling an acceleration factor of 10
000 compared to the biological time domain. This mapping is followed by the
in-the-loop training, where in each training step, the network activity is
first recorded in hardware and then used to compute the parameter updates in
software via backpropagation. An essential finding is that the parameter
updates do not have to be precise, but only need to approximately follow the
correct gradient, which simplifies the computation of updates. Using this
approach, after only several tens of iterations, the spiking network shows an
accuracy close to the ideal software-emulated prototype. The presented
techniques show that deep spiking networks emulated on analog neuromorphic
devices can attain good computational performance despite the inherent
variations of the analog substrate.Comment: 8 pages, 10 figures, submitted to IJCNN 201
Memory Organization for Energy-Efficient Learning and Inference in Digital Neuromorphic Accelerators
The energy efficiency of neuromorphic hardware is greatly affected by the
energy of storing, accessing, and updating synaptic parameters. Various methods
of memory organisation targeting energy-efficient digital accelerators have
been investigated in the past, however, they do not completely encapsulate the
energy costs at a system level. To address this shortcoming and to account for
various overheads, we synthesize the controller and memory for different
encoding schemes and extract the energy costs from these synthesized blocks.
Additionally, we introduce functional encoding for structured connectivity such
as the connectivity in convolutional layers. Functional encoding offers a 58%
reduction in the energy to implement a backward pass and weight update in such
layers compared to existing index-based solutions. We show that for a 2 layer
spiking neural network trained to retain a spatio-temporal pattern, bitmap
(PB-BMP) based organization can encode the sparser networks more efficiently.
This form of encoding delivers a 1.37x improvement in energy efficiency coming
at the cost of a 4% degradation in network retention accuracy as measured by
the van Rossum distance.Comment: submitted to ISCAS202
Versatile emulation of spiking neural networks on an accelerated neuromorphic substrate
We present first experimental results on the novel BrainScaleS-2 neuromorphic
architecture based on an analog neuro-synaptic core and augmented by embedded
microprocessors for complex plasticity and experiment control. The high
acceleration factor of 1000 compared to biological dynamics enables the
execution of computationally expensive tasks, by allowing the fast emulation of
long-duration experiments or rapid iteration over many consecutive trials. The
flexibility of our architecture is demonstrated in a suite of five distinct
experiments, which emphasize different aspects of the BrainScaleS-2 system
Full Wafer Redistribution and Wafer Embedding as Key Technologies for a Multi-Scale Neuromorphic Hardware Cluster
Together with the Kirchhoff-Institute for Physics(KIP) the Fraunhofer IZM has
developed a full wafer redistribution and embedding technology as base for a
large-scale neuromorphic hardware system. The paper will give an overview of
the neuromorphic computing platform at the KIP and the associated hardware
requirements which drove the described technological developments. In the first
phase of the project standard redistribution technologies from wafer level
packaging were adapted to enable a high density reticle-to-reticle routing on
200mm CMOS wafers. Neighboring reticles were interconnected across the scribe
lines with an 8{\mu}m pitch routing based on semi-additive copper
metallization. Passivation by photo sensitive benzocyclobutene was used to
enable a second intra-reticle routing layer. Final IO pads with flash gold were
generated on top of each reticle. With that concept neuromorphic systems based
on full wafers could be assembled and tested. The fabricated high density
inter-reticle routing revealed a very high yield of larger than 99.9%. In order
to allow an upscaling of the system size to a large number of wafers with
feasible effort a full wafer embedding concept for printed circuit boards was
developed and proven in the second phase of the project. The wafers were
thinned to 250{\mu}m and laminated with additional prepreg layers and copper
foils into a core material. After lamination of the PCB panel the reticle IOs
of the embedded wafer were accessed by micro via drilling, copper
electroplating, lithography and subtractive etching of the PCB wiring
structure. The created wiring with 50um line width enabled an access of the
reticle IOs on the embedded wafer as well as a board level routing. The panels
with the embedded wafers were subsequently stressed with up to 1000 thermal
cycles between 0C and 100C and have shown no severe failure formation over the
cycle time.Comment: Accepted at EPTC 201
Image edge detection with a photonic spiking VCSEL-neuron
We report both experimentally and in theory on the detection of edge features in digital images with an artificial optical spiking neuron based on a vertical-cavity surface-emitting laser (VCSEL). The latter delivers fast (< 100 ps) neuron-like optical spikes in response to optical inputs pre-processed using convolution techniques; hence representing image feature information with a spiking data output directly in the optical domain. The proposed technique is able to detect target edges of different directionalities in digital images by applying individual kernel operators and can achieve complete image edge detection using gradient magnitude. Importantly, the neuromorphic (brain-like) spiking edge detection of this work uses commercially sourced VCSELs exhibiting responses at sub-nanosecond rates (many orders of magnitude faster than biological neurons) and operating at the important telecom wavelength of 1300 nm; hence making our approach compatible with optical communication and data-centre technologies