Abstract. We present our view of the main physical ingredients determining the evolution of neutron star magnetic fields. This includes the basic properties of neutron star matter, possible scenarios for the origin of the magnetic field, constraints and mechanisms for its evolution, and a discussion of our recent work on the Hall drift

Araya, P.

Benguria, R.

Lai, D.

Prieto, J.

Reisenegger, A.

BMC Neuroscience

arXiv

James P Turner

Thomas Nowotny

Springer - Publisher Connector

Estimating numerical error in neural network simulations on Graphics Processing Units

POSTER PRESENTATION Open Access
Estimating numerical error in neural network
simulations on Graphics Processing Units
James P Turner*, Thomas Nowotny
From 24th Annual Computational Neuroscience Meeting: CNS*2015
Prague, Czech Republic. 18-23 July 2015
Modern graphics processing units (GPUs) are becoming
a popular hardware substrate for spiking neural network
simulations [1-3], due to their massive parallelism and
impressive cost-to-speed ratio. However, verifying and
interpreting the results of a GPU simulation can be dif-
ficult because the results are never exactly reproducible,
unlike those from an equivalent serial simulation on a
CPU; not only is the simulation subject to the usual
rounding errors of floating-point arithmetic, but there
are also elements of stochasticity due to the non-deter-
minism of the thread scheduling mechanism on the hard-
ware, alongside the non-associativity of floating-point
addition and multiplication. Consider, for example, a
typical postsynaptic integration step for the summation
of incoming synapse currents, executed on a GPU. If
there are multiple threads, which are each simulating an
incoming synapse, there is no guarantee for the order in
which each synapse thread’s current will be accumulated
into the total current. Therefore, rounding errors will
be different depending on this order and the result could
be different every time the simulation runs. Such
effects would initially be small but can be amplified in
unstable or chaotic systems to a degree that the final
results appear completely random across different runs
(see Figure 1).
When comparing runs between GPU and CPU imple-
mentations there are additional sources of divergence.
There are subtle differences in the way each architecture
* Correspondence: J.P.Turner@sussex.ac.uk
Centre for Computational Neuroscience and Robotics, University of Sussex,
Brighton, UK
Figure 1 In repeated runs, results of numerical simulations on GPUs can vary. Mean, standard deviation and range of observed membrane
potential of a neuron in a network of 10,000 Izhikevich neurons, 8,000 excitatory and 2,000 inhibitory, with 1,000 random connections each; after
only 190 ms simulation the results start to diverge visibly, and after only 210 ms, they differ largely.
Turner and Nowotny BMC Neuroscience 2015, 16(Suppl 1):P182
http://www.biomedcentral.com/1471-2202/16/S1/P182
© 2015 Turner and Nowotny This is an Open Access article distributed under the terms of the Creative Commons Attribution License
(http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided
the original work is properly cited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/
publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
implements floating-point arithmetic. For instance, the
NVIDIA C2070 GPU tested in this study implements
the fused multiply-add (FMA) operation, introduced in
the latest IEEE-754-2008 floating-point standard,
whereas most Intel CPUs perform the multiplication
and addition operations separately, with lower accuracy.
Only Intel’s most recent Haswell CPU architecture
implements the more accurate FMA operation, whilst
many current lab workstations contain chips that do not.
The aim of the work presented here is to analytically
determine the theoretical worst-case and average-case
numerical absolute error incurred when simulating
neural network models on an NVIDIA CUDA GPU.
These error measurements are also compared with the
absolute error resulting from the equivalent serial algo-
rithm running on a single CPU core, using standard
float32 (float) and float64 (double) precision floating-
point arithmetic, to determine a reasonable error margin
for verifying the results of parallel GPU simulations
against those of equivalent serial CPU simulations.
Furthermore, both CPU and GPU implementations are
compared against an equivalent simulation using an
accurate arbitrary-precision floating-point arithmetic
library, to determine how far the CPU and GPU simula-
tion trajectories deviate from the analytically ‘correct’
trajectory. For illustration, the divergence of a single
neuron in a 10,000 neuron Izhikevich network is plotted
in the figure. Finally, we also analyse the role of errors
originating from approximate integration methods and
compare them to the underlying numerical errors dis-
cussed thus far.
Published: 18 December 2015
References
1. GeNN. [https://github.com/genn-team/genn], accessed 20-02-2015.
2. NeMo. [http://nemosim.sourceforge.net/], accessed 20-02-2015.
3. CARLsim. [http://www.socsci.uci.edu/~jkrichma/CARLsim/], accessed 20-02-
2015.
doi:10.1186/1471-2202-16-S1-P182
Cite this article as: Turner and Nowotny: Estimating numerical error in
neural network simulations on Graphics Processing Units. BMC
Neuroscience 2015 16(Suppl 1):P182.
Submit your next manuscript to BioMed Central
and take full advantage of: 
• Convenient online submission
• Thorough peer review
• No space constraints or color figure charges
• Immediate publication on acceptance
• Inclusion in PubMed, CAS, Scopus and Google Scholar
• Research which is freely available for redistribution
Submit your manuscript at 
www.biomedcentral.com/submit
Turner and Nowotny BMC Neuroscience 2015, 16(Suppl 1):P182
http://www.biomedcentral.com/1471-2202/16/S1/P182
Page 2 of 2


Abstract: We present our view of the main physical ingredients determining the evolution of neutron star magnetic fields. This includes the basic properties of neutron star matter, possible scenarios for the origin of the magnetic field, constraints and mechanisms for its evolution, and a discussion of our recent work on the Hall drift.

Estimating numerical error in neural network simulations on Graphics Processing Units

Abstract

Similar works

Full text

Available Versions

Springer - Publisher Connector

Springer - Publisher Connector

University of Groningen Digital Archive

Crossref

University of Groningen Research Database