Conversion of Artificial Recurrent Neural Networks to Spiking Neural Networks for Low-power Neuromorphic Hardware

In recent years the field of neuromorphic low-power systems that consume orders of magnitude less power gained significant momentum. However, their wider use is still hindered by the lack of algorithms that can harness the strengths of such architectures. While neuromorphic adaptations of representation learning algorithms are now emerging, efficient processing of temporal sequences or variable length-inputs remain difficult. Recurrent neural networks (RNN) are widely used in machine learning to solve a variety of sequence learning tasks. In this work we present a train-and-constrain methodology that enables the mapping of machine learned (Elman) RNNs on a substrate of spiking neurons, while being compatible with the capabilities of current and near-future neuromorphic systems. This "train-and-constrain" method consists of first training RNNs using backpropagation through time, then discretizing the weights and finally converting them to spiking RNNs by matching the responses of artificial neurons with those of the spiking neurons. We demonstrate our approach by mapping a natural language processing task (question classification), where we demonstrate the entire mapping process of the recurrent layer of the network on IBM's Neurosynaptic System "TrueNorth", a spike-based digital neuromorphic hardware architecture. TrueNorth imposes specific constraints on connectivity, neural and synaptic parameters. To satisfy these constraints, it was necessary to discretize the synaptic weights and neural activities to 16 levels, and to limit fan-in to 64 inputs. We find that short synaptic delays are sufficient to implement the dynamical (temporal) aspect of the RNN in the question classification task. The hardware-constrained model achieved 74% accuracy in question classification while using less than 0.025% of the cores on one TrueNorth chip, resulting in an estimated power consumption of ~17 uW

Pionic Fusion of ⁴He + ¹²C

Pionic fusion is the process by which two nuclei collide, undergo complete fusion, and then de-excite solely by the emission of a pion. Previously measured pionic fusion cross sections are inconsistent with known mechanisms for pion production and suggest unknown collective processes might dominate production at low energies in heavy ion collisions. In this work, an experiment was developed to make the first coincident measurement of pionic fusion for a charged pion channel of a reaction for which there are no previous measurements. The pionic fusion reaction ⁴He (55 MeV/u) + ¹²C → ¹⁶N + π⁺ was studied at the Texas A&M University Cyclotron Institute. The ¹⁶N fusion residues were detected using a dE-E silicon telescope at the focal plane of the MARS spectrometer and the newly designed ParTI phoswich detector array was used to detect the charged pions. Fast-sampling digitizers recorded the waveform responses of the phoswiches which were used to identify the pions through fast vs. slow (dEE) pulse shape discrimination and through the characteristic decay of the muon daughters of the implanted pions. An energy calibration method for light charged particles including charged pions is developed for the ParTI phoswich detectors and the geometrical and particle identification efficiencies of the array are explored. The "muon decay trigger" which was implemented in the firmware of the onboard FPGA in the digitizers is discussed and its ability to increase the pion event selectivity is characterized. A detailed characterization of the transmission efficiency and particle identification capabilities at the focal plane of MARS is also given. Cross sections are reported for all species detected at the focal plane of MARS and upper limits for the cross section of pionic fusion based on the measurement of ¹⁶N in MARS and charged pions in the ParTI array are reported

Using Light Charged Particles to Probe the Asymmetry Dependence of the Nuclear Caloric Curve

Recently, we observed a clear dependence of the nuclear caloric curve on neutron-proton asymmetry $\frac{N-Z}{A}$ through examination of fully reconstructed equilibrated quasi-projectile sources produced in heavy ion collisions at E/A = 35 MeV. In the present work, we extend our analysis using multiple light charged particle probes of the temperature. Temperatures are extracted with five distinct probes using a kinetic thermometer approach. Additionally, temperatures are extracted using two probes within a chemical thermometer approach (Albergo method). All seven measurements show a significant linear dependence of the source temperature on the source asymmetry. For the kinetic thermometer, the strength of the asymmetry dependence varies with the probe particle species in a way which is consistent with an average emission-time ordering.Comment: 7 pages, 4 figure

Pionic Fusion of ⁴He + ¹²C

Pionic fusion is the process by which two nuclei collide, undergo complete fusion, and then de-excite solely by the emission of a pion. Previously measured pionic fusion cross sections are inconsistent with known mechanisms for pion production and suggest unknown collective processes might dominate production at low energies in heavy ion collisions. In this work, an experiment was developed to make the first coincident measurement of pionic fusion for a charged pion channel of a reaction for which there are no previous measurements. The pionic fusion reaction ⁴He (55 MeV/u) + ¹²C → ¹⁶N + π⁺ was studied at the Texas A&M University Cyclotron Institute. The ¹⁶N fusion residues were detected using a dE-E silicon telescope at the focal plane of the MARS spectrometer and the newly designed ParTI phoswich detector array was used to detect the charged pions. Fast-sampling digitizers recorded the waveform responses of the phoswiches which were used to identify the pions through fast vs. slow (dEE) pulse shape discrimination and through the characteristic decay of the muon daughters of the implanted pions. An energy calibration method for light charged particles including charged pions is developed for the ParTI phoswich detectors and the geometrical and particle identification efficiencies of the array are explored. The "muon decay trigger" which was implemented in the firmware of the onboard FPGA in the digitizers is discussed and its ability to increase the pion event selectivity is characterized. A detailed characterization of the transmission efficiency and particle identification capabilities at the focal plane of MARS is also given. Cross sections are reported for all species detected at the focal plane of MARS and upper limits for the cross section of pionic fusion based on the measurement of ¹⁶N in MARS and charged pions in the ParTI array are reported

Conversion of Artificial Recurrent Neural Networks to Spiking Neural Networks for Low-power Neuromorphic Hardware

In recent years the field of neuromorphic low-power systems gained significant momentum, spurring brain-inspired hardware systems which operate on principles that are fundamentally different from standard digital computers and thereby consume orders of magnitude less power. However, their wider use is still hindered by the lack of algorithms that can harness the strengths of such architectures. While neuromorphic adaptations of representation learning algorithms are now emerging, the efficient processing of temporal sequences or variable length-inputs remains difficult, partly due to challenges in representing and configuring the dynamics of spiking neural networks. Recurrent neural networks (RNN) are widely used in machine learning to solve a variety of sequence learning tasks. In this work we present a train-and-constrain methodology that enables the mapping of machine learned (Elman) RNNs on a substrate of spiking neurons, while being compatible with the capabilities of current and near-future neuromorphic systems. This "train-and-constrain" method consists of first training RNNs using backpropagation through time, then discretizing the weights and finally converting them to spiking RNNs by matching the responses of artificial neurons with those of the spiking neurons. We demonstrate our approach by mapping a natural language processing task (question classification), where we demonstrate the entire mapping process of the recurrent layer of the network on IBM's Neurosynaptic System TrueNorth, a spike-based digital neuromorphic hardware architecture. TrueNorth imposes specific constraints on connectivity, neural and synaptic parameters. To satisfy these constraints, it was necessary to discretize the synaptic weights to 16 levels, discretize the neural activities to 16 levels, and to limit fan-in to 64 inputs. Surprisingly, we find that short synaptic delays are sufficient to implement the dynamic (temporal) aspect of the RNN in the question classification task. Furthermore we observed that the discretization of the neural activities is beneficial to our train-and-constrain approach. The hardware-constrained model achieved 74% accuracy in question classification while using less than 0.025% of the cores on one TrueNorth chip, resulting in an estimated power consumption of approx. 17uW

Conversion of Artificial Recurrent Neural Networks to Spiking Neural Networks for Low-power Neuromorphic Hardware

In recent years the field of neuromorphic low-power systems that consume orders of magnitude less power gained significant momentum. However, their wider use is still hindered by the lack of algorithms that can harness the strengths of such architectures. While neuromorphic adaptations of representation learning algorithms are now emerging, efficient processing of temporal sequences or variable length-inputs remain difficult. Recurrent neural networks (RNN) are widely used in machine learning to solve a variety of sequence learning tasks. In this work we present a train-and-constrain methodology that enables the mapping of machine learned (Elman) RNNs on a substrate of spiking neurons, while being compatible with the capabilities of current and near-future neuromorphic systems. This "train-and-constrain" method consists of first training RNNs using backpropagation through time, then discretizing the weights and finally converting them to spiking RNNs by matching the responses of artificial neurons with those of the spiking neurons. We demonstrate our approach by mapping a natural language processing task (question classification), where we demonstrate the entire mapping process of the recurrent layer of the network on IBM's Neurosynaptic System "TrueNorth", a spike-based digital neuromorphic hardware architecture. TrueNorth imposes specific constraints on connectivity, neural and synaptic parameters. To satisfy these constraints, it was necessary to discretize the synaptic weights and neural activities to 16 levels, and to limit fan-in to 64 inputs. We find that short synaptic delays are sufficient to implement the dynamical (temporal) aspect of the RNN in the question classification task. The hardware-constrained model achieved 74% accuracy in question classification while using less than 0.025% of the cores on one TrueNorth chip, resulting in an estimated power consumption of ~17 uW

TrueHappiness: Neuromorphic Emotion Recognition on TrueNorth

We present an approach to constructing a neuromorphic device that responds to language input by producing neuron spikes in proportion to the strength of the appropriate positive or negative emotional response. Specifically, we perform a fine-grained sentiment analysis task with implementations on two different systems: one using conventional spiking neural network (SNN) simulators and the other one using IBM's Neurosynaptic System TrueNorth. Input words are projected into a high-dimensional semantic space and processed through a fully-connected neural network (FCNN) containing rectified linear units (ReLU) trained via backpropagation. After training, this FCNN is converted to a SNN by substituting the ReLUs with integrate-and-fire neurons. We show that there is practically no performance loss due to conversion to a spiking network on a sentiment analysis test set, i.e. correlations with human annotations differ by less than 0.02 between the original DNN and its spiking equivalent. Additionally, we show that the SNN generated with this technique can be mapped to existing neuromorphic hardware - in our case, the TrueNorth chip. Mapping to the chip involves 4-bit synaptic weight discretization and adjustment of the neuron thresholds. The resulting end-to-end system can take a user input, i.e. a word in a vocabulary of over 300,000 words, and estimate its sentiment on TrueNorth with a power consumption of approximately 50 μW