1,180 research outputs found
Technology Aware Training in Memristive Neuromorphic Systems based on non-ideal Synaptic Crossbars
The advances in the field of machine learning using neuromorphic systems have
paved the pathway for extensive research on possibilities of hardware
implementations of neural networks. Various memristive technologies such as
oxide-based devices, spintronics and phase change materials have been explored
to implement the core functional units of neuromorphic systems, namely the
synaptic network, and the neuronal functionality, in a fast and energy
efficient manner. However, various non-idealities in the crossbar
implementations of the synaptic arrays can significantly degrade performance of
neural networks and hence, impose restrictions on feasible crossbar sizes. In
this work, we build mathematical models of various non-idealities that occur in
crossbar implementations such as source resistance, neuron resistance and
chip-to-chip device variations and analyze their impact on the classification
accuracy of a fully connected network (FCN) and convolutional neural network
(CNN) trained with standard training algorithm. We show that a network trained
under ideal conditions can suffer accuracy degradation as large as 59.84% for
FCNs and 62.4% for CNNs when implemented on non-ideal crossbars for relevant
non-ideality ranges. This severely constrains the sizes for crossbars. As a
solution, we propose a technology aware training algorithm which incorporates
the mathematical models of the non-idealities in the standard training
algorithm. We demonstrate that our proposed methodology achieves significant
recovery of testing accuracy within 1.9% of the ideal accuracy for FCNs and
1.5% for CNNs. We further show that our proposed training algorithm can
potentially allow the use of significantly larger crossbar arrays of sizes
784500 for FCNs and 4096512 for CNNs with a minor or no
trade-off in accuracyComment: 10 pages, 7 figures, submitted to IEEE Transactions on Emerging
Topics in Computational Intelligenc
Tree-CNN: A Hierarchical Deep Convolutional Neural Network for Incremental Learning
Over the past decade, Deep Convolutional Neural Networks (DCNNs) have shown
remarkable performance in most computer vision tasks. These tasks traditionally
use a fixed dataset, and the model, once trained, is deployed as is. Adding new
information to such a model presents a challenge due to complex training
issues, such as "catastrophic forgetting", and sensitivity to hyper-parameter
tuning. However, in this modern world, data is constantly evolving, and our
deep learning models are required to adapt to these changes. In this paper, we
propose an adaptive hierarchical network structure composed of DCNNs that can
grow and learn as new data becomes available. The network grows in a tree-like
fashion to accommodate new classes of data, while preserving the ability to
distinguish the previously trained classes. The network organizes the
incrementally available data into feature-driven super-classes and improves
upon existing hierarchical CNN models by adding the capability of self-growth.
The proposed hierarchical model, when compared against fine-tuning a deep
network, achieves significant reduction of training effort, while maintaining
competitive accuracy on CIFAR-10 and CIFAR-100.Comment: 8 pages, 6 figures, 7 tables Accepted in Neural Networks, 201
Exploring Ultra Low-Power on-Chip Clocking Using Functionality Enhanced Spin-Torque Switches
Emerging spin-torque (ST) phenomena may lead to ultra-low-voltage, high-speed
nano-magnetic switches. Such current-based-switches can be attractive for
designing low swing global-interconnects, like, clocking-networks and
databuses. In this work we present the basic idea of using such ST-switches for
low-power on-chip clocking. For clockingnetworks, Spin-Hall-Effect (SHE) can be
used to produce an assist-field for fast ST-switching using global-mesh-clock
with less than 100mV swing. The ST-switch acts as a compact-latch, written by
ultra-low-voltage input-pulses. The data is read using a high-resistance
tunnel-junction. The clock-driven SHE write-assist can be shared among large
number of ST-latches, thereby reducing the load-capacitance for
clock-distribution. The SHE assist can be activated by a low-swing clock
(~150mV) and hence can facilitate ultra-low voltage clock-distribution. Owing
to reduced clock-load and low-voltage operation, the proposed scheme can
achieve 97% low-power for on-chip clocking as compared to the state of the art
CMOS design. Rigorous device-circuit simulations and system-level modelling for
the proposed scheme will be addressed in future
Image Edge Detection based on Swarm Intelligence using Memristive Networks
Recent advancements in the development of memristive devices has opened new
opportunities for hardware implementation of non-Boolean computing. To this
end, the suitability of memristive devices for swarm intelligence algorithms
has enabled researchers to solve a maze in hardware. In this paper, we utilize
swarm intelligence of memristive networks to perform image edge detection.
First, we propose a hardware-friendly algorithm for image edge detection based
on ant colony. Second, we implement the image edge detection algorithm using
memristive networks. Furthermore, we explain the impact of various parameters
of the memristors on the efficacy of the implementation. Our results show 28%
improvement in the energy compared to a low power CMOS hardware implementation
based on stochastic circuits. Furthermore, our design occupies up to 5x less
area.Comment: This paper was submitted to IEEE Transactions on Computer Aided
desig
Short-Term Plasticity and Long-Term Potentiation in Magnetic Tunnel Junctions: Towards Volatile Synapses
Synaptic memory is considered to be the main element responsible for learning
and cognition in humans. Although traditionally non-volatile long-term
plasticity changes have been implemented in nanoelectronic synapses for
neuromorphic applications, recent studies in neuroscience have revealed that
biological synapses undergo meta-stable volatile strengthening followed by a
long-term strengthening provided that the frequency of the input stimulus is
sufficiently high. Such "memory strengthening" and "memory decay"
functionalities can potentially lead to adaptive neuromorphic architectures. In
this paper, we demonstrate the close resemblance of the magnetization dynamics
of a Magnetic Tunnel Junction (MTJ) to short-term plasticity and long-term
potentiation observed in biological synapses. We illustrate that, in addition
to the magnitude and duration of the input stimulus, frequency of the stimulus
plays a critical role in determining long-term potentiation of the MTJ. Such
MTJ synaptic memory arrays can be utilized to create compact, ultra-fast and
low power intelligent neural systems.Comment: The article will appear in a future issue of Physical Review Applie
Encoding Neural and Synaptic Functionalities in Electron Spin: A Pathway to Efficient Neuromorphic Computing
Present day computers expend orders of magnitude more computational resources
to perform various cognitive and perception related tasks that humans routinely
perform everyday. This has recently resulted in a seismic shift in the field of
computation where research efforts are being directed to develop a
neurocomputer that attempts to mimic the human brain by nanoelectronic
components and thereby harness its efficiency in recognition problems. Bridging
the gap between neuroscience and nanoelectronics, this paper attempts to
provide a review of the recent developments in the field of spintronic device
based neuromorphic computing. Description of various spin-transfer torque
mechanisms that can be potentially utilized for realizing device structures
mimicking neural and synaptic functionalities is provided. A cross-layer
perspective extending from the device to the circuit and system level is
presented to envision the design of an All-Spin neuromorphic processor enabled
with on-chip learning functionalities. Device-circuit-algorithm co-simulation
framework calibrated to experimental results suggest that such All-Spin
neuromorphic systems can potentially achieve almost two orders of magnitude
energy improvement in comparison to state-of-the-art CMOS implementations.Comment: The paper will appear in a future issue of Applied Physics Review
Chaos-guided Input Structuring for Improved Learning in Recurrent Neural Networks
Anatomical studies demonstrate that brain reformats input information to
generate reliable responses for performing computations. However, it remains
unclear how neural circuits encode complex spatio-temporal patterns. We show
that neural dynamics are strongly influenced by the phase alignment between the
input and the spontaneous chaotic activity. Input structuring along the
dominant chaotic projections causes the chaotic trajectories to become stable
channels (or attractors), hence, improving the computational capability of a
recurrent network. Using mean field analysis, we derive the impact of input
structuring on the overall stability of attractors formed. Our results indicate
that input alignment determines the extent of intrinsic noise suppression and
hence, alters the attractor state stability, thereby controlling the network's
inference ability.Comment: 11 pages with 5 figures including supplementary materia
Magnetic Skyrmions for Cache Memory
Magnetic skyrmions (MS) are particle-like spin structures with whirling
configuration, which are promising candidates for spin-based memory. MS
contains alluring features including remarkably high stability, ultra low
driving current density, and compact size. Due to their higher stability and
lower drive current requirement for movement, skyrmions have great potential in
energy efficient spintronic device applications. We propose a skyrmion-based
cache memory where data can be stored in a long nanotrack as multiple bits.
Write operation (formation of skyrmion) can be achieved by injecting spin
polarized current in a magnetic nanotrack and subsequently shifting the MS in
either direction along the nanotrack using charge current through a spin-Hall
metal (SHM), underneath the magnetic layer. The presence of skyrmion can alter
the resistance of a magnetic tunneling junction (MTJ) at the read port.
Considering the read and write latency along the long nanotrack cache memory, a
strategy of multiple read and write operations is discussed. Besides, the size
of a skyrmion affects the packing density, current induced motion velocity, and
readability (change of resistance while sensing the existence of a skyrmion).
Design optimization to mitigate the above effects is also investigated
Quantum theory of light emission from quantum dots coupled to structured photonic reservoirs and acoustic phonons
Electron-phonon coupling in semiconductor quantum dots plays a significant
role in determining the optical properties of excited excitons, especially the
spectral nature of emitted photons. This paper presents a comprehensive theory
and analysis of emission spectra from artificial atoms or quantum dots coupled
to structured photon reservoirs and acoustic phonons, when excited with
incoherent pump fields. As specific examples of structured reservoirs, we chose
a Lorentzian cavity and a coupled cavity waveguide, which are of current
experimental interest. For the case of optical cavities, we directly compare
and contrast the spectra from three distinct theoretical approaches to treat
electron-phonon coupling, including a Markovian polaron master equation, a
non-Markovian phonon correlation expansion technique and a semiclassical linear
susceptibility approach, and we point out the limitations of these models. For
the cavity-QED polaron master equation, we give closed form analytical
solutions to the phonon-assisted scattering rates in the weak excitation
approximation, fully accounting for temperature, cavity-exciton detuning and
cavity dot coupling. We show explicitly why the semiclassical linear
susceptibility approach fails to correctly account for phonon-mediated cavity
feeding. For weakly coupled cavities, we calculate the optical spectra using a
more general reservoir approach and explain its differences from the above
approaches in the low Q limit of a Lorentzian cavity. We subsequently use this
general reservoir to calculate the emission spectra from quantum dots coupled
to slow-light photonic crystal waveguides, which demonstrate a number of
striking photon-phonon coupling effects. Our quantum theory can be applied to a
wide range of photonic structures including photonic molecules and
coupled-cavity waveguide systems
Controlled Forgetting: Targeted Stimulation and Dopaminergic Plasticity Modulation for Unsupervised Lifelong Learning in Spiking Neural Networks
Stochastic gradient descent requires that training samples be drawn from a
uniformly random distribution of the data. For a deployed system that must
learn online from an uncontrolled and unknown environment, the ordering of
input samples often fails to meet this criterion, making lifelong learning a
difficult challenge. We exploit the locality of the unsupervised Spike Timing
Dependent Plasticity (STDP) learning rule to target local representations in a
Spiking Neural Network (SNN) to adapt to novel information while protecting
essential information in the remainder of the SNN from catastrophic forgetting.
In our Controlled Forgetting Networks (CFNs), novel information triggers
stimulated firing and heterogeneously modulated plasticity, inspired by
biological dopamine signals, to cause rapid and isolated adaptation in the
synapses of neurons associated with outlier information. This targeting
controls the forgetting process in a way that reduces the degradation of
accuracy for older tasks while learning new tasks. Our experimental results on
the MNIST dataset validate the capability of CFNs to learn successfully over
time from an unknown, changing environment, achieving 95.36% accuracy, which we
believe is the best unsupervised accuracy ever achieved by a fixed-size,
single-layer SNN on a completely disjoint MNIST dataset
- …