242 research outputs found
Neuroinspired unsupervised learning and pruning with subquantum CBRAM arrays.
Resistive RAM crossbar arrays offer an attractive solution to minimize off-chip data transfer and parallelize on-chip computations for neural networks. Here, we report a hardware/software co-design approach based on low energy subquantum conductive bridging RAM (CBRAM®) devices and a network pruning technique to reduce network level energy consumption. First, we demonstrate low energy subquantum CBRAM devices exhibiting gradual switching characteristics important for implementing weight updates in hardware during unsupervised learning. Then we develop a network pruning algorithm that can be employed during training, different from previous network pruning approaches applied for inference only. Using a 512 kbit subquantum CBRAM array, we experimentally demonstrate high recognition accuracy on the MNIST dataset for digital implementation of unsupervised learning. Our hardware/software co-design approach can pave the way towards resistive memory based neuro-inspired systems that can autonomously learn and process information in power-limited settings
Towards Accurate and High-Speed Spiking Neuromorphic Systems with Data Quantization-Aware Deep Networks
Deep Neural Networks (DNNs) have gained immense success in cognitive
applications and greatly pushed today's artificial intelligence forward. The
biggest challenge in executing DNNs is their extremely data-extensive
computations. The computing efficiency in speed and energy is constrained when
traditional computing platforms are employed in such computational hungry
executions. Spiking neuromorphic computing (SNC) has been widely investigated
in deep networks implementation own to their high efficiency in computation and
communication. However, weights and signals of DNNs are required to be
quantized when deploying the DNNs on the SNC, which results in unacceptable
accuracy loss. %However, the system accuracy is limited by quantizing data
directly in deep networks deployment. Previous works mainly focus on weights
discretize while inter-layer signals are mainly neglected. In this work, we
propose to represent DNNs with fixed integer inter-layer signals and
fixed-point weights while holding good accuracy. We implement the proposed DNNs
on the memristor-based SNC system as a deployment example. With 4-bit data
representation, our results show that the accuracy loss can be controlled
within 0.02% (2.3%) on MNIST (CIFAR-10). Compared with the 8-bit dynamic
fixed-point DNNs, our system can achieve more than 9.8x speedup, 89.1% energy
saving, and 30% area saving.Comment: 6 pages, 4 figure
EPIM: Efficient Processing-In-Memory Accelerators based on Epitome
The exploration of Processing-In-Memory (PIM) accelerators has garnered
significant attention within the research community. However, the utilization
of large-scale neural networks on Processing-In-Memory (PIM) accelerators
encounters challenges due to constrained on-chip memory capacity. To tackle
this issue, current works explore model compression algorithms to reduce the
size of Convolutional Neural Networks (CNNs). Most of these algorithms either
aim to represent neural operators with reduced-size parameters (e.g.,
quantization) or search for the best combinations of neural operators (e.g.,
neural architecture search). Designing neural operators to align with PIM
accelerators' specifications is an area that warrants further study. In this
paper, we introduce the Epitome, a lightweight neural operator offering
convolution-like functionality, to craft memory-efficient CNN operators for PIM
accelerators (EPIM). On the software side, we evaluate epitomes' latency and
energy on PIM accelerators and introduce a PIM-aware layer-wise design method
to enhance their hardware efficiency. We apply epitome-aware quantization to
further reduce the size of epitomes. On the hardware side, we modify the
datapath of current PIM accelerators to accommodate epitomes and implement a
feature map reuse technique to reduce computation cost. Experimental results
reveal that our 3-bit quantized EPIM-ResNet50 attains 71.59% top-1 accuracy on
ImageNet, reducing crossbar areas by 30.65 times. EPIM surpasses the
state-of-the-art pruning methods on PIM
- …