21 research outputs found
XNORBIN: A 95 TOp/s/W Hardware Accelerator for Binary Convolutional Neural Networks
Deploying state-of-the-art CNNs requires power-hungry processors and off-chip
memory. This precludes the implementation of CNNs in low-power embedded
systems. Recent research shows CNNs sustain extreme quantization, binarizing
their weights and intermediate feature maps, thereby saving 8-32\x memory and
collapsing energy-intensive sum-of-products into XNOR-and-popcount operations.
We present XNORBIN, an accelerator for binary CNNs with computation tightly
coupled to memory for aggressive data reuse. Implemented in UMC 65nm technology
XNORBIN achieves an energy efficiency of 95 TOp/s/W and an area efficiency of
2.0 TOp/s/MGE at 0.8 V
WHYPE: A Scale-Out Architecture with Wireless Over-the-Air Majority for Scalable In-memory Hyperdimensional Computing
Hyperdimensional computing (HDC) is an emerging computing paradigm that
represents, manipulates, and communicates data using long random vectors known
as hypervectors. Among different hardware platforms capable of executing HDC
algorithms, in-memory computing (IMC) has shown promise as it is very efficient
in performing matrix-vector multiplications, which are common in the HDC
algebra. Although HDC architectures based on IMC already exist, how to scale
them remains a key challenge due to collective communication patterns that
these architectures required and that traditional chip-scale networks were not
designed for. To cope with this difficulty, we propose a scale-out HDC
architecture called WHYPE, which uses wireless in-package communication
technology to interconnect a large number of physically distributed IMC cores
that either encode hypervectors or perform multiple similarity searches in
parallel. In this context, the key enabler of WHYPE is the opportunistic use of
the wireless network as a medium for over-the-air computation. WHYPE implements
an optimized source coding that allows receivers to calculate the bit-wise
majority of multiple hypervectors (a useful operation in HDC) being transmitted
concurrently over the wireless channel. By doing so, we achieve a joint
broadcast distribution and computation with a performance and efficiency
unattainable with wired interconnects, which in turn enables massive
parallelization of the architecture. Through evaluations at the on-chip network
and complete architecture levels, we demonstrate that WHYPE can bundle and
distribute hypervectors faster and more efficiently than a hypothetical wired
implementation, and that it scales well to tens of receivers. We show that the
average error rate of the majority computation is low, such that it has
negligible impact on the accuracy of HDC classification tasks.Comment: Accepted at IEEE Journal on Emerging and Selected Topics in Circuits
and Systems (JETCAS). arXiv admin note: text overlap with arXiv:2205.1088
ESSOP: Efficient and Scalable Stochastic Outer Product Architecture for Deep Learning
Deep neural networks (DNNs) have surpassed human-level accuracy in a variety
of cognitive tasks but at the cost of significant memory/time requirements in
DNN training. This limits their deployment in energy and memory limited
applications that require real-time learning. Matrix-vector multiplications
(MVM) and vector-vector outer product (VVOP) are the two most expensive
operations associated with the training of DNNs. Strategies to improve the
efficiency of MVM computation in hardware have been demonstrated with minimal
impact on training accuracy. However, the VVOP computation remains a relatively
less explored bottleneck even with the aforementioned strategies. Stochastic
computing (SC) has been proposed to improve the efficiency of VVOP computation
but on relatively shallow networks with bounded activation functions and
floating-point (FP) scaling of activation gradients. In this paper, we propose
ESSOP, an efficient and scalable stochastic outer product architecture based on
the SC paradigm. We introduce efficient techniques to generalize SC for weight
update computation in DNNs with the unbounded activation functions (e.g.,
ReLU), required by many state-of-the-art networks. Our architecture reduces the
computational cost by re-using random numbers and replacing certain FP
multiplication operations by bit shift scaling. We show that the ResNet-32
network with 33 convolution layers and a fully-connected layer can be trained
with ESSOP on the CIFAR-10 dataset to achieve baseline comparable accuracy.
Hardware design of ESSOP at 14nm technology node shows that, compared to a
highly pipelined FP16 multiplier design, ESSOP is 82.2% and 93.7% better in
energy and area efficiency respectively for outer product computation.Comment: 5 pages. 5 figures. Accepted at ISCAS 2020 for publicatio
Scale up your In-Memory Accelerator: leveraging wireless-on-chip communication for AIMC-based CNN inference
Analog In-Memory Computing (AIMC) is emerging as a disruptive paradigm for heterogeneous computing, potentially delivering orders of magnitude better peak performance and efficiency over traditional digital signal processing architectures on Matrix-Vector multiplication. However, to sustain this throughput in real-world applications, AIMC tiles must be supplied with data at very high bandwidth and low latency; this poses an unprecedented pressure on the on-chip communication infrastructure, which becomes the system's performance and efficiency bottleneck. In this context, the performance and plasticity of emerging on-chip wireless communication paradigms provide the required breakthrough to up-scale on-chip communication in large AIMC devices. This work presents a many-tile AIMC architecture with inter-tile wireless communication that integrates multiple heterogeneous computing clusters, embedding a mix of parallel RISC-V cores and AIMC tiles. We perform an extensive design space exploration of the proposed architecture and discuss the benefits of exploiting emerging on-chip communication technologies such as wireless transceivers in the millimeter-wave and terahertz bands.This work was supported by the WiPLASH project (g.a. 863337), founded from the European Union’s Horizon 2020 research and innovation program.Peer ReviewedPostprint (author's final draft
Mixed-precision deep learning based on computational memory
Deep neural networks (DNNs) have revolutionized the field of artificial
intelligence and have achieved unprecedented success in cognitive tasks such as
image and speech recognition. Training of large DNNs, however, is
computationally intensive and this has motivated the search for novel computing
architectures targeting this application. A computational memory unit with
nanoscale resistive memory devices organized in crossbar arrays could store the
synaptic weights in their conductance states and perform the expensive weighted
summations in place in a non-von Neumann manner. However, updating the
conductance states in a reliable manner during the weight update process is a
fundamental challenge that limits the training accuracy of such an
implementation. Here, we propose a mixed-precision architecture that combines a
computational memory unit performing the weighted summations and imprecise
conductance updates with a digital processing unit that accumulates the weight
updates in high precision. A combined hardware/software training experiment of
a multilayer perceptron based on the proposed architecture using a phase-change
memory (PCM) array achieves 97.73% test accuracy on the task of classifying
handwritten digits (based on the MNIST dataset), within 0.6% of the software
baseline. The architecture is further evaluated using accurate behavioral
models of PCM on a wide class of networks, namely convolutional neural
networks, long-short-term-memory networks, and generative-adversarial networks.
Accuracies comparable to those of floating-point implementations are achieved
without being constrained by the non-idealities associated with the PCM
devices. A system-level study demonstrates 173x improvement in energy
efficiency of the architecture when used for training a multilayer perceptron
compared with a dedicated fully digital 32-bit implementation
In-memory Realization of In-situ Few-shot Continual Learning with a Dynamically Evolving Explicit Memory
Continually learning new classes from a few training examples without
forgetting previous old classes demands a flexible architecture with an
inevitably growing portion of storage, in which new examples and classes can be
incrementally stored and efficiently retrieved. One viable architectural
solution is to tightly couple a stationary deep neural network to a dynamically
evolving explicit memory (EM). As the centerpiece of this architecture, we
propose an EM unit that leverages energy-efficient in-memory compute (IMC)
cores during the course of continual learning operations. We demonstrate for
the first time how the EM unit can physically superpose multiple training
examples, expand to accommodate unseen classes, and perform similarity search
during inference, using operations on an IMC core based on phase-change memory
(PCM). Specifically, the physical superposition of a few encoded training
examples is realized via in-situ progressive crystallization of PCM devices.
The classification accuracy achieved on the IMC core remains within a range of
1.28%--2.5% compared to that of the state-of-the-art full-precision baseline
software model on both the CIFAR-100 and miniImageNet datasets when continually
learning 40 novel classes (from only five examples per class) on top of 60 old
classes.Comment: Accepted at the European Solid-state Devices and Circuits Conference
(ESSDERC), September 202
Graphene-based Wireless Agile Interconnects for Massive Heterogeneous Multi-chip Processors
The main design principles in computer architecture have recently shifted
from a monolithic scaling-driven approach to the development of heterogeneous
architectures that tightly co-integrate multiple specialized processor and
memory chiplets. In such data-hungry multi-chip architectures, current
Networks-in-Package (NiPs) may not be enough to cater to their heterogeneous
and fast-changing communication demands. This position paper makes the case for
wireless in-package nanonetworking as the enabler of efficient and versatile
wired-wireless interconnect fabrics for massive heterogeneous processors. To
that end, the use of graphene-based antennas and transceivers with unique
frequency-beam reconfigurability in the terahertz band is proposed. The
feasibility of such a nanonetworking vision and the main research challenges
towards its realization are analyzed from the technological, communications,
and computer architecture perspectives.Comment: 8 pages, 4 figures, 1 table - Accepted at IEEE Wireless
Communications Magazin
In-memory Vector Symbolic Architectures
The field of Artificial Intelligence (AI) has achieved enormous progress in the past decade thanks primarily to deep neural network architectures and specialized hardware that support training the models within a reasonable time. However, since then a trend has emerged where, for solving increasingly difficult cognitive tasks, the model complexity in terms of the number of parameters and energy spent on training the models and the size of the datasets used for benchmarking has grown steadily every year. Yet, the capabilities of each model are limited to a narrow task such as classification or translation. The validity of this approach in building bigger and more power-hungry models needs to be critically questioned. For example, the human brain originally conceived from a tip of a 3-millimeter-long neural tube slowly grows over a period of approximately 20 years into a device that can perform a wide variety of complex cognitive tasks. It operates at a frugal power budget as low as 20W, requiring fewer examples than an AI model to learn new concepts.
The emerging brain-inspired computing paradigm known as vector symbolic architecture (VSA) offers interesting avenues to advance the field of AI along the path of how the human brain works. For starters, it only requires a few examples for training and does not entail computationally expensive iterative gradient updates. It however requires data to be represented in extremely high dimensional vectors, having dimensions typically in the order of thousands, certainly larger than the size of the data path of any classical computer. Because of its necessity to manipulate these high dimensional (HD) vectors, the intensity of memory accesses dominates the computational intensity, creating a bottleneck in conventional von Neumann computing architectures.
In-memory computing (IMC) on the other hand is a type of so-called non-von Neumann computing architecture that can bring down the cost of data movement between the processing unit and memory unit by keeping the majority of data stationary in memory and executing parallel computations using enhanced peripheral circuits. At the core of IMC, computations are performed on a noisy analog fabric by exploiting the laws of physics. Thus results are not guaranteed to be deterministic, as they are reproducible only in a probabilistic sense, leading to an interesting set of opportunities but at the same time challenges that need to be carefully considered.
This doctoral thesis investigates the usability of vector symbolic architectures on IMC hardware. It entails the following key contributions: 1) Design of VSA and neuro-VSA architectures for several applications, namely categorical and numerical sequence encoding, few-shot learning, and few-shot continual learning. In each application, the models are engineered by taking the strengths and limitations of in-memory computing architecture into consideration. 2) Simulation of the architectures using IMC models and experiment with IMC prototype chips to characterize the model performance in terms of accuracy and robustness. 3) Design of full systems with CMOS peripherals complementing the IMC tiles and estimate the power, performance, and area of the proposed systems.
The following is a brief overview of what is presented in each chapter. Chapter 1 provides the background and motivation for implementing VSA models on IMC hardware. Chapter 2 introduces the concept of in-memory hyperdimensional computing, where it is shown how VSA modules can be mapped to IMC hardware taking categorical symbol sequence classification as an example. Chapter 3 generalizes this idea to include encoding for spatio-temporal signals which, encompass not only categorical but also numerical symbols. To directly work with raw data, in Chapter 4, a novel neuro-VSA architecture and training methodology are proposed with an application for the few-shot learning problem, backed up with simulation and experimental results from IMC models and real IMC hardware, respectively. Chapter 5 and Chapter 6 present an extended version of the neuro-VSA architecture for the few-shot continual learning problem with an emphasis on software and hardware innovations respectively. Finally, Chapter 7, provides conclusions to the thesis and outlook for future research directions