8 research outputs found
Efficient Communication Acceleration for Next-Gen Scale-up Deep Learning Training Platforms
Deep Learning (DL) training platforms are built by interconnecting multiple
DL accelerators (e.g., GPU/TPU) via fast, customized interconnects. As the size
of DL models and the compute efficiency of the accelerators has continued to
increase, there has also been a corresponding steady increase in the bandwidth
of these interconnects.Systems today provide 100s of gigabytes (GBs) of
inter-connect bandwidth via a mix of solutions such as Multi-Chip packaging
modules (MCM) and proprietary interconnects(e.g., NVlink) that together from
the scale-up network of accelerators. However, as we identify in this work, a
significant portion of this bandwidth goes under-utilized. This is because(i)
using compute cores for executing collective operations such as all-reduce
decreases overall compute efficiency, and(ii) there is memory bandwidth
contention between the accesses for arithmetic operations vs those for
collectives, and(iii) there are significant internal bus congestions that
increase the latency of communication operations. To address this challenge, we
propose a novel microarchitecture, calledAccelerator Collectives Engine(ACE),
forDL collective communication offload. ACE is a smart net-work interface (NIC)
tuned to cope with the high-bandwidth and low latency requirements of scale-up
networks and is able to efficiently drive the various scale-up network
systems(e.g. switch-based or point-to-point topologies). We evaluate the
benefits of the ACE with micro-benchmarks (e.g. single collective performance)
and popular DL models using an end-to-end DL training simulator. For modern DL
workloads, ACE on average increases the net-work bandwidth utilization by
1.97X, resulting in 2.71X and 1.44X speedup in iteration time for ResNet-50 and
GNMT, respectively
SPRING: A Sparsity-Aware Reduced-Precision Monolithic 3D CNN Accelerator Architecture for Training and Inference
CNNs outperform traditional machine learning algorithms across a wide range
of applications. However, their computational complexity makes it necessary to
design efficient hardware accelerators. Most CNN accelerators focus on
exploring dataflow styles that exploit computational parallelism. However,
potential performance speedup from sparsity has not been adequately addressed.
The computation and memory footprint of CNNs can be significantly reduced if
sparsity is exploited in network evaluations. To take advantage of sparsity,
some accelerator designs explore sparsity encoding and evaluation on CNN
accelerators. However, sparsity encoding is just performed on activation or
weight and only in inference. It has been shown that activation and weight also
have high sparsity levels during training. Hence, sparsity-aware computation
should also be considered in training. To further improve performance and
energy efficiency, some accelerators evaluate CNNs with limited precision.
However, this is limited to the inference since reduced precision sacrifices
network accuracy if used in training. In addition, CNN evaluation is usually
memory-intensive, especially in training. In this paper, we propose SPRING, a
SParsity-aware Reduced-precision Monolithic 3D CNN accelerator for trainING and
inference. SPRING supports both CNN training and inference. It uses a binary
mask scheme to encode sparsities in activation and weight. It uses the
stochastic rounding algorithm to train CNNs with reduced precision without
accuracy loss. To alleviate the memory bottleneck in CNN evaluation, especially
in training, SPRING uses an efficient monolithic 3D NVM interface to increase
memory bandwidth. Compared to GTX 1080 Ti, SPRING achieves 15.6X, 4.2X and
66.0X improvements in performance, power reduction, and energy efficiency,
respectively, for CNN training, and 15.5X, 4.5X and 69.1X improvements for
inference