2 research outputs found
An Efficient Hardware Accelerator for Structured Sparse Convolutional Neural Networks on FPGAs
Deep Convolutional Neural Networks (CNNs) have achieved state-of-the-art
performance in a wide range of applications. However, deeper CNN models, which
are usually computation consuming, are widely required for complex Artificial
Intelligence (AI) tasks. Though recent research progress on network compression
such as pruning has emerged as a promising direction to mitigate computational
burden, existing accelerators are still prevented from completely utilizing the
benefits of leveraging sparsity owing to the irregularity caused by pruning. On
the other hand, Field-Programmable Gate Arrays (FPGAs) have been regarded as a
promising hardware platform for CNN inference acceleration. However, most
existing FPGA accelerators focus on dense CNN and cannot address the
irregularity problem. In this paper, we propose a sparse wise dataflow to skip
the cycles of processing Multiply-and-Accumulates (MACs) with zero weights and
exploit data statistics to minimize energy through zeros gating to avoid
unnecessary computations. The proposed sparse wise dataflow leads to a low
bandwidth requirement and a high data sharing. Then we design an FPGA
accelerator containing a Vector Generator Module (VGM) which can match the
index between sparse weights and input activations according to the proposed
dataflow. Experimental results demonstrate that our implementation can achieve
987 imag/s and 48 imag/s performance for AlexNet and VGG-16 on Xilinx ZCU102,
respectively, which provides 1.5x to 6.7x speedup and 2.0x to 6.2x
energy-efficiency over previous CNN FPGA accelerators
Heterogeneous Dataflow Accelerators for Multi-DNN Workloads
Emerging AI-enabled applications such as augmented/virtual reality (AR/VR)
leverage multiple deep neural network (DNN) models for sub-tasks such as object
detection, hand tracking, and so on. Because of the diversity of the sub-tasks,
the layers within and across the DNN models are highly heterogeneous in
operation and shape. Such layer heterogeneity is a challenge for a fixed
dataflow accelerator (FDA) that employs a fixed dataflow on a single
accelerator substrate since each layer prefers different dataflows (computation
order and parallelization) and tile sizes. Reconfigurable DNN accelerators
(RDAs) have been proposed to adapt their dataflows to diverse layers to address
the challenge. However, the dataflow flexibility in RDAs is enabled at the area
and energy costs of expensive hardware structures (switches, controller, etc.)
and per-layer reconfiguration.
Alternatively, this work proposes a new class of accelerators, heterogeneous
dataflow accelerators (HDAs), which deploys multiple sub-accelerators each
supporting a different dataflow. HDAs enable coarser-grained dataflow
flexibility than RDAs with higher energy efficiency and lower area cost
comparable to FDAs. To exploit such benefits, hardware resource partitioning
across sub-accelerators and layer execution schedule need to be carefully
optimized. Therefore, we also present Herald, which co-optimizes hardware
partitioning and layer execution schedule. Using Herald on a suite of AR/VR and
MLPerf workloads, we identify a promising HDA architecture, Maelstrom, which
demonstrates 65.3% lower latency and 5.0% lower energy than the best FDAs and
22.0% lower energy at the cost of 20.7% higher latency than a state-of-the-art
RDA. The results suggest that HDA is an alternative class of Pareto-optimal
accelerators to RDA with strength in energy, which can be a better choice than
RDAs depending on the use cases.Comment: This paper is accepted at HPCA 202