1,449 research outputs found
Learning to infer: RL-based search for DNN primitive selection on Heterogeneous Embedded Systems
Deep Learning is increasingly being adopted by industry for computer vision
applications running on embedded devices. While Convolutional Neural Networks'
accuracy has achieved a mature and remarkable state, inference latency and
throughput are a major concern especially when targeting low-cost and low-power
embedded platforms. CNNs' inference latency may become a bottleneck for Deep
Learning adoption by industry, as it is a crucial specification for many
real-time processes. Furthermore, deployment of CNNs across heterogeneous
platforms presents major compatibility issues due to vendor-specific technology
and acceleration libraries. In this work, we present QS-DNN, a fully automatic
search based on Reinforcement Learning which, combined with an inference engine
optimizer, efficiently explores through the design space and empirically finds
the optimal combinations of libraries and primitives to speed up the inference
of CNNs on heterogeneous embedded devices. We show that, an optimized
combination can achieve 45x speedup in inference latency on CPU compared to a
dependency-free baseline and 2x on average on GPGPU compared to the best vendor
library. Further, we demonstrate that, the quality of results and time
"to-solution" is much better than with Random Search and achieves up to 15x
better results for a short-time search
Automatic generation of multi-precision multi-arithmetic CNN accelerators for FPGAs
Modern deep Convolutional Neural Networks (CNNs) are computationally
demanding, yet real applications often require high throughput and low latency.
To help tackle these problems, we propose Tomato, a framework designed to
automate the process of generating efficient CNN accelerators. The generated
design is pipelined and each convolution layer uses different arithmetics at
various precisions. Using Tomato, we showcase state-of-the-art multi-precision
multi-arithmetic networks, including MobileNet-V1, running on FPGAs. To our
knowledge, this is the first multi-precision multi-arithmetic auto-generation
framework for CNNs. In software, Tomato fine-tunes pretrained networks to use a
mixture of short powers-of-2 and fixed-point weights with a minimal loss in
classification accuracy. The fine-tuned parameters are combined with the
templated hardware designs to automatically produce efficient inference
circuits in FPGAs. We demonstrate how our approach significantly reduces model
sizes and computation complexities, and permits us to pack a complete ImageNet
network onto a single FPGA without accessing off-chip memories for the first
time. Furthermore, we show how Tomato produces implementations of networks with
various sizes running on single or multiple FPGAs. To the best of our
knowledge, our automatically generated accelerators outperform closest
FPGA-based competitors by at least 2-4x for lantency and throughput; the
generated accelerator runs ImageNet classification at a rate of more than 3000
frames per second.EPSRC Doctoral Scholarship
Peterhouse Graduate Studentshi
- …