223 research outputs found
Tiramisu: A Polyhedral Compiler for Expressing Fast and Portable Code
This paper introduces Tiramisu, a polyhedral framework designed to generate
high performance code for multiple platforms including multicores, GPUs, and
distributed machines. Tiramisu introduces a scheduling language with novel
extensions to explicitly manage the complexities that arise when targeting
these systems. The framework is designed for the areas of image processing,
stencils, linear algebra and deep learning. Tiramisu has two main features: it
relies on a flexible representation based on the polyhedral model and it has a
rich scheduling language allowing fine-grained control of optimizations.
Tiramisu uses a four-level intermediate representation that allows full
separation between the algorithms, loop transformations, data layouts, and
communication. This separation simplifies targeting multiple hardware
architectures with the same algorithm. We evaluate Tiramisu by writing a set of
image processing, deep learning, and linear algebra benchmarks and compare them
with state-of-the-art compilers and hand-tuned libraries. We show that Tiramisu
matches or outperforms existing compilers and libraries on different hardware
architectures, including multicore CPUs, GPUs, and distributed machines.Comment: arXiv admin note: substantial text overlap with arXiv:1803.0041
Accelerating and pruning CNNs for semantic segmentation on FPGA
Semantic segmentation is one of the popular tasks in computer vision, providing pixel-wise annotations for scene understanding. However, segmentation-based convolutional neural networks require tremendous computational power. In this work, a fully-pipelined hardware accelerator with support for dilated convolution is introduced, which cuts down the redundant zero multiplications. Furthermore, we propose a genetic algorithm based automated channel pruning technique to jointly optimize computational complexity and model accuracy. Finally, hardware heuristics and an accurate model of the custom accelerator design enable a hardware-aware pruning framework. We achieve 2.44X lower latency with minimal degradation in semantic prediction quality (−1.98 pp lower mean intersection over union) compared to the baseline DeepLabV3+ model, evaluated on an Arria-10 FPGA. The binary files of the FPGA design, baseline and pruned models can be found in github.com/pierpaolomori/SemanticSegmentationFPGA
Fused Depthwise Tiling for Memory Optimization in TinyML Deep Neural Network Inference
Memory optimization for deep neural network (DNN) inference gains high
relevance with the emergence of TinyML, which refers to the deployment of DNN
inference tasks on tiny, low-power microcontrollers. Applications such as audio
keyword detection or radar-based gesture recognition are heavily constrained by
the limited memory on such tiny devices because DNN inference requires large
intermediate run-time buffers to store activations and other intermediate data,
which leads to high memory usage. In this paper, we propose a new Fused
Depthwise Tiling (FDT) method for the memory optimization of DNNs, which,
compared to existing tiling methods, reduces memory usage without inducing any
run time overhead. FDT applies to a larger variety of network layers than
existing tiling methods that focus on convolutions. It improves TinyML memory
optimization significantly by reducing memory of models where this was not
possible before and additionally providing alternative design points for models
that show high run time overhead with existing methods. In order to identify
the best tiling configuration, an end-to-end flow with a new path discovery
method is proposed, which applies FDT and existing tiling methods in a fully
automated way, including the scheduling of the operations and planning of the
layout of buffers in memory. Out of seven evaluated models, FDT achieved
significant memory reduction for two models by 76.2% and 18.1% where existing
tiling methods could not be applied. Two other models showed a significant run
time overhead with existing methods and FDT provided alternative design points
with no overhead but reduced memory savings.Comment: Accepted as a full paper by the TinyML Research Symposium 202
Recommended from our members
Languages and Compilers for Writing Efficient High-Performance Computing Applications
Many everyday applications, such as web search, speech recognition, and weather prediction, are executed on high-performance systems containing thousands of Central Processing Units (CPUs) and Graphics Processing Units (GPUs). These applications can be written in either low-level programming languages, such as NVIDIA CUDA, or domain specific languages, like Halide for image processing and PyTorch for machine learning programs. Despite the popularity of these languages, there are several challenges that programmers face when developing efficient high-performance computing applications. First, since every hardware support a different low-level programming model, to utilize new hardware programmers need to rewrite their applications in another programming language. Second, writing efficient code involves restructuring the computation to ensure (i) regular memory access patterns, (ii) non-divergent control flow, and (iii) complete utilization of different programmer managed caches. Furthermore, since these low-level optimizations are known only to hardware experts, it is difficult for a domain expert to write optimized code for new computations. Third, existing domain specific languages suffer from optimization barriers in the language constructs that prevent new optimizations and hence, these languages provide sub-optimal performance. To address these challenges this thesis presents the following novel abstractions and compiler techniques for writing image processing and machine learning applications that can run efficiently on a variety of high-performance systems. First, this thesis presents techniques to optimize image processing programs on GPUs using the features of modern GPUs. These techniques improve the concurrency and register usage of generated code to provide better performance than the state-of-the-art. Second, this thesis presents NextDoor, which is the first system to provide an abstraction for writing graph sampling applications and efficiently executing these applications on GPUs. Third, this thesis presents CoCoNet, which is a domain specific language to co-optimize communication and computation in distributed machine learning workloads. By breaking the optimization barriers in existing domain specific languages, these techniques help programmers write correct and efficient code for diverse high-performance computing workloads
PolyDL: Polyhedral Optimizations for Creation of High Performance DL primitives
Deep Neural Networks (DNNs) have revolutionized many aspects of our lives.
The use of DNNs is becoming ubiquitous including in softwares for image
recognition, speech recognition, speech synthesis, language translation, to
name a few. he training of DNN architectures however is computationally
expensive. Once the model is created, its use in the intended application - the
inference task, is computationally heavy too and the inference needs to be fast
for real time use. For obtaining high performance today, the code of Deep
Learning (DL) primitives optimized for specific architectures by expert
programmers exposed via libraries is the norm. However, given the constant
emergence of new DNN architectures, creating hand optimized code is expensive,
slow and is not scalable.
To address this performance-productivity challenge, in this paper we present
compiler algorithms to automatically generate high performance implementations
of DL primitives that closely match the performance of hand optimized
libraries. We develop novel data reuse analysis algorithms using the polyhedral
model to derive efficient execution schedules automatically. In addition,
because most DL primitives use some variant of matrix multiplication at their
core, we develop a flexible framework where it is possible to plug in library
implementations of the same in lieu of a subset of the loops. We show that such
a hybrid compiler plus a minimal library-use approach results in
state-of-the-art performance. We develop compiler algorithms to also perform
operator fusions that reduce data movement through the memory hierarchy of the
computer system.Comment: arXiv admin note: substantial text overlap with arXiv:2002.0214
- …