606 research outputs found
Ansor : Generating High-Performance Tensor Programs for Deep Learning
High-performance tensor programs are crucial to guarantee efficient execution
of deep neural networks. However, obtaining performant tensor programs for
different operators on various hardware platforms is notoriously challenging.
Currently, deep learning systems rely on vendor-provided kernel libraries or
various search strategies to get performant tensor programs. These approaches
either require significant engineering effort to develop platform-specific
optimization code or fall short of finding high-performance programs due to
restricted search space and ineffective exploration strategy.
We present Ansor, a tensor program generation framework for deep learning
applications. Compared with existing search strategies, Ansor explores many
more optimization combinations by sampling programs from a hierarchical
representation of the search space. Ansor then fine-tunes the sampled programs
with evolutionary search and a learned cost model to identify the best
programs. Ansor can find high-performance programs that are outside the search
space of existing state-of-the-art approaches. In addition, Ansor utilizes a
task scheduler to simultaneously optimize multiple subgraphs in deep neural
networks. We show that Ansor improves the execution performance of deep neural
networks relative to the state-of-the-art on the Intel CPU, ARM CPU, and NVIDIA
GPU by up to , , and , respectively.Comment: Published in OSDI 202
LoopTune: Optimizing Tensor Computations with Reinforcement Learning
Advanced compiler technology is crucial for enabling machine learning
applications to run on novel hardware, but traditional compilers fail to
deliver performance, popular auto-tuners have long search times and
expert-optimized libraries introduce unsustainable costs. To address this, we
developed LoopTune, a deep reinforcement learning compiler that optimizes
tensor computations in deep learning models for the CPU. LoopTune optimizes
tensor traversal order while using the ultra-fast lightweight code generator
LoopNest to perform hardware-specific optimizations. With a novel graph-based
representation and action space, LoopTune speeds up LoopNest by 3.2x,
generating an order of magnitude faster code than TVM, 2.8x faster than
MetaSchedule, and 1.08x faster than AutoTVM, consistently performing at the
level of the hand-tuned library Numpy. Moreover, LoopTune tunes code in order
of seconds
Relay: A New IR for Machine Learning Frameworks
Machine learning powers diverse services in industry including search,
translation, recommendation systems, and security. The scale and importance of
these models require that they be efficient, expressive, and portable across an
array of heterogeneous hardware devices. These constraints are often at odds;
in order to better accommodate them we propose a new high-level intermediate
representation (IR) called Relay. Relay is being designed as a
purely-functional, statically-typed language with the goal of balancing
efficient compilation, expressiveness, and portability. We discuss the goals of
Relay and highlight its important design constraints. Our prototype is part of
the open source NNVM compiler framework, which powers Amazon's deep learning
framework MxNet
OpenTuner: An Extensible Framework for Program Autotuning
Program autotuning has been shown to achieve better or more portable performance in a number of domains. However, autotuners themselves are rarely portable between projects, for a number of reasons: using a domain-informed search space representation is critical to achieving good results; search spaces can be intractably large and require advanced machine learning techniques; and the landscape of search spaces can vary greatly between different problems, sometimes requiring domain specific search techniques to explore efficiently. This paper introduces OpenTuner, a new open source framework for building domain-specific multi-objective program autotuners. OpenTuner supports fully-customizable configuration representations, an extensible technique representation to allow for domain-specific techniques, and an easy to use interface for communicating with the program to be autotuned. A key capability inside OpenTuner is the use of ensembles of disparate search techniques simultaneously; techniques that perform well will dynamically be allocated a larger proportion of tests. We demonstrate the efficacy and generality of OpenTuner by building autotuners for 6 distinct projects and 14 total benchmarks, showing speedups over prior techniques of these projects of up to 2.8x with little programmer effort.This work is partially supported by DOE award DE-SC0005288 and DOD DARPA award HR0011-10-9-0009. This research used resources of the National Energy Research Scientific Computing Center, which is supported by the Office of Science of the U.S. Department of Energy under Contract No. DE-AC02-05CH11231
TensorIR: An Abstraction for Automatic Tensorized Program Optimization
Deploying deep learning models on various devices has become an important
topic. The wave of hardware specialization brings a diverse set of acceleration
primitives for multi-dimensional tensor computations. These new acceleration
primitives, along with the emerging machine learning models, bring tremendous
engineering challenges. In this paper, we present TensorIR, a compiler
abstraction for optimizing programs with these tensor computation primitives.
TensorIR generalizes the loop nest representation used in existing machine
learning compilers to bring tensor computation as the first-class citizen.
Finally, we build an end-to-end framework on top of our abstraction to
automatically optimize deep learning models for given tensor computation
primitives. Experimental results show that TensorIR compilation automatically
uses the tensor computation primitives for given hardware backends and delivers
performance that is competitive to state-of-art hand-optimized systems across
platforms.Comment: Accepted to ASPLOS 202
- …