21 research outputs found
HAPI: Hardware-Aware Progressive Inference
Convolutional neural networks (CNNs) have recently become the
state-of-the-art in a diversity of AI tasks. Despite their popularity, CNN
inference still comes at a high computational cost. A growing body of work aims
to alleviate this by exploiting the difference in the classification difficulty
among samples and early-exiting at different stages of the network.
Nevertheless, existing studies on early exiting have primarily focused on the
training scheme, without considering the use-case requirements or the
deployment platform. This work presents HAPI, a novel methodology for
generating high-performance early-exit networks by co-optimising the placement
of intermediate exits together with the early-exit strategy at inference time.
Furthermore, we propose an efficient design space exploration algorithm which
enables the faster traversal of a large number of alternative architectures and
generates the highest-performing design, tailored to the use-case requirements
and target hardware. Quantitative evaluation shows that our system consistently
outperforms alternative search mechanisms and state-of-the-art early-exit
schemes across various latency budgets. Moreover, it pushes further the
performance of highly optimised hand-crafted early-exit CNNs, delivering up to
5.11x speedup over lightweight models on imposed latency-driven SLAs for
embedded devices.Comment: Accepted at the 39th International Conference on Computer-Aided
Design (ICCAD), 202
Recommended from our members
A Fuzzy Approach for Determining a Feasible Point in a Constrained Problem
In an engineering design process, it is often desirable or even necessary to find a feasible starting point before finding the actual optimal design. The identification of unapparent feasible points provides problem insight with little computational cost. In addition, some optimization algorithms either require, or function more efficiently with, an initial point that satisfies all constraints. The advantages of using an initial feasible design are especially evident with complex structural problems and with problems that involve potential safety concerns such as the design of large pressure vessels. Although some methods are available, literature concerning the development and use of feasible-point determination is scarce. In this paper, fuzzy function theory is used to develop a novel, simple, and efficient algorithm for determining a feasible point by starting with a potentially infeasible point. Each constraint is written in fuzzy form with a newly developed membership function. The resulting optimization problem is easily solved using conventional techniques. This approach is applied to structural and manufacturing problems, including the design of a general pressure vessel. When compared with an established method, the new approach is successful and more than twenty-five times faster with larger problems