48,169 research outputs found
AxOMaP: Designing FPGA-based Approximate Arithmetic Operators using Mathematical Programming
With the increasing application of machine learning (ML) algorithms in
embedded systems, there is a rising necessity to design low-cost computer
arithmetic for these resource-constrained systems. As a result, emerging models
of computation, such as approximate and stochastic computing, that leverage the
inherent error-resilience of such algorithms are being actively explored for
implementing ML inference on resource-constrained systems. Approximate
computing (AxC) aims to provide disproportionate gains in the power,
performance, and area (PPA) of an application by allowing some level of
reduction in its behavioral accuracy (BEHAV). Using approximate operators
(AxOs) for computer arithmetic forms one of the more prevalent methods of
implementing AxC. AxOs provide the additional scope for finer granularity of
optimization, compared to only precision scaling of computer arithmetic. To
this end, designing platform-specific and cost-efficient approximate operators
forms an important research goal. Recently, multiple works have reported using
AI/ML-based approaches for synthesizing novel FPGA-based AxOs. However, most of
such works limit usage of AI/ML to designing ML-based surrogate functions used
during iterative optimization processes. To this end, we propose a novel data
analysis-driven mathematical programming-based approach to synthesizing
approximate operators for FPGAs. Specifically, we formulate mixed integer
quadratically constrained programs based on the results of correlation analysis
of the characterization data and use the solutions to enable a more directed
search approach for evolutionary optimization algorithms. Compared to
traditional evolutionary algorithms-based optimization, we report up to 21%
improvement in the hypervolume, for joint optimization of PPA and BEHAV, in the
design of signed 8-bit multipliers.Comment: 23 pages, Under review at ACM TRET
Boosting Variational Inference: an Optimization Perspective
Variational inference is a popular technique to approximate a possibly
intractable Bayesian posterior with a more tractable one. Recently, boosting
variational inference has been proposed as a new paradigm to approximate the
posterior by a mixture of densities by greedily adding components to the
mixture. However, as is the case with many other variational inference
algorithms, its theoretical properties have not been studied. In the present
work, we study the convergence properties of this approach from a modern
optimization viewpoint by establishing connections to the classic Frank-Wolfe
algorithm. Our analyses yields novel theoretical insights regarding the
sufficient conditions for convergence, explicit rates, and algorithmic
simplifications. Since a lot of focus in previous works for variational
inference has been on tractability, our work is especially important as a much
needed attempt to bridge the gap between probabilistic models and their
corresponding theoretical properties
Automatic Differentiation Variational Inference
Probabilistic modeling is iterative. A scientist posits a simple model, fits
it to her data, refines it according to her analysis, and repeats. However,
fitting complex models to large data is a bottleneck in this process. Deriving
algorithms for new models can be both mathematically and computationally
challenging, which makes it difficult to efficiently cycle through the steps.
To this end, we develop automatic differentiation variational inference (ADVI).
Using our method, the scientist only provides a probabilistic model and a
dataset, nothing else. ADVI automatically derives an efficient variational
inference algorithm, freeing the scientist to refine and explore many models.
ADVI supports a broad class of models-no conjugacy assumptions are required. We
study ADVI across ten different models and apply it to a dataset with millions
of observations. ADVI is integrated into Stan, a probabilistic programming
system; it is available for immediate use
- …