50,173 research outputs found
AxOMaP: Designing FPGA-based Approximate Arithmetic Operators using Mathematical Programming
With the increasing application of machine learning (ML) algorithms in
embedded systems, there is a rising necessity to design low-cost computer
arithmetic for these resource-constrained systems. As a result, emerging models
of computation, such as approximate and stochastic computing, that leverage the
inherent error-resilience of such algorithms are being actively explored for
implementing ML inference on resource-constrained systems. Approximate
computing (AxC) aims to provide disproportionate gains in the power,
performance, and area (PPA) of an application by allowing some level of
reduction in its behavioral accuracy (BEHAV). Using approximate operators
(AxOs) for computer arithmetic forms one of the more prevalent methods of
implementing AxC. AxOs provide the additional scope for finer granularity of
optimization, compared to only precision scaling of computer arithmetic. To
this end, designing platform-specific and cost-efficient approximate operators
forms an important research goal. Recently, multiple works have reported using
AI/ML-based approaches for synthesizing novel FPGA-based AxOs. However, most of
such works limit usage of AI/ML to designing ML-based surrogate functions used
during iterative optimization processes. To this end, we propose a novel data
analysis-driven mathematical programming-based approach to synthesizing
approximate operators for FPGAs. Specifically, we formulate mixed integer
quadratically constrained programs based on the results of correlation analysis
of the characterization data and use the solutions to enable a more directed
search approach for evolutionary optimization algorithms. Compared to
traditional evolutionary algorithms-based optimization, we report up to 21%
improvement in the hypervolume, for joint optimization of PPA and BEHAV, in the
design of signed 8-bit multipliers.Comment: 23 pages, Under review at ACM TRET
Boosting Variational Inference: an Optimization Perspective
Variational inference is a popular technique to approximate a possibly
intractable Bayesian posterior with a more tractable one. Recently, boosting
variational inference has been proposed as a new paradigm to approximate the
posterior by a mixture of densities by greedily adding components to the
mixture. However, as is the case with many other variational inference
algorithms, its theoretical properties have not been studied. In the present
work, we study the convergence properties of this approach from a modern
optimization viewpoint by establishing connections to the classic Frank-Wolfe
algorithm. Our analyses yields novel theoretical insights regarding the
sufficient conditions for convergence, explicit rates, and algorithmic
simplifications. Since a lot of focus in previous works for variational
inference has been on tractability, our work is especially important as a much
needed attempt to bridge the gap between probabilistic models and their
corresponding theoretical properties
- …