A Factor Graph Approach to Automated Design of Bayesian Signal Processing Algorithms

The benefits of automating design cycles for Bayesian inference-based algorithms are becoming increasingly recognized by the machine learning community. As a result, interest in probabilistic programming frameworks has much increased over the past few years. This paper explores a specific probabilistic programming paradigm, namely message passing in Forney-style factor graphs (FFGs), in the context of automated design of efficient Bayesian signal processing algorithms. To this end, we developed "ForneyLab" (https://github.com/biaslab/ForneyLab.jl) as a Julia toolbox for message passing-based inference in FFGs. We show by example how ForneyLab enables automatic derivation of Bayesian signal processing algorithms, including algorithms for parameter estimation and model comparison. Crucially, due to the modular makeup of the FFG framework, both the model specification and inference methods are readily extensible in ForneyLab. In order to test this framework, we compared variational message passing as implemented by ForneyLab with automatic differentiation variational inference (ADVI) and Monte Carlo methods as implemented by state-of-the-art tools "Edward" and "Stan". In terms of performance, extensibility and stability issues, ForneyLab appears to enjoy an edge relative to its competitors for automated inference in state-space models.Comment: Accepted for publication in the International Journal of Approximate Reasonin

A low multiplicative complexity fast recursive DCT-2 algorithm

A fast Discrete Cosine Transform (DCT) algorithm is introduced that can be of particular interest in image processing. The main features of the algorithm are regularity of the graph and very low arithmetic complexity. The 16-point version of the algorithm requires only 32 multiplications and 81 additions. The computational core of the algorithm consists of only 17 nontrivial multiplications, the rest 15 are scaling factors that can be compensated in the post-processing. The derivation of the algorithm is based on the algebraic signal processing theory (ASP).Comment: 4 pages, 2 figure

Type-II/III DCT/DST algorithms with reduced number of arithmetic operations

We present algorithms for the discrete cosine transform (DCT) and discrete sine transform (DST), of types II and III, that achieve a lower count of real multiplications and additions than previously published algorithms, without sacrificing numerical accuracy. Asymptotically, the operation count is reduced from ~ 2N log_2 N to ~ (17/9) N log_2 N for a power-of-two transform size N. Furthermore, we show that a further N multiplications may be saved by a certain rescaling of the inputs or outputs, generalizing a well-known technique for N=8 by Arai et al. These results are derived by considering the DCT to be a special case of a DFT of length 4N, with certain symmetries, and then pruning redundant operations from a recent improved fast Fourier transform algorithm (based on a recursive rescaling of the conjugate-pair split radix algorithm). The improved algorithms for DCT-III, DST-II, and DST-III follow immediately from the improved count for the DCT-II.Comment: 9 page

Estimation-based synthesis of H∞-optimal adaptive FIR filtersfor filtered-LMS problems

This paper presents a systematic synthesis procedure for H∞-optimal adaptive FIR filters in the context of an active noise cancellation (ANC) problem. An estimation interpretation of the adaptive control problem is introduced first. Based on this interpretation, an H∞ estimation problem is formulated, and its finite horizon prediction (filtering) solution is discussed. The solution minimizes the maximum energy gain from the disturbances to the predicted (filtered) estimation error and serves as the adaptation criterion for the weight vector in the adaptive FIR filter. We refer to this adaptation scheme as estimation-based adaptive filtering (EBAF). We show that the steady-state gain vector in the EBAF algorithm approaches that of the classical (normalized) filtered-X LMS algorithm. The error terms, however, are shown to be different. Thus, these classical algorithms can be considered to be approximations of our algorithm. We examine the performance of the proposed EBAF algorithm (both experimentally and in simulation) in an active noise cancellation problem of a one-dimensional (1-D) acoustic duct for both narrowband and broadband cases. Comparisons to the results from a conventional filtered-LMS (FxLMS) algorithm show faster convergence without compromising steady-state performance and/or robustness of the algorithm to feedback contamination of the reference signal

A Phase Vocoder based on Nonstationary Gabor Frames

We propose a new algorithm for time stretching music signals based on the theory of nonstationary Gabor frames (NSGFs). The algorithm extends the techniques of the classical phase vocoder (PV) by incorporating adaptive time-frequency (TF) representations and adaptive phase locking. The adaptive TF representations imply good time resolution for the onsets of attack transients and good frequency resolution for the sinusoidal components. We estimate the phase values only at peak channels and the remaining phases are then locked to the values of the peaks in an adaptive manner. During attack transients we keep the stretch factor equal to one and we propose a new strategy for determining which channels are relevant for reinitializing the corresponding phase values. In contrast to previously published algorithms we use a non-uniform NSGF to obtain a low redundancy of the corresponding TF representation. We show that with just three times as many TF coefficients as signal samples, artifacts such as phasiness and transient smearing can be greatly reduced compared to the classical PV. The proposed algorithm is tested on both synthetic and real world signals and compared with state of the art algorithms in a reproducible manner.Comment: 10 pages, 6 figure