1,732 research outputs found
Inference for Generalized Linear Models via Alternating Directions and Bethe Free Energy Minimization
Generalized Linear Models (GLMs), where a random vector is
observed through a noisy, possibly nonlinear, function of a linear transform
arise in a range of applications in nonlinear
filtering and regression. Approximate Message Passing (AMP) methods, based on
loopy belief propagation, are a promising class of approaches for approximate
inference in these models. AMP methods are computationally simple, general, and
admit precise analyses with testable conditions for optimality for large i.i.d.
transforms . However, the algorithms can easily diverge for general
. This paper presents a convergent approach to the generalized AMP
(GAMP) algorithm based on direct minimization of a large-system limit
approximation of the Bethe Free Energy (LSL-BFE). The proposed method uses a
double-loop procedure, where the outer loop successively linearizes the LSL-BFE
and the inner loop minimizes the linearized LSL-BFE using the Alternating
Direction Method of Multipliers (ADMM). The proposed method, called ADMM-GAMP,
is similar in structure to the original GAMP method, but with an additional
least-squares minimization. It is shown that for strictly convex, smooth
penalties, ADMM-GAMP is guaranteed to converge to a local minima of the
LSL-BFE, thus providing a convergent alternative to GAMP that is stable under
arbitrary transforms. Simulations are also presented that demonstrate the
robustness of the method for non-convex penalties as well
Regularized Optimal Transport and the Rot Mover's Distance
This paper presents a unified framework for smooth convex regularization of
discrete optimal transport problems. In this context, the regularized optimal
transport turns out to be equivalent to a matrix nearness problem with respect
to Bregman divergences. Our framework thus naturally generalizes a previously
proposed regularization based on the Boltzmann-Shannon entropy related to the
Kullback-Leibler divergence, and solved with the Sinkhorn-Knopp algorithm. We
call the regularized optimal transport distance the rot mover's distance in
reference to the classical earth mover's distance. We develop two generic
schemes that we respectively call the alternate scaling algorithm and the
non-negative alternate scaling algorithm, to compute efficiently the
regularized optimal plans depending on whether the domain of the regularizer
lies within the non-negative orthant or not. These schemes are based on
Dykstra's algorithm with alternate Bregman projections, and further exploit the
Newton-Raphson method when applied to separable divergences. We enhance the
separable case with a sparse extension to deal with high data dimensions. We
also instantiate our proposed framework and discuss the inherent specificities
for well-known regularizers and statistical divergences in the machine learning
and information geometry communities. Finally, we demonstrate the merits of our
methods with experiments using synthetic data to illustrate the effect of
different regularizers and penalties on the solutions, as well as real-world
data for a pattern recognition application to audio scene classification
- …