3,860 research outputs found
Regression with Linear Factored Functions
Many applications that use empirically estimated functions face a curse of
dimensionality, because the integrals over most function classes must be
approximated by sampling. This paper introduces a novel regression-algorithm
that learns linear factored functions (LFF). This class of functions has
structural properties that allow to analytically solve certain integrals and to
calculate point-wise products. Applications like belief propagation and
reinforcement learning can exploit these properties to break the curse and
speed up computation. We derive a regularized greedy optimization scheme, that
learns factored basis functions during training. The novel regression algorithm
performs competitively to Gaussian processes on benchmark tasks, and the
learned LFF functions are with 4-9 factored basis functions on average very
compact.Comment: Under review as conference paper at ECML/PKDD 201
Asymptotic Analysis of MAP Estimation via the Replica Method and Applications to Compressed Sensing
The replica method is a non-rigorous but well-known technique from
statistical physics used in the asymptotic analysis of large, random, nonlinear
problems. This paper applies the replica method, under the assumption of
replica symmetry, to study estimators that are maximum a posteriori (MAP) under
a postulated prior distribution. It is shown that with random linear
measurements and Gaussian noise, the replica-symmetric prediction of the
asymptotic behavior of the postulated MAP estimate of an n-dimensional vector
"decouples" as n scalar postulated MAP estimators. The result is based on
applying a hardening argument to the replica analysis of postulated posterior
mean estimators of Tanaka and of Guo and Verdu.
The replica-symmetric postulated MAP analysis can be readily applied to many
estimators used in compressed sensing, including basis pursuit, lasso, linear
estimation with thresholding, and zero norm-regularized estimation. In the case
of lasso estimation the scalar estimator reduces to a soft-thresholding
operator, and for zero norm-regularized estimation it reduces to a
hard-threshold. Among other benefits, the replica method provides a
computationally-tractable method for precisely predicting various performance
metrics including mean-squared error and sparsity pattern recovery probability.Comment: 22 pages; added details on the replica symmetry assumptio
Distributed Regression in Sensor Networks: Training Distributively with Alternating Projections
Wireless sensor networks (WSNs) have attracted considerable attention in
recent years and motivate a host of new challenges for distributed signal
processing. The problem of distributed or decentralized estimation has often
been considered in the context of parametric models. However, the success of
parametric methods is limited by the appropriateness of the strong statistical
assumptions made by the models. In this paper, a more flexible nonparametric
model for distributed regression is considered that is applicable in a variety
of WSN applications including field estimation. Here, starting with the
standard regularized kernel least-squares estimator, a message-passing
algorithm for distributed estimation in WSNs is derived. The algorithm can be
viewed as an instantiation of the successive orthogonal projection (SOP)
algorithm. Various practical aspects of the algorithm are discussed and several
numerical simulations validate the potential of the approach.Comment: To appear in the Proceedings of the SPIE Conference on Advanced
Signal Processing Algorithms, Architectures and Implementations XV, San
Diego, CA, July 31 - August 4, 200
Distributed Large Scale Network Utility Maximization
Recent work by Zymnis et al. proposes an efficient primal-dual interior-point
method, using a truncated Newton method, for solving the network utility
maximization (NUM) problem. This method has shown superior performance relative
to the traditional dual-decomposition approach. Other recent work by Bickson et
al. shows how to compute efficiently and distributively the Newton step, which
is the main computational bottleneck of the Newton method, utilizing the
Gaussian belief propagation algorithm.
In the current work, we combine both approaches to create an efficient
distributed algorithm for solving the NUM problem. Unlike the work of Zymnis,
which uses a centralized approach, our new algorithm is easily distributed.
Using an empirical evaluation we show that our new method outperforms previous
approaches, including the truncated Newton method and dual-decomposition
methods. As an additional contribution, this is the first work that evaluates
the performance of the Gaussian belief propagation algorithm vs. the
preconditioned conjugate gradient method, for a large scale problem.Comment: In the International Symposium on Information Theory (ISIT) 200
Regularized Newton Methods for X-ray Phase Contrast and General Imaging Problems
Like many other advanced imaging methods, x-ray phase contrast imaging and
tomography require mathematical inversion of the observed data to obtain
real-space information. While an accurate forward model describing the
generally nonlinear image formation from a given object to the observations is
often available, explicit inversion formulas are typically not known. Moreover,
the measured data might be insufficient for stable image reconstruction, in
which case it has to be complemented by suitable a priori information. In this
work, regularized Newton methods are presented as a general framework for the
solution of such ill-posed nonlinear imaging problems. For a proof of
principle, the approach is applied to x-ray phase contrast imaging in the
near-field propagation regime. Simultaneous recovery of the phase- and
amplitude from a single near-field diffraction pattern without homogeneity
constraints is demonstrated for the first time. The presented methods further
permit all-at-once phase contrast tomography, i.e. simultaneous phase retrieval
and tomographic inversion. We demonstrate the potential of this approach by
three-dimensional imaging of a colloidal crystal at 95 nm isotropic resolution.Comment: (C)2016 Optical Society of America. One print or electronic copy may
be made for personal use only. Systematic reproduction and distribution,
duplication of any material in this paper for a fee or for commercial
purposes, or modifications of the content of this paper are prohibite
Nonparametric Belief Propagation and Facial Appearance Estimation
In many applications of graphical models arising in computer vision, the hidden variables of interest are most naturally specified by continuous, non-Gaussian distributions. There exist inference algorithms for discrete approximations to these continuous distributions, but for the high-dimensional variables typically of interest, discrete inference becomes infeasible. Stochastic methods such as particle filters provide an appealing alternative. However, existing techniques fail to exploit the rich structure of the graphical models describing many vision problems. Drawing on ideas from regularized particle filters and belief propagation (BP), this paper develops a nonparametric belief propagation (NBP) algorithm applicable to general graphs. Each NBP iteration uses an efficient sampling procedure to update kernel-based approximations to the true, continuous likelihoods. The algorithm can accomodate an extremely broad class of potential functions, including nonparametric representations. Thus, NBP extends particle filtering methods to the more general vision problems that graphical models can describe. We apply the NBP algorithm to infer component interrelationships in a parts-based face model, allowing location and reconstruction of occluded features
- …