477 research outputs found
Computational Methods for Sparse Solution of Linear Inverse Problems
The goal of the sparse approximation problem is to approximate a target signal using a linear combination of a few elementary signals drawn from a fixed collection. This paper surveys the major practical algorithms for sparse approximation. Specific attention is paid to computational issues, to the circumstances in which individual methods tend to perform well, and to the theoretical guarantees available. Many fundamental questions in electrical engineering, statistics, and applied mathematics can be posed as sparse approximation problems, making these algorithms versatile and relevant to a plethora of applications
Generalized Dantzig Selector: Application to the k-support norm
We propose a Generalized Dantzig Selector (GDS) for linear models, in which
any norm encoding the parameter structure can be leveraged for estimation. We
investigate both computational and statistical aspects of the GDS. Based on
conjugate proximal operator, a flexible inexact ADMM framework is designed for
solving GDS, and non-asymptotic high-probability bounds are established on the
estimation error, which rely on Gaussian width of unit norm ball and suitable
set encompassing estimation error. Further, we consider a non-trivial example
of the GDS using -support norm. We derive an efficient method to compute the
proximal operator for -support norm since existing methods are inapplicable
in this setting. For statistical analysis, we provide upper bounds for the
Gaussian widths needed in the GDS analysis, yielding the first statistical
recovery guarantee for estimation with the -support norm. The experimental
results confirm our theoretical analysis.Comment: Updates to boun
High-Dimensional Dependency Structure Learning for Physical Processes
In this paper, we consider the use of structure learning methods for
probabilistic graphical models to identify statistical dependencies in
high-dimensional physical processes. Such processes are often synthetically
characterized using PDEs (partial differential equations) and are observed in a
variety of natural phenomena, including geoscience data capturing atmospheric
and hydrological phenomena. Classical structure learning approaches such as the
PC algorithm and variants are challenging to apply due to their high
computational and sample requirements. Modern approaches, often based on sparse
regression and variants, do come with finite sample guarantees, but are usually
highly sensitive to the choice of hyper-parameters, e.g., parameter
for sparsity inducing constraint or regularization. In this paper, we present
ACLIME-ADMM, an efficient two-step algorithm for adaptive structure learning,
which estimates an edge specific parameter in the first step,
and uses these parameters to learn the structure in the second step. Both steps
of our algorithm use (inexact) ADMM to solve suitable linear programs, and all
iterations can be done in closed form in an efficient block parallel manner. We
compare ACLIME-ADMM with baselines on both synthetic data simulated by partial
differential equations (PDEs) that model advection-diffusion processes, and
real data (50 years) of daily global geopotential heights to study information
flow in the atmosphere. ACLIME-ADMM is shown to be efficient, stable, and
competitive, usually better than the baselines especially on difficult
problems. On real data, ACLIME-ADMM recovers the underlying structure of global
atmospheric circulation, including switches in wind directions at the equator
and tropics entirely from the data.Comment: 21 pages, 8 figures, International Conference on Data Mining 201
- β¦