32,831 research outputs found
Structured Sparsity: Discrete and Convex approaches
Compressive sensing (CS) exploits sparsity to recover sparse or compressible
signals from dimensionality reducing, non-adaptive sensing mechanisms. Sparsity
is also used to enhance interpretability in machine learning and statistics
applications: While the ambient dimension is vast in modern data analysis
problems, the relevant information therein typically resides in a much lower
dimensional space. However, many solutions proposed nowadays do not leverage
the true underlying structure. Recent results in CS extend the simple sparsity
idea to more sophisticated {\em structured} sparsity models, which describe the
interdependency between the nonzero components of a signal, allowing to
increase the interpretability of the results and lead to better recovery
performance. In order to better understand the impact of structured sparsity,
in this chapter we analyze the connections between the discrete models and
their convex relaxations, highlighting their relative advantages. We start with
the general group sparse model and then elaborate on two important special
cases: the dispersive and the hierarchical models. For each, we present the
models in their discrete nature, discuss how to solve the ensuing discrete
problems and then describe convex relaxations. We also consider more general
structures as defined by set functions and present their convex proxies.
Further, we discuss efficient optimization solutions for structured sparsity
problems and illustrate structured sparsity in action via three applications.Comment: 30 pages, 18 figure
Reflection methods for user-friendly submodular optimization
Recently, it has become evident that submodularity naturally captures widely
occurring concepts in machine learning, signal processing and computer vision.
Consequently, there is need for efficient optimization procedures for
submodular functions, especially for minimization problems. While general
submodular minimization is challenging, we propose a new method that exploits
existing decomposability of submodular functions. In contrast to previous
approaches, our method is neither approximate, nor impractical, nor does it
need any cumbersome parameter tuning. Moreover, it is easy to implement and
parallelize. A key component of our method is a formulation of the discrete
submodular minimization problem as a continuous best approximation problem that
is solved through a sequence of reflections, and its solution can be easily
thresholded to obtain an optimal discrete solution. This method solves both the
continuous and discrete formulations of the problem, and therefore has
applications in learning, inference, and reconstruction. In our experiments, we
illustrate the benefits of our method on two image segmentation tasks.Comment: Neural Information Processing Systems (NIPS), \'Etats-Unis (2013
Deterministic global optimization using space-filling curves and multiple estimates of Lipschitz and Holder constants
In this paper, the global optimization problem with
being a hyperinterval in and satisfying the Lipschitz condition
with an unknown Lipschitz constant is considered. It is supposed that the
function can be multiextremal, non-differentiable, and given as a
`black-box'. To attack the problem, a new global optimization algorithm based
on the following two ideas is proposed and studied both theoretically and
numerically. First, the new algorithm uses numerical approximations to
space-filling curves to reduce the original Lipschitz multi-dimensional problem
to a univariate one satisfying the H\"{o}lder condition. Second, the algorithm
at each iteration applies a new geometric technique working with a number of
possible H\"{o}lder constants chosen from a set of values varying from zero to
infinity showing so that ideas introduced in a popular DIRECT method can be
used in the H\"{o}lder global optimization. Convergence conditions of the
resulting deterministic global optimization method are established. Numerical
experiments carried out on several hundreds of test functions show quite a
promising performance of the new algorithm in comparison with its direct
competitors.Comment: 26 pages, 10 figures, 4 table
- …