64 research outputs found
The Geometry of Uniqueness, Sparsity and Clustering in Penalized Estimation
We provide a necessary and sufficient condition for the uniqueness of
penalized least-squares estimators whose penalty term is given by a norm with a
polytope unit ball, covering a wide range of methods including SLOPE and LASSO,
as well as the related method of basis pursuit. We consider a strong type of
uniqueness that is relevant for statistical problems. The uniqueness condition
is geometric and involves how the row span of the design matrix intersects the
faces of the dual norm unit ball, which for SLOPE is given by the sign
permutahedron. Further considerations based this condition also allow to derive
results on sparsity and clustering features. In particular, we define the
notion of a SLOPE model to describe both sparsity and clustering properties of
this method and also provide a geometric characterization of accessible SLOPE
models.Comment: new title, minor change
Effective Condition Number Bounds for Convex Regularization
We derive bounds relating Renegar's condition number to quantities that
govern the statistical performance of convex regularization in settings that
include the -analysis setting. Using results from conic integral
geometry, we show that the bounds can be made to depend only on a random
projection, or restriction, of the analysis operator to a lower dimensional
space, and can still be effective if these operators are ill-conditioned. As an
application, we get new bounds for the undersampling phase transition of
composite convex regularizers. Key tools in the analysis are Slepian's
inequality and the kinematic formula from integral geometry.Comment: 17 pages, 4 figures . arXiv admin note: text overlap with
arXiv:1408.301
Projection Methods in Sparse and Low Rank Feasibility
In this thesis, we give an analysis of fixed point algorithms involving projections onto closed, not necessarily convex, subsets of finite dimensional vector
spaces.
These methods are used in applications such as imaging science, signal processing, and inverse problems. The tools used in the analysis
place this work at the intersection of optimization and variational analysis. Based on the underlying optimization problems, this work is devided into two main parts. The first one is the compressed sensing problem. Because the problem
is NP-hard, we relax it to a feasibility problem with two sets, namely,
the set of vectors with at most s nonzero entries and, for a linear mapping M
the affine subspace B of vectors satisfying Mx=p for p given.
This problem will be referred to as the sparse-affine-feasibility problem. For the Douglas-Rachford algorithm, we give the proof of linear convergence to a fixed point in the case of a feasibility problem of two affine subspaces.
It allows us to conclude a result of local linear convergence of the Douglas-Rachford algorithm in the sparse affine feasibility problem.
Proceeding, we name sufficient conditions for the alternating projections algorithm to converge to the intersection of an affine subspace with lower level sets
of point symmetric, lower semicontinuous, subadditive functions.
This implies convergence of alternating projections to a solution of the sparse affine feasibility problem.
Together with a result of local linear convergence of the alternating projections algorithm, this allows us to deduce linear convergence after finitely many steps
for any initial point of a sequence of points generated by the alternating projections algorithm. The second part of this dissertation deals with the minimization of the rank of matrices satisfying a set of linear equations.
This problem will be called rank-constrained-affine-feasibility problem.
The motivation for the analysis of the rank minimization problem comes from the physical application of phase retrieval and a reformulation of the same as a
rank minimization problem. We show that, locally, the method of alternating projections must converge at linear rate to a solution of the rank
constrained affine feasibility problem
Extremal Area of Polygons sliding along Curves
In this paper we study the area function of polygons, where the vertices are
sliding along curves. We give geometric criteria for the critical points and
determine also the Hesse matrix at those points. This is the starting point for
a Morse-theoretic approach, which includes the relation with the topology of
the configuration spaces. Moreover the condition for extremal area gives rise
to a new type of billiard: the inner area billiard.Comment: 20 pages, 12 figure
-Penalization in Functional Linear Regression with Subgaussian Design
We study functional regression with random subgaussian design and real-valued
response. The focus is on the problems in which the regression function can be
well approximated by a functional linear model with the slope function being
"sparse" in the sense that it can be represented as a sum of a small number of
well separated "spikes". This can be viewed as an extension of now classical
sparse estimation problems to the case of infinite dictionaries. We study an
estimator of the regression function based on penalized empirical risk
minimization with quadratic loss and the complexity penalty defined in terms of
-norm (a continuous version of LASSO). The main goal is to introduce
several important parameters characterizing sparsity in this class of problems
and to prove sharp oracle inequalities showing how the -error of the
continuous LASSO estimator depends on the underlying sparsity of the problem
- …