28,706 research outputs found
Direction of arrival estimation using robust complex Lasso
The Lasso (Least Absolute Shrinkage and Selection Operator) has been a
popular technique for simultaneous linear regression estimation and variable
selection. In this paper, we propose a new novel approach for robust Lasso that
follows the spirit of M-estimation. We define -Lasso estimates of regression
and scale as solutions to generalized zero subgradient equations. Another
unique feature of this paper is that we consider complex-valued measurements
and regression parameters, which requires careful mathematical characterization
of the problem. An explicit and efficient algorithm for computing the -Lasso
solution is proposed that has comparable computational complexity as
state-of-the-art algorithm for computing the Lasso solution. Usefulness of the
-Lasso method is illustrated for direction-of-arrival (DoA) estimation with
sensor arrays in a single snapshot case.Comment: Paper has appeared in the Proceedings of the 10th European Conference
on Antennas and Propagation (EuCAP'2016), Davos, Switzerland, April 10-15,
201
Multichannel sparse recovery of complex-valued signals using Huber's criterion
In this paper, we generalize Huber's criterion to multichannel sparse
recovery problem of complex-valued measurements where the objective is to find
good recovery of jointly sparse unknown signal vectors from the given multiple
measurement vectors which are different linear combinations of the same known
elementary vectors. This requires careful characterization of robust
complex-valued loss functions as well as Huber's criterion function for the
multivariate sparse regression problem. We devise a greedy algorithm based on
simultaneous normalized iterative hard thresholding (SNIHT) algorithm. Unlike
the conventional SNIHT method, our algorithm, referred to as HUB-SNIHT, is
robust under heavy-tailed non-Gaussian noise conditions, yet has a negligible
performance loss compared to SNIHT under Gaussian noise. Usefulness of the
method is illustrated in source localization application with sensor arrays.Comment: To appear in CoSeRa'15 (Pisa, Italy, June 16-19, 2015). arXiv admin
note: text overlap with arXiv:1502.0244
Non-convex Optimization for Machine Learning
A vast majority of machine learning algorithms train their models and perform
inference by solving optimization problems. In order to capture the learning
and prediction problems accurately, structural constraints such as sparsity or
low rank are frequently imposed or else the objective itself is designed to be
a non-convex function. This is especially true of algorithms that operate in
high-dimensional spaces or that train non-linear models such as tensor models
and deep networks.
The freedom to express the learning problem as a non-convex optimization
problem gives immense modeling power to the algorithm designer, but often such
problems are NP-hard to solve. A popular workaround to this has been to relax
non-convex problems to convex ones and use traditional methods to solve the
(convex) relaxed optimization problems. However this approach may be lossy and
nevertheless presents significant challenges for large scale optimization.
On the other hand, direct approaches to non-convex optimization have met with
resounding success in several domains and remain the methods of choice for the
practitioner, as they frequently outperform relaxation-based techniques -
popular heuristics include projected gradient descent and alternating
minimization. However, these are often poorly understood in terms of their
convergence and other properties.
This monograph presents a selection of recent advances that bridge a
long-standing gap in our understanding of these heuristics. The monograph will
lead the reader through several widely used non-convex optimization techniques,
as well as applications thereof. The goal of this monograph is to both,
introduce the rich literature in this area, as well as equip the reader with
the tools and techniques needed to analyze these simple procedures for
non-convex problems.Comment: The official publication is available from now publishers via
http://dx.doi.org/10.1561/220000005
Quality-based Multimodal Classification Using Tree-Structured Sparsity
Recent studies have demonstrated advantages of information fusion based on
sparsity models for multimodal classification. Among several sparsity models,
tree-structured sparsity provides a flexible framework for extraction of
cross-correlated information from different sources and for enforcing group
sparsity at multiple granularities. However, the existing algorithm only solves
an approximated version of the cost functional and the resulting solution is
not necessarily sparse at group levels. This paper reformulates the
tree-structured sparse model for multimodal classification task. An accelerated
proximal algorithm is proposed to solve the optimization problem, which is an
efficient tool for feature-level fusion among either homogeneous or
heterogeneous sources of information. In addition, a (fuzzy-set-theoretic)
possibilistic scheme is proposed to weight the available modalities, based on
their respective reliability, in a joint optimization problem for finding the
sparsity codes. This approach provides a general framework for quality-based
fusion that offers added robustness to several sparsity-based multimodal
classification algorithms. To demonstrate their efficacy, the proposed methods
are evaluated on three different applications - multiview face recognition,
multimodal face recognition, and target classification.Comment: To Appear in 2014 IEEE Conference on Computer Vision and Pattern
Recognition (CVPR 2014
Adaptive Relaxed ADMM: Convergence Theory and Practical Implementation
Many modern computer vision and machine learning applications rely on solving
difficult optimization problems that involve non-differentiable objective
functions and constraints. The alternating direction method of multipliers
(ADMM) is a widely used approach to solve such problems. Relaxed ADMM is a
generalization of ADMM that often achieves better performance, but its
efficiency depends strongly on algorithm parameters that must be chosen by an
expert user. We propose an adaptive method that automatically tunes the key
algorithm parameters to achieve optimal performance without user oversight.
Inspired by recent work on adaptivity, the proposed adaptive relaxed ADMM
(ARADMM) is derived by assuming a Barzilai-Borwein style linear gradient. A
detailed convergence analysis of ARADMM is provided, and numerical results on
several applications demonstrate fast practical convergence.Comment: CVPR 201
Nonparametric Simultaneous Sparse Recovery: an Application to Source Localization
We consider multichannel sparse recovery problem where the objective is to
find good recovery of jointly sparse unknown signal vectors from the given
multiple measurement vectors which are different linear combinations of the
same known elementary vectors. Many popular greedy or convex algorithms perform
poorly under non-Gaussian heavy-tailed noise conditions or in the face of
outliers. In this paper, we propose the usage of mixed norms on
data fidelity (residual matrix) term and the conventional -norm
constraint on the signal matrix to promote row-sparsity. We devise a greedy
pursuit algorithm based on simultaneous normalized iterative hard thresholding
(SNIHT) algorithm. Simulation studies highlight the effectiveness of the
proposed approaches to cope with different noise environments (i.i.d., row
i.i.d, etc) and outliers. Usefulness of the methods are illustrated in source
localization application with sensor arrays.Comment: Paper appears in Proc. European Signal Processing Conference
(EUSIPCO'15), Nice, France, Aug 31 -- Sep 4, 201
- …