43 research outputs found
Scalable sparse covariance estimation via self-concordance
We consider the class of convex minimization problems, composed of a
self-concordant function, such as the metric, a convex data fidelity
term and, a regularizing -- possibly non-smooth -- function
. This type of problems have recently attracted a great deal of
interest, mainly due to their omnipresence in top-notch applications. Under
this \emph{locally} Lipschitz continuous gradient setting, we analyze the
convergence behavior of proximal Newton schemes with the added twist of a
probable presence of inexact evaluations. We prove attractive convergence rate
guarantees and enhance state-of-the-art optimization schemes to accommodate
such developments. Experimental results on sparse covariance estimation show
the merits of our algorithm, both in terms of recovery efficiency and
complexity.Comment: 7 pages, 1 figure, Accepted at AAAI-1
Generalized self-concordant Hessian-barrier algorithms
Many problems in statistical learning, imaging, and computer vision involve the optimization of a non-convex objective function with singularities at the boundary of the feasible set. For such challenging instances, we develop a new interior-point technique building on the Hessian-barrier algorithm recently introduced in Bomze, Mertikopoulos, Schachinger and Staudigl, [SIAM J. Opt. 2019 29(3), pp. 2100-2127], where the Riemannian metric is induced by a generalized selfconcordant function. This class of functions is sufficiently general to include most of the commonly used barrier functions in the literature of interior point methods. We prove global convergence to an approximate stationary point of the method, and in cases where the feasible set admits an easily computable self-concordant barrier, we verify worst-case optimal iteration complexity of the method. Applications in non-convex statistical estimation and Lp-minimization are discussed to given the efficiency of the method
Convexity in source separation: Models, geometry, and algorithms
Source separation or demixing is the process of extracting multiple
components entangled within a signal. Contemporary signal processing presents a
host of difficult source separation problems, from interference cancellation to
background subtraction, blind deconvolution, and even dictionary learning.
Despite the recent progress in each of these applications, advances in
high-throughput sensor technology place demixing algorithms under pressure to
accommodate extremely high-dimensional signals, separate an ever larger number
of sources, and cope with more sophisticated signal and mixing models. These
difficulties are exacerbated by the need for real-time action in automated
decision-making systems.
Recent advances in convex optimization provide a simple framework for
efficiently solving numerous difficult demixing problems. This article provides
an overview of the emerging field, explains the theory that governs the
underlying procedures, and surveys algorithms that solve them efficiently. We
aim to equip practitioners with a toolkit for constructing their own demixing
algorithms that work, as well as concrete intuition for why they work
Sparsity Based Poisson Denoising with Dictionary Learning
The problem of Poisson denoising appears in various imaging applications,
such as low-light photography, medical imaging and microscopy. In cases of high
SNR, several transformations exist so as to convert the Poisson noise into an
additive i.i.d. Gaussian noise, for which many effective algorithms are
available. However, in a low SNR regime, these transformations are
significantly less accurate, and a strategy that relies directly on the true
noise statistics is required. A recent work by Salmon et al. took this route,
proposing a patch-based exponential image representation model based on GMM
(Gaussian mixture model), leading to state-of-the-art results. In this paper,
we propose to harness sparse-representation modeling to the image patches,
adopting the same exponential idea. Our scheme uses a greedy pursuit with
boot-strapping based stopping condition and dictionary learning within the
denoising process. The reconstruction performance of the proposed scheme is
competitive with leading methods in high SNR, and achieving state-of-the-art
results in cases of low SNR.Comment: 13 pages, 9 figure
Generalized Self-concordant Hessian-barrier algorithms
Many problems in statistical learning, imaging, and computer vision involve
the optimization of a non-convex objective function with singularities at the
boundary of the feasible set. For such challenging instances, we develop a new
interior-point technique building on the Hessian-barrier algorithm recently
introduced in Bomze, Mertikopoulos, Schachinger and Staudigl, [SIAM J. Opt.
2019 29(3), pp. 2100-2127], where the Riemannian metric is induced by a
generalized self-concordant function. This class of functions is sufficiently
general to include most of the commonly used barrier functions in the
literature of interior point methods. We prove global convergence to an
approximate stationary point of the method, and in cases where the feasible set
admits an easily computable self-concordant barrier, we verify worst-case
optimal iteration complexity of the method. Applications in non-convex
statistical estimation and -minimization are discussed to given the
efficiency of the method