2,883 research outputs found
A Nonconvex Projection Method for Robust PCA
Robust principal component analysis (RPCA) is a well-studied problem with the
goal of decomposing a matrix into the sum of low-rank and sparse components. In
this paper, we propose a nonconvex feasibility reformulation of RPCA problem
and apply an alternating projection method to solve it. To the best of our
knowledge, we are the first to propose a method that solves RPCA problem
without considering any objective function, convex relaxation, or surrogate
convex constraints. We demonstrate through extensive numerical experiments on a
variety of applications, including shadow removal, background estimation, face
detection, and galaxy evolution, that our approach matches and often
significantly outperforms current state-of-the-art in various ways.Comment: In the proceedings of Thirty-Third AAAI Conference on Artificial
Intelligence (AAAI-19
Deep Metric Learning with Chance Constraints
Deep metric learning (DML) aims to minimize empirical expected loss of the
pairwise intra-/inter- class proximity violations in the embedding image. We
relate DML to feasibility problem of finite chance constraints. We show that
minimizer of proxy-based DML satisfies certain chance constraints, and that the
worst case generalization performance of the proxy-based methods can be
characterized by the radius of the smallest ball around a class proxy to cover
the entire domain of the corresponding class samples, suggesting multiple
proxies per class helps performance. To provide a scalable algorithm as well as
exploiting more proxies, we consider the chance constraints implied by the
minimizers of proxy-based DML instances and reformulate DML as finding a
feasible point in intersection of such constraints, resulting in a problem to
be approximately solved by iterative projections. Simply put, we repeatedly
train a regularized proxy-based loss and re-initialize the proxies with the
embeddings of the deliberately selected new samples. We apply our method with
the well-accepted losses and evaluate on four popular benchmark datasets for
image retrieval. Outperforming state-of-the-art, our method consistently
improves the performance of the applied losses. Code is available at:
https://github.com/yetigurbuz/ccp-dmlComment: Under review at IEEE Transactions on Neural Networks and Learning
System
Predicting diabetes-related hospitalizations based on electronic health records
OBJECTIVE: To derive a predictive model to identify patients likely to be hospitalized during the following year due to complications attributed to Type II diabetes. METHODS: A variety of supervised machine learning classification methods were tested and a new method that discovers hidden patient clusters in the positive class (hospitalized) was developed while, at the same time, sparse linear support vector machine classifiers were derived to separate positive samples from the negative ones (non-hospitalized). The convergence of the new method was established and theoretical guarantees were proved on how the classifiers it produces generalize to a test set not seen during training. RESULTS: The methods were tested on a large set of patients from the Boston Medical Center - the largest safety net hospital in New England. It is found that our new joint clustering/classification method achieves an accuracy of 89% (measured in terms of area under the ROC Curve) and yields informative clusters which can help interpret the classification results, thus increasing the trust of physicians to the algorithmic output and providing some guidance towards preventive measures. While it is possible to increase accuracy to 92% with other methods, this comes with increased computational cost and lack of interpretability. The analysis shows that even a modest probability of preventive actions being effective (more than 19%) suffices to generate significant hospital care savings. CONCLUSIONS: Predictive models are proposed that can help avert hospitalizations, improve health outcomes and drastically reduce hospital expenditures. The scope for savings is significant as it has been estimated that in the USA alone, about $5.8 billion are spent each year on diabetes-related hospitalizations that could be prevented.Accepted manuscrip
Non-convex Optimization for Machine Learning
A vast majority of machine learning algorithms train their models and perform
inference by solving optimization problems. In order to capture the learning
and prediction problems accurately, structural constraints such as sparsity or
low rank are frequently imposed or else the objective itself is designed to be
a non-convex function. This is especially true of algorithms that operate in
high-dimensional spaces or that train non-linear models such as tensor models
and deep networks.
The freedom to express the learning problem as a non-convex optimization
problem gives immense modeling power to the algorithm designer, but often such
problems are NP-hard to solve. A popular workaround to this has been to relax
non-convex problems to convex ones and use traditional methods to solve the
(convex) relaxed optimization problems. However this approach may be lossy and
nevertheless presents significant challenges for large scale optimization.
On the other hand, direct approaches to non-convex optimization have met with
resounding success in several domains and remain the methods of choice for the
practitioner, as they frequently outperform relaxation-based techniques -
popular heuristics include projected gradient descent and alternating
minimization. However, these are often poorly understood in terms of their
convergence and other properties.
This monograph presents a selection of recent advances that bridge a
long-standing gap in our understanding of these heuristics. The monograph will
lead the reader through several widely used non-convex optimization techniques,
as well as applications thereof. The goal of this monograph is to both,
introduce the rich literature in this area, as well as equip the reader with
the tools and techniques needed to analyze these simple procedures for
non-convex problems.Comment: The official publication is available from now publishers via
http://dx.doi.org/10.1561/220000005
- …