1,055 research outputs found
Worst-Case Linear Discriminant Analysis as Scalable Semidefinite Feasibility Problems
In this paper, we propose an efficient semidefinite programming (SDP)
approach to worst-case linear discriminant analysis (WLDA). Compared with the
traditional LDA, WLDA considers the dimensionality reduction problem from the
worst-case viewpoint, which is in general more robust for classification.
However, the original problem of WLDA is non-convex and difficult to optimize.
In this paper, we reformulate the optimization problem of WLDA into a sequence
of semidefinite feasibility problems. To efficiently solve the semidefinite
feasibility problems, we design a new scalable optimization method with
quasi-Newton methods and eigen-decomposition being the core components. The
proposed method is orders of magnitude faster than standard interior-point
based SDP solvers.
Experiments on a variety of classification problems demonstrate that our
approach achieves better performance than standard LDA. Our method is also much
faster and more scalable than standard interior-point SDP solvers based WLDA.
The computational complexity for an SDP with constraints and matrices of
size by is roughly reduced from to
( in our case).Comment: 14 page
Sticky Brownian Rounding and its Applications to Constraint Satisfaction Problems
Semidefinite programming is a powerful tool in the design and analysis of
approximation algorithms for combinatorial optimization problems. In
particular, the random hyperplane rounding method of Goemans and Williamson has
been extensively studied for more than two decades, resulting in various
extensions to the original technique and beautiful algorithms for a wide range
of applications. Despite the fact that this approach yields tight approximation
guarantees for some problems, e.g., Max-Cut, for many others, e.g., Max-SAT and
Max-DiCut, the tight approximation ratio is still unknown. One of the main
reasons for this is the fact that very few techniques for rounding semidefinite
relaxations are known.
In this work, we present a new general and simple method for rounding
semi-definite programs, based on Brownian motion. Our approach is inspired by
recent results in algorithmic discrepancy theory. We develop and present tools
for analyzing our new rounding algorithms, utilizing mathematical machinery
from the theory of Brownian motion, complex analysis, and partial differential
equations. Focusing on constraint satisfaction problems, we apply our method to
several classical problems, including Max-Cut, Max-2SAT, and MaxDiCut, and
derive new algorithms that are competitive with the best known results. To
illustrate the versatility and general applicability of our approach, we give
new approximation algorithms for the Max-Cut problem with side constraints that
crucially utilizes measure concentration results for the Sticky Brownian
Motion, a feature missing from hyperplane rounding and its generalization
Probabilistic Clustering Using Maximal Matrix Norm Couplings
In this paper, we present a local information theoretic approach to
explicitly learn probabilistic clustering of a discrete random variable. Our
formulation yields a convex maximization problem for which it is NP-hard to
find the global optimum. In order to algorithmically solve this optimization
problem, we propose two relaxations that are solved via gradient ascent and
alternating maximization. Experiments on the MSR Sentence Completion Challenge,
MovieLens 100K, and Reuters21578 datasets demonstrate that our approach is
competitive with existing techniques and worthy of further investigation.Comment: Presented at 56th Annual Allerton Conference on Communication,
Control, and Computing, 201
- …