Worst-Case Linear Discriminant Analysis as Scalable Semidefinite Feasibility Problems

In this paper, we propose an efficient semidefinite programming (SDP) approach to worst-case linear discriminant analysis (WLDA). Compared with the traditional LDA, WLDA considers the dimensionality reduction problem from the worst-case viewpoint, which is in general more robust for classification. However, the original problem of WLDA is non-convex and difficult to optimize. In this paper, we reformulate the optimization problem of WLDA into a sequence of semidefinite feasibility problems. To efficiently solve the semidefinite feasibility problems, we design a new scalable optimization method with quasi-Newton methods and eigen-decomposition being the core components. The proposed method is orders of magnitude faster than standard interior-point based SDP solvers. Experiments on a variety of classification problems demonstrate that our approach achieves better performance than standard LDA. Our method is also much faster and more scalable than standard interior-point SDP solvers based WLDA. The computational complexity for an SDP with $m$ constraints and matrices of size $d$ by $d$ is roughly reduced from $\mathcal{O}(m^3+md^3+m^2d^2)$ to $\mathcal{O}(d^3)$ ($m>d$ in our case).Comment: 14 page

COSMO: A conic operator splitting method for convex conic problems

This paper describes the Conic Operator Splitting Method (COSMO) solver, an operator splitting algorithm for convex optimisation problems with quadratic objective function and conic constraints. At each step the algorithm alternates between solving a quasi-definite linear system with a constant coefficient matrix and a projection onto convex sets. The low per-iteration computational cost makes the method particularly efficient for large problems, e.g. semidefinite programs that arise in portfolio optimisation, graph theory, and robust control. Moreover, the solver uses chordal decomposition techniques and a new clique merging algorithm to effectively exploit sparsity in large, structured semidefinite programs. A number of benchmarks against other state-of-the-art solvers for a variety of problems show the effectiveness of our approach. Our Julia implementation is open-source, designed to be extended and customised by the user, and is integrated into the Julia optimisation ecosystem.Comment: 45 pages, 11 figure

An analytic center cutting plane method to determine complete positivity of a matrix

We propose an analytic center cutting plane method to determine whether a matrix is completely positive and return a cut that separates it from the completely positive cone if not. This was stated as an open (computational) problem by Berman et al. [Berman A, Dur M, Shaked-Monderer N (2015) Open problems in the theory of completely positive and copositive matrices. Electronic 1. Linear Algebra 29(1):46-58]. Our method optimizes over the intersection of a ball and the copositive cone, where membership is determined by solving a mixed-integer linear program suggested by Xia et al. [Xia W, Vera JC, Zuluaga LF (2020) Globally solving nonconvex quadratic programs via linear integer programming techniques. INFORMS J. Comput 32(1):40-561 Thus, our algorithm can, more generally, be used to solve any copositive optimization problem, provided one knows the radius of a ball containing an optimal solution. Numerical experiments show that the number of oracle calls (matrix copositivity checks) for our implementation scales well with the matrix size, growing roughly like O(d(2)) for d x d matrices. The method is implemented in Julia and available at https://github.com/rileybadenbroek/CopositiveAnalyticCenter.jl. Summary of Contribution: Completely positive matrices play an important role in operations research. They allow many NP-hard problems to be formulated as optimization problems over a proper cone, which enables them to benefit from the duality theory of convex programming. We propose an analytic center cutting plane method to determine whether a matrix is completely positive by solving an optimization problem over the copositive cone. In fact, we can use our method to solve any copositive optimization problem, provided we know the radius of a ball containing an optimal solution. We emphasize numerical performance and stability in developing this method. A software implementation in Julia is provided