Worst-Case Linear Discriminant Analysis as Scalable Semidefinite Feasibility Problems

In this paper, we propose an efficient semidefinite programming (SDP) approach to worst-case linear discriminant analysis (WLDA). Compared with the traditional LDA, WLDA considers the dimensionality reduction problem from the worst-case viewpoint, which is in general more robust for classification. However, the original problem of WLDA is non-convex and difficult to optimize. In this paper, we reformulate the optimization problem of WLDA into a sequence of semidefinite feasibility problems. To efficiently solve the semidefinite feasibility problems, we design a new scalable optimization method with quasi-Newton methods and eigen-decomposition being the core components. The proposed method is orders of magnitude faster than standard interior-point based SDP solvers. Experiments on a variety of classification problems demonstrate that our approach achieves better performance than standard LDA. Our method is also much faster and more scalable than standard interior-point SDP solvers based WLDA. The computational complexity for an SDP with $m$ constraints and matrices of size $d$ by $d$ is roughly reduced from $\mathcal{O}(m^3+md^3+m^2d^2)$ to $\mathcal{O}(d^3)$ ($m>d$ in our case).Comment: 14 page

A Compact Formulation for the ℓ2,1\ell_{2,1} Mixed-Norm Minimization Problem

Parameter estimation from multiple measurement vectors (MMVs) is a fundamental problem in many signal processing applications, e.g., spectral analysis and direction-of- arrival estimation. Recently, this problem has been address using prior information in form of a jointly sparse signal structure. A prominent approach for exploiting joint sparsity considers mixed-norm minimization in which, however, the problem size grows with the number of measurements and the desired resolution, respectively. In this work we derive an equivalent, compact reformulation of the $\ell_{2,1}$ mixed-norm minimization problem which provides new insights on the relation between different existing approaches for jointly sparse signal reconstruction. The reformulation builds upon a compact parameterization, which models the row-norms of the sparse signal representation as parameters of interest, resulting in a significant reduction of the MMV problem size. Given the sparse vector of row-norms, the jointly sparse signal can be computed from the MMVs in closed form. For the special case of uniform linear sampling, we present an extension of the compact formulation for gridless parameter estimation by means of semidefinite programming. Furthermore, we derive in this case from our compact problem formulation the exact equivalence between the $\ell_{2,1}$ mixed-norm minimization and the atomic-norm minimization. Additionally, for the case of irregular sampling or a large number of samples, we present a low complexity, grid-based implementation based on the coordinate descent method