Convexity Conditions of Kantorovich Function and Related Semi-infinite Linear Matrix Inequalities

The Kantorovich function $(x^TAx)(x^T A^{-1} x)$, where $A$ is a positive definite matrix, is not convex in general. From matrix/convex analysis point of view, it is interesting to address the question: When is this function convex? In this paper, we investigate the convexity of this function by the condition number of its matrix. In 2-dimensional space, we prove that the Kantorovich function is convex if and only if the condition number of its matrix is bounded above by $3+2\sqrt{2},$ and thus the convexity of the function with two variables can be completely characterized by the condition number. The upper bound `$3+2\sqrt{2}$' is turned out to be a necessary condition for the convexity of Kantorovich functions in any finite-dimensional spaces. We also point out that when the condition number of the matrix (which can be any dimensional) is less than or equal to $\sqrt{5+2\sqrt{6}},$ the Kantorovich function is convex. Furthermore, we prove that this general sufficient convexity condition can be remarkably improved in 3-dimensional space. Our analysis shows that the convexity of the function is closely related to some modern optimization topics such as the semi-infinite linear matrix inequality or 'robust positive semi-definiteness' of symmetric matrices. In fact, our main result for 3-dimensional cases has been proved by finding an explicit solution range to some semi-infinite linear matrix inequalities.Comment: 24 page

The Legendre–Fenchel Conjugate of the Product of Two Positive Definite Quadratic Forms

It is well-known that the Legendre-Fenchel conjugate of a positive-denite quadratic form can be explicitly expressed as another positive-denite quadratic form, and that the conjugate of the sum of several positive-denite quadratic forms can be expressed via inf-convolution. However, the Legendre-Fenchel conjugate of the product of two positive-denite quadratic forms is not clear at present. Jean-Baptiste Hiriart-Urruty posted it as an open question in the eld of nonlinear analysis and optimization [`Question 11 ' in SIAM Review 49 (2007), 255-273]. From convex analysis point of view, it is interesting and important to address such a question. The purpose of this paper is to answer this question and to provide a formula for the conjugate of the product of two positive-denite quadratic forms. We prove that the computation of the conjugate can be implemented via nding a root to certain univariate polynomial equation. Furthermore, we show that the conjugate can be explicitly expressed as a single function in some situations. Our analysis shows that the relationship between the matrices of quadratic forms plays a vital role in determining whether or not the conjugate can be expressed explicitly, and our analysis also sheds some light on the computational complexity of the Legendre-Fenchel conjugate for the product of quadratic forms

Optimal k-thresholding algorithms for sparse optimization problems

The simulations indicate that the existing hard thresholding technique independent of the residual function may cause a dramatic increase or numerical oscillation of the residual. This inherit drawback of the hard thresholding renders the traditional thresholding algorithms unstable and thus generally inefficient for solving practical sparse optimization problems. How to overcome this weakness and develop a truly efficient thresholding method is a fundamental question in this field. The aim of this paper is to address this question by proposing a new thresholding technique based on the notion of optimal $k$-thresholding. The central idea for this new development is to connect the $k$-thresholding directly to the residual reduction during the course of algorithms. This leads to a natural design principle for the efficient thresholding methods. Under the restricted isometry property (RIP), we prove that the optimal thresholding based algorithms are globally convergent to the solution of sparse optimization problems. The numerical experiments demonstrate that when solving sparse optimization problems, the traditional hard thresholding methods have been significantly transcended by the proposed algorithms which can even outperform the classic $\ell_1$-minimization method in many situations