Generalized Hadamard Product and the Derivatives of Spectral Functions

In this work we propose a generalization of the Hadamard product between two matrices to a tensor-valued, multi-linear product between k matrices for any $k \ge 1$. A multi-linear dual operator to the generalized Hadamard product is presented. It is a natural generalization of the Diag x operator, that maps a vector $x \in \R^n$ into the diagonal matrix with x on its main diagonal. Defining an action of the $n \times n$ orthogonal matrices on the space of k-dimensional tensors, we investigate its interactions with the generalized Hadamard product and its dual. The research is motivated, as illustrated throughout the paper, by the apparent suitability of this language to describe the higher-order derivatives of spectral functions and the tools needed to compute them. For more on the later we refer the reader to [14] and [15], where we use the language and properties developed here to study the higher-order derivatives of spectral functions.Comment: 24 page

Extention of Apolarity and Grace Theorem

MSC 2010: 30C10The classical notion of apolarity is defined for two algebraic polynomials of equal degree. The main property of two apolar polynomials p and q is the classical Grace theorem: Every circular domain containing all zeros of p contains at least one zero of q and vice versa. In this paper, the definition of apolarity is extended to polynomials of different degree and an extension of the Grace theorem is proved. This leads to simplification of the conditions of several well-known results about apolarity

The higher-order derivatives of spectral functions

AbstractWe are interested in higher-order derivatives of functions of the eigenvalues of real symmetric matrices with respect to the matrix argument. We describe a formula for the k-th derivative of such functions in two general cases.The first case concerns the derivatives of the composition of an arbitrary (not necessarily symmetric) k-times differentiable function with the eigenvalues of symmetric matrices at a symmetric matrix with distinct eigenvalues.The second case describes the derivatives of the composition of a k-times differentiable separable symmetric function with the eigenvalues of symmetric matrices at an arbitrary symmetric matrix. We show that the formula significantly simplifies when the separable symmetric function is k-times continuously differentiable.As an application of the developed techniques, we re-derive the formula for the Hessian of a general spectral function at an arbitrary symmetric matrix. The new tools lead to a shorter, cleaner derivation than the original one.To make the exposition as self contained as possible, we have included the necessary background results and definitions. Proofs of the intermediate technical results are collected in the appendices

Locally symmetric submanifolds lift to spectral manifolds

In this work we prove that every locally symmetric smooth submanifold gives rise to a naturally defined smooth submanifold of the space of symmetric matrices, called spectral manifold, consisting of all matrices whose ordered vector of eigenvalues belongs to the locally symmetric manifold. We also present an explicit formula for the dimension of the spectral manifold in terms of the dimension and the intrinsic properties of the locally symmetric manifold

Variational Spectral Analysis

We present results on smooth and nonsmooth variational properties of {it symmetric} functions of the eigenvalues of a real symmetric matrix argument, as well as {it absolutely symmetric} functions of the singular values of a real rectangular matrix. Such results underpin the theory of optimization problems involving such functions. We answer the question of when a symmetric function of the eigenvalues allows a quadratic expansion around a matrix, and then the stronger question of when it is twice differentiable. We develop simple formulae for the most important nonsmooth subdifferentials of functions depending on the singular values of a real rectangular matrix argument and give several examples. The analysis of the above two classes of functions may be generalized in various larger abstract frameworks. In particular, we investigate how functions depending on the eigenvalues or the singular values of a matrix argument may be viewed as the composition of symmetric functions with the roots of {it hyperbolic polynomials}. We extend the relationship between hyperbolic polynomials and {it self-concordant barriers} (an extremely important class of functions in contemporary interior point methods for convex optimization) by exhibiting a new class of self-concordant barriers obtainable from hyperbolic polynomials

Spectral (Isotropic) Manifolds and Their Dimension

International audienceA set of symmetric matrices whose ordered vector of eigenvalues belongs to a fixed set in Rn is called spectral or isotropic. In this paper, we establish that every locally symmetric submanifold M of Rn gives rise to a spectral manifold, for k ∈ {2, 3, . . . , ∞, ω}. An explicit formula for the dimension of the spectral manifold in terms of the dimension and the intrinsic properties of M is derived