Matrix inequalities involving the Khatri-Rao product

summary:We extend three inequalities involving the Hadamard product in three ways. First, the results are extended to any partitioned blocks Hermitian matrices. Second, the Hadamard product is replaced by the Khatri-Rao product. Third, the necessary and sufficient conditions under which equalities occur are presented. Thereby, we generalize two inequalities involving the Khatri–Rao product

Fast Exact Leverage Score Sampling from Khatri-Rao Products with Applications to Tensor Decomposition

We present a data structure to randomly sample rows from the Khatri-Rao product of several matrices according to the exact distribution of its leverage scores. Our proposed sampler draws each row in time logarithmic in the height of the Khatri-Rao product and quadratic in its column count, with persistent space overhead at most the size of the input matrices. As a result, it tractably draws samples even when the matrices forming the Khatri-Rao product have tens of millions of rows each. When used to sketch the linear least squares problems arising in CANDECOMP / PARAFAC tensor decomposition, our method achieves lower asymptotic complexity per solve than recent state-of-the-art methods. Experiments on billion-scale sparse tensors validate our claims, with our algorithm achieving higher accuracy than competing methods as the decomposition rank grows.Comment: To appear at the 37th Conference on Neural Information Processing Systems (Neurips'23). 28 pages, 10 figures, 6 table

DFacTo: Distributed Factorization of Tensors

We present a technique for significantly speeding up Alternating Least Squares (ALS) and Gradient Descent (GD), two widely used algorithms for tensor factorization. By exploiting properties of the Khatri-Rao product, we show how to efficiently address a computationally challenging sub-step of both algorithms. Our algorithm, DFacTo, only requires two sparse matrix-vector products and is easy to parallelize. DFacTo is not only scalable but also on average 4 to 10 times faster than competing algorithms on a variety of datasets. For instance, DFacTo only takes 480 seconds on 4 machines to perform one iteration of the ALS algorithm and 1,143 seconds to perform one iteration of the GD algorithm on a 6.5 million x 2.5 million x 1.5 million dimensional tensor with 1.2 billion non-zero entries.Comment: Under review for NIPS 201

New Holder - Type Inequalities for the Tracy-Singh and Khatri-Rao Products of Positive Matrices

Recently, the authors establised a number of inequalities involving Khatri-Rao product of two positives matrices. Here, in this paper, the result are establised in three ways. First, we find new Holder-type inequalities for Tracy-Singh and Khatri-Rao products products of positive semi-devinite matrices. Second, the result are extended to provide estimates of sums of the Khatri-Rao and Tracy-Singh products of any finite number of positive semi-definite matrices. Three, the result lead to inequalities involving the Hadamard and Kronecker, as a special case