Numerically stable coded matrix computations via circulant and rotation matrix embeddings

Several recent works have used coding-theoretic ideas for mitigating the effect of stragglers in distributed matrix computations (matrix-vector and matrix-matrix multiplication) over the reals. In particular, a polynomial code based approach distributes matrix-matrix multiplication among n worker nodes by means of polynomial evaluations. This allows for an ``optimal\u27\u27 recovery threshold whereby the intended result can be decoded as long as at least (n−s) worker nodes complete their tasks; s is the number of stragglers that the scheme can handle. However, a major issue with these approaches is the high condition number of the corresponding Vandermonde-structured recovery matrices. This presents serious numerical precision issues when decoding the desired result. It is well known that the condition number of real Vandermonde matrices grows exponentially in n. In contrast, the condition numbers of Vandermonde matrices with parameters on the unit circle are much better behaved. In this work we leverage the properties of circulant permutation matrices and rotation matrices to obtain coded computation schemes with significantly lower worst case condition numbers; these matrices have eigenvalues that lie on the unit circle. Our scheme is such that the associated recovery matrices have a condition number corresponding to Vandermonde matrices with parameters given by the eigenvalues of the corresponding circulant permutation and rotation matrices. We demonstrate an upper bound on the worst case condition number of these matrices which grows as ≈O(ns+6). In essence, we leverage the well-behaved conditioning of complex Vandermonde matrices with parameters on the unit circle, while still working with computation over the reals. Experimental results demonstrate that our proposed method has condition numbers that are several orders of magnitude better than prior work

Is polynomial interpolation in the monomial basis unstable?

In this paper, we show that the monomial basis is generally as good as a well-conditioned polynomial basis for interpolation, provided that the condition number of the Vandermonde matrix is smaller than the reciprocal of machine epsilon. We also show that the monomial basis is more advantageous than other polynomial bases in a number of applications.Comment: 30 pages, 12 figure

A Study of Vandermonde-like Matrix Systems With Emphasis on Preconditioning and Krylov Matrix Connection.

The study focuses primarily on Vandermonde-like matrix systems. The idea is to express Vandermonde and Vandermonde-like matrix systems as the problems related to Krylov Matrices. The connection provides a different angle to view the Vandermonde-like systems. Krylov subspace methods are strongly related to polynomial spaces, hence a nice connection can be established using LU factorization as proposed by Bjorck and Pereyra and QR factorization by Reichel. Further an algorithm to generate a preconditioner is incorporated in GR algorithm given by Reichel . This generates a preconditioner for Vandermonde-like matrices consisting of polynomials which obey a three term recurrence relation. This general preconditioner works effectively for Vandermonde matrices as well. The preconditioner is then tested on various distinct nodes. Based on results obtained, it is established that the condition number of Vandermonde -like matrices can be lowered significantly by application of the preconditioner, for some cases

RLWE and PLWE over cyclotomic fields are not equivalent

We prove that the Ring Learning With Errors (RLWE) and the Polynomial Learning With Errors (PLWE) problems over the cyclotomic field $\mathbb{Q}(\zeta_n)$ are not equivalent. Precisely, we show that reducing one problem to the other increases the noise by a factor that is more than polynomial in $n$. We do so by providing a lower bound, holding for infinitely many positive integers $n$, for the condition number of the Vandermonde matrix of the $n$th cyclotomic polynomial

Super-resolution, Extremal Functions and the Condition Number of Vandermonde Matrices

Super-resolution is a fundamental task in imaging, where the goal is to extract fine-grained structure from coarse-grained measurements. Here we are interested in a popular mathematical abstraction of this problem that has been widely studied in the statistics, signal processing and machine learning communities. We exactly resolve the threshold at which noisy super-resolution is possible. In particular, we establish a sharp phase transition for the relationship between the cutoff frequency ($m$) and the separation ($\Delta$). If $m > 1/\Delta + 1$, our estimator converges to the true values at an inverse polynomial rate in terms of the magnitude of the noise. And when $m < (1-\epsilon) /\Delta$ no estimator can distinguish between a particular pair of $\Delta$-separated signals even if the magnitude of the noise is exponentially small. Our results involve making novel connections between {\em extremal functions} and the spectral properties of Vandermonde matrices. We establish a sharp phase transition for their condition number which in turn allows us to give the first noise tolerance bounds for the matrix pencil method. Moreover we show that our methods can be interpreted as giving preconditioners for Vandermonde matrices, and we use this observation to design faster algorithms for super-resolution. We believe that these ideas may have other applications in designing faster algorithms for other basic tasks in signal processing.Comment: 19 page