MDS matrices over small fields: A proof of the GM-MDS conjecture

An MDS matrix is a matrix whose minors all have full rank. A question arising in coding theory is what zero patterns can MDS matrices have. There is a natural combinatorial characterization (called the MDS condition) which is necessary over any field, as well as sufficient over very large fields by a probabilistic argument. Dau et al. (ISIT 2014) conjectured that the MDS condition is sufficient over small fields as well, where the construction of the matrix is algebraic instead of probabilistic. This is known as the GM-MDS conjecture. Concretely, if a $k \times n$ zero pattern satisfies the MDS condition, then they conjecture that there exists an MDS matrix with this zero pattern over any field of size $|\mathbb{F}| \ge n+k-1$. In recent years, this conjecture was proven in several special cases. In this work, we resolve the conjecture

Combinatorial nullstellensatz and its applications

In 1999, Noga Alon proved a theorem, which he called the Combinatorial Nullstellensatz, that gives an upper bound to the number of zeros of a multivariate polynomial. The theorem has since seen heavy use in combinatorics, and more specifically in graph theory. In this thesis we will give an overview of the theorem, and of how it has since been applied by various researchers. Finally, we will provide an attempt at a proof utilizing a generalized version of the Combinatorial Nullstellensatz of the GM-MDS Conjecture

On the Existence of MDS Codes Over Small Fields With Constrained Generator Matrices

We study the existence over small fields of Maximum Distance Separable (MDS) codes with generator matrices having specified supports (i.e. having specified locations of zero entries). This problem unifies and simplifies the problems posed in recent works of Yan and Sprintson (NetCod'13) on weakly secure cooperative data exchange, of Halbawi et al. (arxiv'13) on distributed Reed-Solomon codes for simple multiple access networks, and of Dau et al. (ISIT'13) on MDS codes with balanced and sparse generator matrices. We conjecture that there exist such $[n,k]_q$ MDS codes as long as $q \geq n + k - 1$, if the specified supports of the generator matrices satisfy the so-called MDS condition, which can be verified in polynomial time. We propose a combinatorial approach to tackle the conjecture, and prove that the conjecture holds for a special case when the sets of zero coordinates of rows of the generator matrix share with each other (pairwise) at most one common element. Based on our numerical result, the conjecture is also verified for all $k \leq 7$. Our approach is based on a novel generalization of the well-known Hall's marriage theorem, which allows (overlapping) multiple representatives instead of a single representative for each subset.Comment: 8 page

Reed-Solomon codes over small fields with constrained generator matrices

We give constructions of some special cases of $[n,k]$ Reed-Solomon codes over finite fields of size at least $n$ and $n+1$ whose generator matrices have constrained support. Furthermore, we consider a generalisation of the GM-MDS conjecture proposed by Lovett in 2018. We show that Lovett's conjecture is false in general and we specify when the conjecture is true.Comment: 21 page

Further Progress on the GM-MDS Conjecture for Reed-Solomon Codes

Designing good error correcting codes whose generator matrix has a support constraint, i.e., one for which only certain entries of the generator matrix are allowed to be non-zero, has found many recent applications, including in distributed coding and storage, multiple access networks, and weakly secure data exchange. The dual problem, where the parity check matrix has a support constraint, comes up in the design of locally repairable codes. The central problem here is to design codes with the largest possible minimum distance, subject to the given support constraint on the generator matrix. An upper bound on the minimum distance can be obtained through a set of singleton bounds, which can be alternatively thought of as a cut-set bound. Furthermore, it is well known that, if the field size is large enough, any random generator matrix obeying the support constraint will achieve the maximum minimum distance with high probability. Since random codes are not easy to decode, structured codes with efficient decoders, e.g., Reed-Solomon codes, are much more desirable. The GM-MDS conjecture of Dau et al states that the maximum minimum distance over all codes satisfying the generator matrix support constraint can be obtained by a Reed Solomon code. If true, this would have significant consequences. The conjecture has been proven for several special case: when the dimension of the code k is less than or equal to five, when the number of distinct support sets on the rows of the generator matrix m, say, is less than or equal to three, or when the generator matrix is sparsest and balanced. In this paper, we report on further progress on the GM-MDS conjecture. In particular, we show that the conjecture is true for all m less than equal to six. This generalizes all previous known results (except for the sparsest and balanced case, which is a very special support constraint).Comment: Submitted to ISIT 201

Gabidulin Codes with Support Constrained Generator Matrices

Gabidulin codes are the first general construction of linear codes that are maximum rank distant (MRD). They have found applications in linear network coding, for example, when the transmitter and receiver are oblivious to the inner workings and topology of the network (the so-called incoherent regime). The reason is that Gabidulin codes can be used to map information to linear subspaces, which in the absence of errors cannot be altered by linear operations, and in the presence of errors can be corrected if the subspace is perturbed by a small rank. Furthermore, in distributed coding and distributed systems, one is led to the design of error correcting codes whose generator matrix must satisfy a given support constraint. In this paper, we give necessary and sufficient conditions on the support of the generator matrix that guarantees the existence of Gabidulin codes and general MRD codes. When the rate of the code is not very high, this is achieved with the same field size necessary for Gabidulin codes with no support constraint. When these conditions are not satisfied, we characterize the largest possible rank distance under the support constraints and show that they can be achieved by subcodes of Gabidulin codes. The necessary and sufficient conditions are identical to those that appear for MDS codes which were recently proven by Yildiz et al. and Lovett in the context of settling the GM-MDS conjecture

Support Constrained Generator Matrices of Gabidulin Codes in Characteristic Zero

Gabidulin codes over fields of characteristic zero were recently constructed by Augot et al., whenever the Galois group of the underlying field extension is cyclic. In parallel, the interest in sparse generator matrices of Reed–Solomon and Gabidulin codes has increased lately, due to applications in distributed computations. In particular, a certain condition pertaining to the intersection of zero entries at different rows, was shown to be necessary and sufficient for the existence of the sparsest possible generator matrix of Gabidulin codes over finite fields. In this paper we complete the picture by showing that the same condition is also necessary and sufficient for Gabidulin codes over fields of characteristic zero.Our proof builds upon and extends tools from the finite-field case, combines them with a variant of the Schwartz–Zippel lemma over automorphisms, and provides a simple randomized construction algorithm whose probability of success can be arbitrarily close to one. In addition, potential applications for low-rank matrix recovery are discussed

Regularization of Toda lattices by Hamiltonian reduction

The Toda lattice defined by the Hamiltonian $H={1\over 2} \sum_{i=1}^n p_i^2 + \sum_{i=1}^{n-1} \nu_i e^{q_i-q_{i+1}}$ with $\nu_i\in \{ \pm 1\}$, which exhibits singular (blowing up) solutions if some of the $\nu_i=-1$, can be viewed as the reduced system following from a symmetry reduction of a subsystem of the free particle moving on the group G=SL(n,\Real ). The subsystem is $T^*G_e$, where $G_e=N_+ A N_-$ consists of the determinant one matrices with positive principal minors, and the reduction is based on the maximal nilpotent group $N_+ \times N_-$. Using the Bruhat decomposition we show that the full reduced system obtained from $T^*G$, which is perfectly regular, contains $2^{n-1}$ Toda lattices. More precisely, if $n$ is odd the reduced system contains all the possible Toda lattices having different signs for the $\nu_i$. If $n$ is even, there exist two non-isomorphic reduced systems with different constituent Toda lattices. The Toda lattices occupy non-intersecting open submanifolds in the reduced phase space, wherein they are regularized by being glued together. We find a model of the reduced phase space as a hypersurface in {\Real}^{2n-1}. If $\nu_i=1$ for all $i$, we prove for $n=2,3,4$ that the Toda phase space associated with $T^*G_e$ is a connected component of this hypersurface. The generalization of the construction for the other simple Lie groups is also presented.Comment: 42 pages, plain TeX, one reference added, to appear in J. Geom. Phy

Further Progress on the GM-MDS Conjecture for Reed-Solomon Codes

Designing good error correcting codes whose generator matrix has a support constraint, i.e., one for which only certain entries of the generator matrix are allowed to be nonzero, has found many recent applications, including in distributed coding and storage, multiple access networks, and weakly secure data exchange. The dual problem, where the parity check matrix has a support constraint, comes up in the design of locally repairable codes. The central problem here is to design codes with the largest possible minimum distance, subject to the given support constraint on the generator matrix. An upper bound on the minimum distance can be obtained through a set of singleton bounds, which can be alternatively thought of as a cut-set bound. Furthermore, it is well known that, if the field size is large enough, any random generator matrix obeying the support constraint will achieve the maximum minimum distance with high probability. Since random codes are not easy to decode, structured codes with efficient decoders, e.g., Reed-Solomon codes, are much more desirable. The GM-MDS conjecture of Dau et al states that the maximum minimum distance over all codes satisfying the generator matrix support constraint can be obtained by a Reed Solomon code. If true, this would have significant consequences. The conjecture has been proven for several special case: when the dimension of the code k is less than or equal to five, when the number of distinct support sets on the rows of the generator matrix m, say, is less than or equal to three, or when the generator matrix is sparsest and balanced. In this paper, we report on further progress on the GM-MDS conjecture. 1. In particular, we show that the conjecture is true for all m less than equal to six. This generalizes all previous known results (except for the sparsest and balanced case, which is a very special support constraint)