2,545 research outputs found
Gradient Coding from Cyclic MDS Codes and Expander Graphs
Gradient coding is a technique for straggler mitigation in distributed
learning. In this paper we design novel gradient codes using tools from
classical coding theory, namely, cyclic MDS codes, which compare favorably with
existing solutions, both in the applicable range of parameters and in the
complexity of the involved algorithms. Second, we introduce an approximate
variant of the gradient coding problem, in which we settle for approximate
gradient computation instead of the exact one. This approach enables graceful
degradation, i.e., the error of the approximate gradient is a
decreasing function of the number of stragglers. Our main result is that
normalized adjacency matrices of expander graphs yield excellent approximate
gradient codes, which enable significantly less computation compared to exact
gradient coding, and guarantee faster convergence than trivial solutions under
standard assumptions. We experimentally test our approach on Amazon EC2, and
show that the generalization error of approximate gradient coding is very close
to the full gradient while requiring significantly less computation from the
workers
The Minimum Distance of Graph Codes
We study codes constructed from graphs where the code symbols are associated with the edges and the symbols connected to a given vertex are restricted to be codewords in a component code. In particular we treat such codes from bipartite expander graphs coming from Euclidean planes and other geometries. We give results on the minimum distances of the codes
Simple Constructions of Unique Neighbor Expanders from Error-correcting Codes
In this note, we give very simple constructions of unique neighbor expander
graphs starting from spectral or combinatorial expander graphs of mild
expansion. These constructions and their analysis are simple variants of the
constructions of LDPC error-correcting codes from expanders, given by
Sipser-Spielman\cite{SS96} (and Tanner\cite{Tanner81}), and their analysis. We
also show how to obtain expanders with many unique neighbors using similar
ideas.
There were many exciting results on this topic recently, starting with
Asherov-Dinur\cite{AD23} and Hsieh-McKenzie-Mohanty-Paredes\cite{HMMP23}, who
gave a similar construction of unique neighbor expander graphs, but using more
sophisticated ingredients (such as almost-Ramanujan graphs) and a more involved
analysis. Subsequent beautiful works of Cohen-Roth-TaShma\cite{CRT23} and
Golowich\cite{Golowich23} gave even stronger objects (lossless expanders), but
also using sophisticated ingredients.
The main contribution of this work is that we get much more elementary
constructions of unique neighbor expanders and with a simpler analysis
Strong blocking sets and minimal codes from expander graphs
A strong blocking set in a finite projective space is a set of points that
intersects each hyperplane in a spanning set. We provide a new graph theoretic
construction of such sets: combining constant-degree expanders with
asymptotically good codes, we explicitly construct strong blocking sets in the
-dimensional projective space over that have size . Since strong blocking sets have recently been shown to be equivalent to
minimal linear codes, our construction gives the first explicit construction of
-linear minimal codes of length and dimension , for every
prime power , for which . This solves one of the main open
problems on minimal codes.Comment: 20 page
Efficient and Robust Compressed Sensing Using Optimized Expander Graphs
Expander graphs have been recently proposed to construct efficient compressed sensing algorithms. In particular, it has been shown that any n-dimensional vector that is k-sparse can be fully recovered using O(klog n) measurements and only O(klog n) simple recovery iterations. In this paper, we improve upon this result by considering expander graphs with expansion coefficient beyond 3/4 and show that, with the same number of measurements, only O(k) recovery iterations are required, which is a significant improvement when n is large. In fact, full recovery can be accomplished by at most 2k very simple iterations. The number of iterations can be reduced arbitrarily close to k, and the recovery algorithm can be implemented very efficiently using a simple priority queue with total recovery time O(nlog(n/k))). We also show that by tolerating a small penal- ty on the number of measurements, and not on the number of recovery iterations, one can use the efficient construction of a family of expander graphs to come up with explicit measurement matrices for this method. We compare our result with other recently developed expander-graph-based methods and argue that it compares favorably both in terms of the number of required measurements and in terms of the time complexity and the simplicity of recovery. Finally, we will show how our analysis extends to give a robust algorithm that finds the position and sign of the k significant elements of an almost k-sparse signal and then, using very simple optimization techniques, finds a k-sparse signal which is close to the best k-term approximation of the original signal
- …