11,978 research outputs found
Communication-Computation Efficient Gradient Coding
This paper develops coding techniques to reduce the running time of
distributed learning tasks. It characterizes the fundamental tradeoff to
compute gradients (and more generally vector summations) in terms of three
parameters: computation load, straggler tolerance and communication cost. It
further gives an explicit coding scheme that achieves the optimal tradeoff
based on recursive polynomial constructions, coding both across data subsets
and vector components. As a result, the proposed scheme allows to minimize the
running time for gradient computations. Implementations are made on Amazon EC2
clusters using Python with mpi4py package. Results show that the proposed
scheme maintains the same generalization error while reducing the running time
by compared to uncoded schemes and compared to prior coded
schemes focusing only on stragglers (Tandon et al., ICML 2017)
Polar Codes for Distributed Hierarchical Source Coding
We show that polar codes can be used to achieve the rate-distortion functions
in the problem of hierarchical source coding also known as the successive
refinement problem. We also analyze the distributed version of this problem,
constructing a polar coding scheme that achieves the rate distortion functions
for successive refinement with side information.Comment: 14 page
Universal Source Polarization and an Application to a Multi-User Problem
We propose a scheme that universally achieves the smallest possible
compression rate for a class of sources with side information, and develop an
application of this result for a joint source channel coding problem over a
broadcast channel.Comment: to be presented at Allerton 201
Error correction based on partial information
We consider the decoding of linear and array codes from errors when we are
only allowed to download a part of the codeword. More specifically, suppose
that we have encoded data symbols using an code with code length
and dimension During storage, some of the codeword coordinates might
be corrupted by errors. We aim to recover the original data by reading the
corrupted codeword with a limit on the transmitting bandwidth, namely, we can
only download an proportion of the corrupted codeword. For a given
our objective is to design a code and a decoding scheme such that we
can recover the original data from the largest possible number of errors. A
naive scheme is to read coordinates of the codeword. This method
used in conjunction with MDS codes guarantees recovery from any errors. In this paper we show that we can instead read an
proportion from each of the codeword's coordinates. For a
well-designed MDS code, this method can guarantee recovery from errors, which is times more than the naive
method, and is also the maximum number of errors that an code can
correct by downloading only an proportion of the codeword. We present
two families of such optimal constructions and decoding schemes. One is a
Reed-Solomon code with evaluation points in a subfield and the other is based
on Folded Reed-Solomon codes. We further show that both code constructions
attain asymptotically optimal list decoding radius when downloading only a part
of the corrupted codeword. We also construct an ensemble of random codes that
with high probability approaches the upper bound on the number of correctable
errors when the decoder downloads an proportion of the corrupted
codeword.Comment: Extended version of the conference paper in ISIT 201
- …