14 research outputs found
A Novel Completely Local Repairable Code Algorithm Based on Erasure Code
Hadoop Distributed File System (HDFS) is widely used in massive data storage. Because of the disadvantage of the multi-copy strategy, the hardware expansion of HDFS cannot keep up with the continuous volume of big data. Now, the traditional data replication strategy has been gradually replaced by Erasure Code due to its smaller redundancy rate and storage overhead. However, compared with replicas, Erasure Code needs to read a certain amount of data blocks during the process of data recovery, resulting in a large amount of overhead for I/O and network. Based on the Reed-Solomon (RS) algorithm, we propose a novel Completely Local Repairable Code (CLRC) algorithm. By grouping RS coded blocks and generating local check blocks, CLRC algorithm can optimize the locality of the RS algorithm, which can reduce the cost of data recovery. Evaluations show that the CLRC algorithm can reduce the bandwidth and I/O consumption during the process of data recovery when a single block is damaged. What\u27s more, the cost of decoding time is only 59% of the RS algorithm
Explicit Low-Bandwidth Evaluation Schemes for Weighted Sums of Reed-Solomon-Coded Symbols
Motivated by applications in distributed storage, distributed computing, and
homomorphic secret sharing, we study communication-efficient schemes for
computing linear combinations of coded symbols. Specifically, we design
low-bandwidth schemes that evaluate the weighted sum of coded symbols in
a codeword , when we are given access to of the
remaining components in .
Formally, suppose that is a field extension of of
degree . Let be a codeword in a Reed-Solomon code of dimension
and our task is to compute the weighted sum of coded symbols. In this
paper, for some , we provide an explicit scheme that performs this task by
downloading sub-symbols in from available nodes,
whenever . In many cases, our scheme
outperforms previous schemes in the literature.
Furthermore, we provide a characterization of evaluation schemes for general
linear codes. Then in the special case of Reed-Solomon codes, we use this
characterization to derive a lower bound for the evaluation bandwidth.Comment: 23 pages, 2 figure