29 research outputs found
FNT-based reed-solomon erasure codes
This paper presents a new construction of Maximum-Distance Separable (MDS) Reed-Solomon erasure codes based on Fermat Number Transform (FNT). Thanks to FNT, these codes support practical coding and decoding algorithms with complexity O(n log n), where n is the number of symbols of a codeword. An open-source implementation shows that the encoding speed can reach 150Mbps for codes of length up to several 10,000s of symbols. These codes can be used as the basic component of the Information Dispersal Algorithm (IDA) system used in a several P2P systems
Novel Polynomial Basis and Its Application to Reed-Solomon Erasure Codes
In this paper, we present a new basis of polynomial over finite fields of
characteristic two and then apply it to the encoding/decoding of Reed-Solomon
erasure codes. The proposed polynomial basis allows that -point polynomial
evaluation can be computed in finite field operations with
small leading constant. As compared with the canonical polynomial basis, the
proposed basis improves the arithmetic complexity of addition, multiplication,
and the determination of polynomial degree from
to . Based on this basis, we then develop the encoding and
erasure decoding algorithms for the Reed-Solomon codes. Thanks to
the efficiency of transform based on the polynomial basis, the encoding can be
completed in finite field operations, and the erasure decoding
in finite field operations. To the best of our knowledge, this
is the first approach supporting Reed-Solomon erasure codes over
characteristic-2 finite fields while achieving a complexity of ,
in both additive and multiplicative complexities. As the complexity leading
factor is small, the algorithms are advantageous in practical applications
Randomized Polar Codes for Anytime Distributed Machine Learning
We present a novel distributed computing framework that is robust to slow
compute nodes, and is capable of both approximate and exact computation of
linear operations. The proposed mechanism integrates the concepts of randomized
sketching and polar codes in the context of coded computation. We propose a
sequential decoding algorithm designed to handle real valued data while
maintaining low computational complexity for recovery. Additionally, we provide
an anytime estimator that can generate provably accurate estimates even when
the set of available node outputs is not decodable. We demonstrate the
potential applications of this framework in various contexts, such as
large-scale matrix multiplication and black-box optimization. We present the
implementation of these methods on a serverless cloud computing system and
provide numerical results to demonstrate their scalability in practice,
including ImageNet scale computations
RandSolomon: Optimally Resilient Random Number Generator with Deterministic Termination
Multi-party random number generation is a key building-block in many practical protocols. While straightforward to solve when all parties are trusted to behave correctly, the problem becomes much more difficult in the presence of faults. This paper presents RandSolomon, a partially synchronous protocol that allows a system of N processes to produce an unpredictable common random number shared by correct participants. The protocol is optimally resilient, as it allows up to f = ?(N-1)/3? of the processes to behave arbitrarily, ensures deterministic termination and, contrary to prior solutions, does not, at any point, expect faulty processes to be responsive
Perceiving and Recovering Degraded Data on Secure Cloud”,
ABSTRACT Cloud computing is Internet-based computing, whereby shared resources, software and information, are provided to computers and devices on-demand. . Cloud Storage deals with file blocks, simplifying storage management and eliminating metadata concern. Data are continuously distributed through multiple servers in cloud. The token is computed dynamically. If data lost, then it must find out that which server gets corrupted. It can be done with byzantine fault tolerance system. The usual way of detecting corrupted data is by computing a signature for the token when it enters the cloud, and whenever it is transmitted across a cloud that is unreliable and hence capable of corrupting the data. The data is deemed to be corrupt if the newly generated signature doesn't match the original signature precomputed by the user. Third Party Auditor (TPA) is responsible for verifying the token they receive before displaying the data and its signature. The TPA verifies all the tokens distributed through multiple server. Distributed cloud server stores replicas of file blocks; it can heal corrupted blocks by retransmitting the corrupt replica block. The RS algorithm is used to guard against corruption due to data loss/node loss by supporting the retransmission. The main aim of this paper is making the file system tolerate node failure without suffering data loss