29 research outputs found

    FNT-based reed-solomon erasure codes

    Get PDF
    This paper presents a new construction of Maximum-Distance Separable (MDS) Reed-Solomon erasure codes based on Fermat Number Transform (FNT). Thanks to FNT, these codes support practical coding and decoding algorithms with complexity O(n log n), where n is the number of symbols of a codeword. An open-source implementation shows that the encoding speed can reach 150Mbps for codes of length up to several 10,000s of symbols. These codes can be used as the basic component of the Information Dispersal Algorithm (IDA) system used in a several P2P systems

    Novel Polynomial Basis and Its Application to Reed-Solomon Erasure Codes

    Full text link
    In this paper, we present a new basis of polynomial over finite fields of characteristic two and then apply it to the encoding/decoding of Reed-Solomon erasure codes. The proposed polynomial basis allows that hh-point polynomial evaluation can be computed in O(hlog2(h))O(h\log_2(h)) finite field operations with small leading constant. As compared with the canonical polynomial basis, the proposed basis improves the arithmetic complexity of addition, multiplication, and the determination of polynomial degree from O(hlog2(h)log2log2(h))O(h\log_2(h)\log_2\log_2(h)) to O(hlog2(h))O(h\log_2(h)). Based on this basis, we then develop the encoding and erasure decoding algorithms for the (n=2r,k)(n=2^r,k) Reed-Solomon codes. Thanks to the efficiency of transform based on the polynomial basis, the encoding can be completed in O(nlog2(k))O(n\log_2(k)) finite field operations, and the erasure decoding in O(nlog2(n))O(n\log_2(n)) finite field operations. To the best of our knowledge, this is the first approach supporting Reed-Solomon erasure codes over characteristic-2 finite fields while achieving a complexity of O(nlog2(n))O(n\log_2(n)), in both additive and multiplicative complexities. As the complexity leading factor is small, the algorithms are advantageous in practical applications

    Randomized Polar Codes for Anytime Distributed Machine Learning

    Full text link
    We present a novel distributed computing framework that is robust to slow compute nodes, and is capable of both approximate and exact computation of linear operations. The proposed mechanism integrates the concepts of randomized sketching and polar codes in the context of coded computation. We propose a sequential decoding algorithm designed to handle real valued data while maintaining low computational complexity for recovery. Additionally, we provide an anytime estimator that can generate provably accurate estimates even when the set of available node outputs is not decodable. We demonstrate the potential applications of this framework in various contexts, such as large-scale matrix multiplication and black-box optimization. We present the implementation of these methods on a serverless cloud computing system and provide numerical results to demonstrate their scalability in practice, including ImageNet scale computations

    RandSolomon: Optimally Resilient Random Number Generator with Deterministic Termination

    Get PDF
    Multi-party random number generation is a key building-block in many practical protocols. While straightforward to solve when all parties are trusted to behave correctly, the problem becomes much more difficult in the presence of faults. This paper presents RandSolomon, a partially synchronous protocol that allows a system of N processes to produce an unpredictable common random number shared by correct participants. The protocol is optimally resilient, as it allows up to f = ?(N-1)/3? of the processes to behave arbitrarily, ensures deterministic termination and, contrary to prior solutions, does not, at any point, expect faulty processes to be responsive

    Perceiving and Recovering Degraded Data on Secure Cloud”,

    Get PDF
    ABSTRACT Cloud computing is Internet-based computing, whereby shared resources, software and information, are provided to computers and devices on-demand. . Cloud Storage deals with file blocks, simplifying storage management and eliminating metadata concern. Data are continuously distributed through multiple servers in cloud. The token is computed dynamically. If data lost, then it must find out that which server gets corrupted. It can be done with byzantine fault tolerance system. The usual way of detecting corrupted data is by computing a signature for the token when it enters the cloud, and whenever it is transmitted across a cloud that is unreliable and hence capable of corrupting the data. The data is deemed to be corrupt if the newly generated signature doesn't match the original signature precomputed by the user. Third Party Auditor (TPA) is responsible for verifying the token they receive before displaying the data and its signature. The TPA verifies all the tokens distributed through multiple server. Distributed cloud server stores replicas of file blocks; it can heal corrupted blocks by retransmitting the corrupt replica block. The RS algorithm is used to guard against corruption due to data loss/node loss by supporting the retransmission. The main aim of this paper is making the file system tolerate node failure without suffering data loss
    corecore