Extended Integrated Interleaved Codes over any Field with Applications to Locally Recoverable Codes

Integrated Interleaved (II) and Extended Integrated Interleaved (EII) codes are a versatile alternative for Locally Recoverable (LRC) codes, since they require fields of relatively small size. II and EII codes are generally defined over Reed-Solomon type of codes. A new comprehensive definition of EII codes is presented, allowing for EII codes over any field, and in particular, over the binary field $GF(2)$. The traditional definition of II and EII codes is shown to be a special case of the new definition. Improvements over previous constructions of LRC codes, in particular, for binary codes, are given, as well as cases meeting an upper bound on the minimum distance. Properties of the codes are presented as well, in particular, an iterative decoding algorithm on rows and columns generalizing the iterative decoding algorithm of product codes. Two applications are also discussed: one is finding a systematic encoding of EII codes such that the parity symbols have a balanced distribution on rows, and the other is the problem of ordering the symbols of an EII code such that the maximum length of a correctable burst is achieved.Comment: 25 page

Integrated Interleaved Codes as Locally Recoverable Codes: Properties and Performance

Considerable interest has been paid in recent literature to codes combining local and global properties for erasure correction. Applications are in cloud type of implementations, in which fast recovery of a failed storage device is important, but additional protection is required in order to avoid data loss, and in RAID type of architectures, in which total device failures coexist with silent failures at the page or sector level in each device. Existing solutions to these problems require in general relatively large finite fields. The techniques of Integrated Interleaved Codes (which are closely related to Generalized Concatenated Codes) are proposed to reduce significantly the size of the finite field, and it is shown that when the parameters of these codes are judiciously chosen, their performance may be competitive with the one of codes optimizing the minimum distance.Comment: 24 pages, 5 figures and 3 table

Extended Product and Integrated Interleaved Codes

A new class of codes, Extended Product (EPC) Codes, consisting of a product code with a number of extra parities added, is presented and applications for erasure decoding are discussed. An upper bound on the minimum distance of EPC codes is given, as well as constructions meeting the bound for some relevant cases. A special case of EPC codes, Extended Integrated Interleaved (EII) codes, which naturally unify Integrated Interleaved (II) codes and product codes, is defined and studied in detail. It is shown that EII codes often improve the minimum distance of II codes with the same rate, and they enhance the decoding algorithm by allowing decoding on columns as well as on rows. It is also shown that EII codes allow for encoding II codes with an uniform distribution of the parity symbols.Comment: 24 pages. arXiv admin note: text overlap with arXiv:1610.0427

Multi-Erasure Locally Recoverable Codes Over Small Fields

Erasure codes play an important role in storage systems to prevent data loss. In this work, we study a class of erasure codes called Multi-Erasure Locally Recoverable Codes (ME-LRCs) for storage arrays. Compared to previous related works, we focus on the construction of ME-LRCs over small fields. We first develop upper and lower bounds on the minimum distance of ME-LRCs. Our main contribution is to propose a general construction of ME-LRCs based on generalized tensor product codes, and study their erasure-correcting properties. A decoding algorithm tailored for erasure recovery is given, and correctable erasure patterns are identified. We then prove that our construction yields optimal ME-LRCs with a wide range of code parameters, and present some explicit ME-LRCs over small fields. Finally, we show that generalized integrated interleaving (GII) codes can be treated as a subclass of generalized tensor product codes, thus defining the exact relation between these codes.Comment: This is an extended version of arXiv:1701.06110. To appear in Allerton 201

On Locally Recoverable (LRC) Codes

We present simple constructions of optimal erasure-correcting LRC codes by exhibiting their parity-check matrices. When the number of local parities in a parity group plus the number of global parities is smaller than the size of the parity group, the constructed codes are optimal with a field of size at least the length of the code. We can reduce the size of the field to at least the size of the parity groups when the number of global parities equals the number of local parities in a parity group plus one.Comment: 11 pages, 2 figure

Multiple-Layer Integrated Interleaved Codes: A Class of Hierarchical Locally Recoverable Codes

The traditional definition of Integrated Interleaved (II) codes generally assumes that the component nested codes are either Reed-Solomon (RS) or shortened Reed-Solomon codes. By taking general classes of codes, we present a recursive construction of Extended Integrated Interleaved (EII) codes into multiple layers, a problem that brought attention in literature for II codes. The multiple layer approach allows for a hierarchical scheme where each layer of the code provides for a different locality. In particular, we present the erasure-correcting capability of the new codes and we show that they are ideally suited as Locally Recoverable (LRC) codes due to their hierarchical locality and the small finite field required by the construction. Properties of the multiple layer EII codes, like their minimum distance and dimension, as well as their erasure decoding algorithms, parity-check matrices and performance analysis, are provided and illustrated with examples. Finally, we will observe that the parity-check matrices of high layer EII codes have low density.Comment: 21 pages, 1 tabl

Hierarchical Hybrid Error Correction for Time-Sensitive Devices at the Edge

Computational storage, known as a solution to significantly reduce the latency by moving data-processing down to the data storage, has received wide attention because of its potential to accelerate data-driven devices at the edge. To meet the insatiable appetite for complicated functionalities tailored for intelligent devices such as autonomous vehicles, properties including heterogeneity, scalability, and flexibility are becoming increasingly important. Based on our prior work on hierarchical erasure coding that enables scalability and flexibility in cloud storage, we develop an efficient decoding algorithm that corrects a mixture of errors and erasures simultaneously. We first extract the basic component code, the so-called extended Cauchy (EC) codes, of the proposed coding solution. We prove that the class of EC codes is strictly larger than that of relevant codes with known explicit decoding algorithms. Motivated by this finding, we then develop an efficient decoding method for the general class of EC codes, based on which we propose the local and global decoding algorithms for the hierarchical codes. Our proposed hybrid error correction not only enables the usage of hierarchical codes in computational storage at the edge, but also applies to any Cauchy-like codes and allows potentially wider applications of the EC codes.Comment: 29 pages (single column), 0 figures, to be submitted to IEEE Transactions on Communication

Hierarchical Coding for Cloud Storage: Topology-Adaptivity, Scalability, and Flexibility

In order to accommodate the ever-growing data from various, possibly independent, sources and the dynamic nature of data usage rates in practical applications, modern cloud data storage systems are required to be scalable, flexible, and heterogeneous. The recent rise of the blockchain technology is also moving various information systems towards decentralization to achieve high privacy at low costs. While codes with hierarchical locality have been intensively studied in the context of centralized cloud storage due to their effectiveness in reducing the average reading time, those for decentralized storage networks (DSNs) have not yet been discussed. In this paper, we propose a joint coding scheme where each node receives extra protection through the cooperation with nodes in its neighborhood in a heterogeneous DSN with any given topology. This work extends and subsumes our prior work on coding for centralized cloud storage. In particular, our proposed construction not only preserves desirable properties such as scalability and flexibility, which are critical in dynamic networks, but also adapts to arbitrary topologies, a property that is essential in DSNs but has been overlooked in existing works.Comment: 25 pages (single column), 19 figures, submitted to the IEEE Transactions on Information Theory (TIT

Generalized Concatenated Types of Codes for Erasure Correction

Generalized Concatenated (GC), also known as Integrated Interleaved (II) Codes, are studied from an erasure correction point of view making them useful for Redundant Arrays of Independent Disks (RAID) types of architectures combining global and local properties. The fundamental erasure-correcting properties of the codes are proven and efficient encoding and decoding algorithms are provided. Although less powerful than the recently developed PMDS codes, this implementation has the advantage of allowing generalization to any range of parameters while the size of the field is much smaller than the one required for PMDS codes

Universal and Dynamic Locally Repairable Codes with Maximal Recoverability via Sum-Rank Codes

Locally repairable codes (LRCs) are considered with equal or unequal localities, local distances and local field sizes. An explicit two-layer architecture with a sum-rank outer code is obtained, having disjoint local groups and achieving maximal recoverability (MR) for all families of local linear codes (MDS or not) simultaneously, up to a specified maximum locality $r$. Furthermore, the local linear codes (thus the localities, local distances and local fields) can be efficiently and dynamically modified without global recoding or changes in architecture or outer code, while preserving the MR property, easily adapting to new configurations in storage or new hot and cold data. In addition, local groups and file components can be added, removed or updated without global recoding. The construction requires global fields of size roughly $g^r$, for $g$ local groups and maximum or specified locality $r$. For equal localities, these global fields are smaller than those of previous MR-LRCs when $r \leq h$ (global parities). For unequal localities, they provide an exponential field size reduction on all previous best known MR-LRCs. For bounded localities and a large number of local groups, the global erasure-correction complexity of the given construction is comparable to that of Tamo-Barg codes or Reed-Solomon codes with local replication, while local repair is as efficient as for the Cartesian product of the local codes. Reed-Solomon codes with local replication and Cartesian products are recovered from the given construction when $r=1$ and $h = 0$, respectively. The given construction can also be adapted to provide hierarchical MR-LRCs for all types of hierarchies and parameters. Finally, subextension subcodes and sum-rank alternant codes are introduced to obtain further exponential field size reductions, at the expense of lower information rates