Integrated Interleaved Codes as Locally Recoverable Codes: Properties and Performance

Considerable interest has been paid in recent literature to codes combining local and global properties for erasure correction. Applications are in cloud type of implementations, in which fast recovery of a failed storage device is important, but additional protection is required in order to avoid data loss, and in RAID type of architectures, in which total device failures coexist with silent failures at the page or sector level in each device. Existing solutions to these problems require in general relatively large finite fields. The techniques of Integrated Interleaved Codes (which are closely related to Generalized Concatenated Codes) are proposed to reduce significantly the size of the finite field, and it is shown that when the parameters of these codes are judiciously chosen, their performance may be competitive with the one of codes optimizing the minimum distance.Comment: 24 pages, 5 figures and 3 table

On Locally Recoverable (LRC) Codes

We present simple constructions of optimal erasure-correcting LRC codes by exhibiting their parity-check matrices. When the number of local parities in a parity group plus the number of global parities is smaller than the size of the parity group, the constructed codes are optimal with a field of size at least the length of the code. We can reduce the size of the field to at least the size of the parity groups when the number of global parities equals the number of local parities in a parity group plus one.Comment: 11 pages, 2 figure

Construction of PMDS and SD Codes extending RAID 5

A construction of Partial Maximum Distance Separable (PMDS) and Sector-Disk (SD) codes extending RAID 5 with two extra parities is given, solving an open problem. Previous constructions relied on computer searches, while our constructions provide a theoretical solution to the problem.Comment: 7 page

Generalized Concatenated Types of Codes for Erasure Correction

Generalized Concatenated (GC), also known as Integrated Interleaved (II) Codes, are studied from an erasure correction point of view making them useful for Redundant Arrays of Independent Disks (RAID) types of architectures combining global and local properties. The fundamental erasure-correcting properties of the codes are proven and efficient encoding and decoding algorithms are provided. Although less powerful than the recently developed PMDS codes, this implementation has the advantage of allowing generalization to any range of parameters while the size of the field is much smaller than the one required for PMDS codes

Modeling Impact of Human Errors on the Data Unavailability and Data Loss of Storage Systems

Data storage systems and their availability play a crucial role in contemporary datacenters. Despite using mechanisms such as automatic fail-over in datacenters, the role of human agents and consequently their destructive errors is inevitable. Due to very large number of disk drives used in exascale datacenters and their high failure rates, the disk subsystem in storage systems has become a major source of Data Unavailability (DU) and Data Loss (DL) initiated by human errors. In this paper, we investigate the effect of Incorrect Disk Replacement Service (IDRS) on the availability and reliability of data storage systems. To this end, we analyze the consequences of IDRS in a disk array, and conduct Monte Carlo simulations to evaluate DU and DL during mission time. The proposed modeling framework can cope with a) different storage array configurations and b) Data Object Survivability (DOS), representing the effect of system level redundancies such as remote backups and mirrors. In the proposed framework, the model parameters are obtained from industrial and scientific reports alongside field data which have been extracted from a datacenter operating with 70 storage racks. The results show that ignoring the impact of IDRS leads to unavailability underestimation by up to three orders of magnitude. Moreover, our study suggests that by considering the effect of human errors, the conventional beliefs about the dependability of different Redundant Array of Independent Disks (RAID) mechanisms should be revised. The results show that RAID1 can result in lower availability compared to RAID5 in the presence of human errors. The results also show that employing automatic fail-over policy (using hot spare disks) can reduce the drastic impacts of human errors by two orders of magnitude.Comment: 17 page

Extended Product and Integrated Interleaved Codes

A new class of codes, Extended Product (EPC) Codes, consisting of a product code with a number of extra parities added, is presented and applications for erasure decoding are discussed. An upper bound on the minimum distance of EPC codes is given, as well as constructions meeting the bound for some relevant cases. A special case of EPC codes, Extended Integrated Interleaved (EII) codes, which naturally unify Integrated Interleaved (II) codes and product codes, is defined and studied in detail. It is shown that EII codes often improve the minimum distance of II codes with the same rate, and they enhance the decoding algorithm by allowing decoding on columns as well as on rows. It is also shown that EII codes allow for encoding II codes with an uniform distribution of the parity symbols.Comment: 24 pages. arXiv admin note: text overlap with arXiv:1610.0427

Extended Integrated Interleaved Codes over any Field with Applications to Locally Recoverable Codes

Integrated Interleaved (II) and Extended Integrated Interleaved (EII) codes are a versatile alternative for Locally Recoverable (LRC) codes, since they require fields of relatively small size. II and EII codes are generally defined over Reed-Solomon type of codes. A new comprehensive definition of EII codes is presented, allowing for EII codes over any field, and in particular, over the binary field $GF(2)$. The traditional definition of II and EII codes is shown to be a special case of the new definition. Improvements over previous constructions of LRC codes, in particular, for binary codes, are given, as well as cases meeting an upper bound on the minimum distance. Properties of the codes are presented as well, in particular, an iterative decoding algorithm on rows and columns generalizing the iterative decoding algorithm of product codes. Two applications are also discussed: one is finding a systematic encoding of EII codes such that the parity symbols have a balanced distribution on rows, and the other is the problem of ordering the symbols of an EII code such that the maximum length of a correctable burst is achieved.Comment: 25 page

Parity Declustering for Fault-Tolerant Storage Systems via tt-designs

Parity declustering allows faster reconstruction of a disk array when some disk fails. Moreover, it guarantees uniform reconstruction workload on all surviving disks. It has been shown that parity declustering for one-failure tolerant array codes can be obtained via Balanced Incomplete Block Designs. We extend this technique for array codes that can tolerate an arbitrary number of disk failures via $t$-designs.Comment: 13 page

A Complete Classification of Partial-MDS (Maximally Recoverable) Codes with One Global Parity

Partial-MDS (PMDS) codes are a family of locally repairable codes, mainly used for distributed storage. They are defined to be able to correct any pattern of $s$ additional erasures, after a given number of erasures per locality group have occurred. This makes them also maximally recoverable (MR) codes, another class of locally repairable codes. It is known that MR codes in general, and PMDS codes in particular, exist for any set of parameters, if the field size is large enough. Moreover, some explicit constructions of PMDS codes are known, mostly (but not always) with a strong restriction on the number of erasures that can be corrected per locality group. In this paper we generalize the notion of PMDS codes to allow locality groups of different sizes. We give a general construction of such PMDS codes with $s=1$ global parity, i.e., one additional erasure can be corrected. Furthermore, we show that all PMDS codes for the given parameters are of this form, i.e., we give a classification of these codes. This implies a necessary and sufficient condition on the underlying field size for the existence of these codes (assuming that the MDS conjecture is true). For some parameter sets our generalized construction gives rise to PMDS codes with a smaller field size than any other known construction

Explicit Maximally Recoverable Codes with Locality

Consider a systematic linear code where some (local) parity symbols depend on few prescribed symbols, while other (heavy) parity symbols may depend on all data symbols. Local parities allow to quickly recover any single symbol when it is erased, while heavy parities provide tolerance to a large number of simultaneous erasures. A code as above is maximally-recoverable if it corrects all erasure patterns which are information theoretically recoverable given the code topology. In this paper we present explicit families of maximally-recoverable codes with locality. We also initiate the study of the trade-off between maximal recoverability and alphabet size