Practical Functional Regenerating Codes for Broadcast Repair of Multiple Nodes

A code construction and repair scheme for optimal functional regeneration of multiple node failures is presented, which is based on stitching together short MDS codes on carefully chosen sets of points lying on a linearized polynomial. The nodes are connected wirelessly, hence all transmissions by helper nodes during a repair round are available to all the nodes being repaired. The scheme is simple and practical because of low subpacketization, low I/O cost and low computational cost. Achievability of the minimum-bandwidth regenerating (MBR) point, as well as an interior point, on the optimal storage-repair bandwidth tradeoff curve is shown. The subspace properties derived in the paper provide insight into the general properties of functional regenerating codes.Comment: 5 pages, ISIT 201

Secure Distributed Storage: Rate-Privacy Trade-Off and XOR-Based Coding Scheme

We consider the problem of storing data in a distributed manner over $T$ servers. We require the data (i) to be recoverable from the $T$ servers, and (ii) to remain private from any $T-1$ colluding servers, where privacy is quantified in terms of mutual information between the data and all the information available at the $T-1$ colluding servers. For this model, we determine (i) the fundamental trade-off between storage size and the level of desired privacy, (ii) the optimal amount of local randomness necessary at the encoder, and (iii)~an explicit low-complexity coding scheme that solely relies on XOR operations and that asymptotically (with the data size) matches the fundamental limits found.Comment: 6 pages, full version of paper accepted to the 2020 IEEE International Symposium on Information Theor

The Storage vs Repair Bandwidth Trade-off for Multiple Failures in Clustered Storage Networks

We study the trade-off between storage overhead and inter-cluster repair bandwidth in clustered storage systems, while recovering from multiple node failures within a cluster. A cluster is a collection of $m$ nodes, and there are $n$ clusters. For data collection, we download the entire content from any $k$ clusters. For repair of $t \geq 2$ nodes within a cluster, we take help from $\ell$ local nodes, as well as $d$ helper clusters. We characterize the optimal trade-off under functional repair, and also under exact repair for the minimum storage and minimum inter-cluster bandwidth (MBR) operating points. Our bounds show the following interesting facts: $1)$ When $t|(m-\ell)$ the trade-off is the same as that under $t=1$, and thus there is no advantage in jointly repairing multiple nodes, $2)$ When $t \nmid (m-\ell)$, the optimal file-size at the MBR point under exact repair can be strictly less than that under functional repair. $3)$ Unlike the case of $t=1$, increasing the number of local helper nodes does not necessarily increase the system capacity under functional repair.Comment: Accepted to IEEE Information Theory Workshop(ITW) 201

Staircase Codes for Secret Sharing with Optimal Communication and Read Overheads

We study the communication efficient secret sharing (CESS) problem introduced by Huang, Langberg, Kliewer and Bruck. A classical threshold secret sharing scheme randomly encodes a secret into $n$ shares given to $n$ parties, such that any set of at least $t$, $t<n$, parties can reconstruct the secret, and any set of at most $z$, $z<t$, parties cannot obtain any information about the secret. Recently, Huang et al. characterized the achievable minimum communication overhead (CO) necessary for a legitimate user to decode the secret when contacting $d\geq t$ parties and presented explicit code constructions achieving minimum CO for $d=n$. The intuition behind the possible savings on CO is that the user is only interested in decoding the secret and does not have to decode the random keys involved in the encoding process. In this paper, we introduce a new class of linear CESS codes called Staircase Codes over any field $GF(q)$, for any prime power $q> n$. We describe two explicit constructions of Staircase codes that achieve minimum communication and read overheads respectively for a fixed $d$, and universally for all possible values of $d, t\leq d\leq n$.Comment: Submitted to IEEE Transactions on Information Theor

Repairing Multiple Failures for Scalar MDS Codes

In distributed storage, erasure codes -- like Reed-Solomon Codes -- are often employed to provide reliability. In this setting, it is desirable to be able to repair one or more failed nodes while minimizing the repair bandwidth. In this work, motivated by Reed-Solomon codes, we study the problem of repairing multiple failed nodes in a scalar MDS code. We extend the framework of (Guruswami and Wootters, 2017) to give a framework for constructing repair schemes for multiple failures in general scalar MDS codes, in the centralized repair model. We then specialize our framework to Reed-Solomon codes, and extend and improve upon recent results of (Dau et al., 2017)

New constructions of cooperative MSR codes: Reducing node size to exp⁡(O(n))\exp(O(n))

We consider the problem of multiple-node repair in distributed storage systems under the cooperative model, where the repair bandwidth includes the amount of data exchanged between any two different storage nodes. Recently, explicit constructions of MDS codes with optimal cooperative repair bandwidth for all possible parameters were given by Ye and Barg (IEEE Transactions on Information Theory, 2019). The node size (or sub-packetization) in this construction scales as $\exp(\Theta(n^h))$, where $h$ is the number of failed nodes and $n$ is the code length. In this paper, we give new explicit constructions of optimal MDS codes for all possible parameters under the cooperative model, and the node size of our new constructions only scales as $\exp(O(n))$ for any number of failed nodes. Furthermore, it is known that any optimal MDS code under the cooperative model (including, in particular, our new code construction) also achieves optimal repair bandwidth under the centralized model, where the amount of data exchanged between failed nodes is not included in the repair bandwidth. We further show that the node size of our new construction is also much smaller than that of the best known MDS code constructions for the centralized model

Capacity of Wireless Distributed Storage Systems with Broadcast Repair

In wireless distributed storage systems, storage nodes are connected by wireless channels, which are broadcast in nature. This paper exploits this unique feature to design an efficient repair mechanism, called broadcast repair, for wireless distributed storage systems in the presence of multiple-node failures. Due to the broadcast nature of wireless transmission, we advocate a new measure on repair performance called repair-transmission bandwidth. In contrast to repair bandwidth, which measures the average number of packets downloaded by a newcomer to replace a failed node, repair-transmission bandwidth measures the average number of packets transmitted by helper nodes per failed node. A fundamental study on the storage capacity of wireless distributed storage systems with broadcast repair is conducted by modeling the storage system as a multicast network and analyzing the minimum cut of the corresponding information flow graph. The fundamental tradeoff between storage efficiency and repair-transmission bandwidth is also obtained for functional repair. The performance of broadcast repair is compared both analytically and numerically with that of cooperative repair, the basic repair method for wired distributed storage systems with multiple-node failures. While cooperative repair is based on the idea of allowing newcomers to exchange packets, broadcast repair is based on the idea of allowing a helper to broadcast packets to all newcomers simultaneously. We show that broadcast repair outperforms cooperative repair, offering a better tradeoff between storage efficiency and repair-transmission bandwidth.Comment: 28 pages, 7 figure

On the Achievability Region of Regenerating Codes for Multiple Erasures

We study the problem of centralized exact repair of multiple failures in distributed storage. We describe constructions that achieve a new set of interior points under exact repair. The constructions build upon the layered code construction by Tian et al., designed for exact repair of single failure. We firstly improve upon the layered construction for general system parameters. Then, we extend the improved construction to support the repair of multiple failures, with varying number of helpers. In particular, we prove the optimality of one point on the functional repair tradeoff of multiple failures for some parameters. Finally, considering minimum bandwidth cooperative repair (MBCR) codes as centralized repair codes, we determine explicitly the best achievable region obtained by space-sharing among all known points, including the MBCR point

Explicit constructions of high-rate MDS array codes with optimal repair bandwidth

Maximum distance separable (MDS) codes are optimal error-correcting codes in the sense that they provide the maximum failure-tolerance for a given number of parity nodes. Suppose that an MDS code with $k$ information nodes and $r=n-k$ parity nodes is used to encode data in a distributed storage system. It is known that if $h$ out of the $n$ nodes are inaccessible and $d$ surviving (helper) nodes are used to recover the lost data, then we need to download at least $h/(d+h-k)$ fraction of the data stored in each of the helper nodes (Dimakis et. al., 2010 and Cadambe et al., 2013). If this lower bound is achieved for the repair of any $h$ erased nodes from any $d$ helper nodes, we say that the MDS code has the $(h,d)$-optimal repair property. We study high-rate MDS array codes with the optimal repair property. Explicit constructions of such codes in the literature are only available for the cases where there are at most 3 parity nodes, and these existing constructions can only optimally repair a single node failure by accessing all the surviving nodes. In this paper, given any $r$ and $n$, we present two explicit constructions of MDS array codes with the $(h,d)$-optimal repair property for all $h\le r$ and $k\le d\le n-h$ simultaneously. Codes in the first family can be constructed over any base field $F$ as long as $|F|\ge sn,$ where $s=\text{lcm}(1,2,\dots,r).$ The encoding, decoding, repair of failed nodes, and update procedures of these codes all have low complexity. Codes in the second family have the optimal access property and can be constructed over any base field $F$ as long as $|F|\ge n+1.$ Moreover, both code families have the optimal error resilience capability when repairing failed nodes. We also construct several other related families of MDS codes with the optimal repair property.Comment: 19pp., submitted for publication. This version contains a few additional reference

Explicit constructions of MSR codes for clustered distributed storage: The rack-aware storage model

The paper is devoted to the problem of erasure coding in distributed storage. We consider a model of storage that assumes that nodes are organized into equally sized groups, called racks, that within each group the nodes can communicate freely without taxing the system bandwidth, and that the only information transmission that counts is the one between the racks. This assumption implies that the nodes within each of the racks can collaborate before providing information to the failed node. The main emphasis of the paper is on code construction for this storage model. We present an explicit family of MDS array codes that support recovery of a single failed node from any number of helper racks using the minimum possible amount of inter-rack communication (such codes are said to provide optimal repair). The codes are constructed over finite fields of size comparable to the code length. We also derive a bound on the number of symbols accessed at helper nodes for the purposes of repair, and construct a code family that approaches this bound, while still maintaining the optimal repair property. Finally, we present a construction of scalar Reed-Solomon codes that support optimal repair for the rack-oriented storage model.Comment: 24 page