Access vs. Bandwidth in Codes for Storage

Maximum distance separable (MDS) codes are widely used in storage systems to protect against disk (node) failures. A node is said to have capacity $l$ over some field $\mathbb{F}$, if it can store that amount of symbols of the field. An $(n,k,l)$ MDS code uses $n$ nodes of capacity $l$ to store $k$ information nodes. The MDS property guarantees the resiliency to any $n-k$ node failures. An \emph{optimal bandwidth} (resp. \emph{optimal access}) MDS code communicates (resp. accesses) the minimum amount of data during the repair process of a single failed node. It was shown that this amount equals a fraction of $1/(n-k)$ of data stored in each node. In previous optimal bandwidth constructions, $l$ scaled polynomially with $k$ in codes with asymptotic rate $<1$. Moreover, in constructions with a constant number of parities, i.e. rate approaches 1, $l$ is scaled exponentially w.r.t. $k$. In this paper, we focus on the later case of constant number of parities $n-k=r$, and ask the following question: Given the capacity of a node $l$ what is the largest number of information disks $k$ in an optimal bandwidth (resp. access) $(k+r,k,l)$ MDS code. We give an upper bound for the general case, and two tight bounds in the special cases of two important families of codes. Moreover, the bounds show that in some cases optimal-bandwidth code has larger $k$ than optimal-access code, and therefore these two measures are not equivalent.Comment: This paper was presented in part at the IEEE International Symposium on Information Theory (ISIT 2012). submitted to IEEE transactions on information theor

An Alternate Construction of an Access-Optimal Regenerating Code with Optimal Sub-Packetization Level

Given the scale of today's distributed storage systems, the failure of an individual node is a common phenomenon. Various metrics have been proposed to measure the efficacy of the repair of a failed node, such as the amount of data download needed to repair (also known as the repair bandwidth), the amount of data accessed at the helper nodes, and the number of helper nodes contacted. Clearly, the amount of data accessed can never be smaller than the repair bandwidth. In the case of a help-by-transfer code, the amount of data accessed is equal to the repair bandwidth. It follows that a help-by-transfer code possessing optimal repair bandwidth is access optimal. The focus of the present paper is on help-by-transfer codes that employ minimum possible bandwidth to repair the systematic nodes and are thus access optimal for the repair of a systematic node. The zigzag construction by Tamo et al. in which both systematic and parity nodes are repaired is access optimal. But the sub-packetization level required is $r^k$ where $r$ is the number of parities and $k$ is the number of systematic nodes. To date, the best known achievable sub-packetization level for access-optimal codes is $r^{k/r}$ in a MISER-code-based construction by Cadambe et al. in which only the systematic nodes are repaired and where the location of symbols transmitted by a helper node depends only on the failed node and is the same for all helper nodes. Under this set-up, it turns out that this sub-packetization level cannot be improved upon. In the present paper, we present an alternate construction under the same setup, of an access-optimal code repairing systematic nodes, that is inspired by the zigzag code construction and that also achieves a sub-packetization level of $r^{k/r}$.Comment: To appear in National Conference on Communications 201

Fundamental Limits on Communication for Oblivious Updates in Storage Networks

In distributed storage systems, storage nodes intermittently go offline for numerous reasons. On coming back online, nodes need to update their contents to reflect any modifications to the data in the interim. In this paper, we consider a setting where no information regarding modified data needs to be logged in the system. In such a setting, a 'stale' node needs to update its contents by downloading data from already updated nodes, while neither the stale node nor the updated nodes have any knowledge as to which data symbols are modified and what their value is. We investigate the fundamental limits on the amount of communication necessary for such an "oblivious" update process. We first present a generic lower bound on the amount of communication that is necessary under any storage code with a linear encoding (while allowing non-linear update protocols). This lower bound is derived under a set of extremely weak conditions, giving all updated nodes access to the entire modified data and the stale node access to the entire stale data as side information. We then present codes and update algorithms that are optimal in that they meet this lower bound. Next, we present a lower bound for an important subclass of codes, that of linear Maximum-Distance-Separable (MDS) codes. We then present an MDS code construction and an associated update algorithm that meets this lower bound. These results thus establish the capacity of oblivious updates in terms of the communication requirements under these settings.Comment: IEEE Global Communications Conference (GLOBECOM) 201

Long MDS Codes for Optimal Repair Bandwidth

MDS codes are erasure-correcting codes that can correct the maximum number of erasures given the number of redundancy or parity symbols. If an MDS code has r parities and no more than r erasures occur, then by transmitting all the remaining data in the code one can recover the original information. However, it was shown that in order to recover a single symbol erasure, only a fraction of 1/r of the information needs to be transmitted. This fraction is called the repair bandwidth (fraction). Explicit code constructions were given in previous works. If we view each symbol in the code as a vector or a column, then the code forms a 2D array and such codes are especially widely used in storage systems. In this paper, we ask the following question: given the length of the column l, can we construct high-rate MDS array codes with optimal repair bandwidth of 1/r, whose code length is as long as possible? In this paper, we give code constructions such that the code length is (r + 1)log_r l

The MDS Queue: Analysing the Latency Performance of Erasure Codes

In order to scale economically, data centers are increasingly evolving their data storage methods from the use of simple data replication to the use of more powerful erasure codes, which provide the same level of reliability as replication but at a significantly lower storage cost. In particular, it is well known that Maximum-Distance-Separable (MDS) codes, such as Reed-Solomon codes, provide the maximum storage efficiency. While the use of codes for providing improved reliability in archival storage systems, where the data is less frequently accessed (or so-called "cold data"), is well understood, the role of codes in the storage of more frequently accessed and active "hot data", where latency is the key metric, is less clear. In this paper, we study data storage systems based on MDS codes through the lens of queueing theory, and term this the "MDS queue." We analytically characterize the (average) latency performance of MDS queues, for which we present insightful scheduling policies that form upper and lower bounds to performance, and are observed to be quite tight. Extensive simulations are also provided and used to validate our theoretical analysis. We also employ the framework of the MDS queue to analyse different methods of performing so-called degraded reads (reading of partial data) in distributed data storage

Data Secrecy in Distributed Storage Systems under Exact Repair

The problem of securing data against eavesdropping in distributed storage systems is studied. The focus is on systems that use linear codes and implement exact repair to recover from node failures.The maximum file size that can be stored securely is determined for systems in which all the available nodes help in repair (i.e., repair degree $d=n-1$, where $n$ is the total number of nodes) and for any number of compromised nodes. Similar results in the literature are restricted to the case of at most two compromised nodes. Moreover, new explicit upper bounds are given on the maximum secure file size for systems with $d<n-1$. The key ingredients for the contribution of this paper are new results on subspace intersection for the data downloaded during repair. The new bounds imply the interesting fact that the maximum data that can be stored securely decreases exponentially with the number of compromised nodes.Comment: Submitted to Netcod 201

Optimal Locally Repairable and Secure Codes for Distributed Storage Systems

This paper aims to go beyond resilience into the study of security and local-repairability for distributed storage systems (DSS). Security and local-repairability are both important as features of an efficient storage system, and this paper aims to understand the trade-offs between resilience, security, and local-repairability in these systems. In particular, this paper first investigates security in the presence of colluding eavesdroppers, where eavesdroppers are assumed to work together in decoding stored information. Second, the paper focuses on coding schemes that enable optimal local repairs. It further brings these two concepts together, to develop locally repairable coding schemes for DSS that are secure against eavesdroppers. The main results of this paper include: a. An improved bound on the secrecy capacity for minimum storage regenerating codes, b. secure coding schemes that achieve the bound for some special cases, c. a new bound on minimum distance for locally repairable codes, d. code construction for locally repairable codes that attain the minimum distance bound, and e. repair-bandwidth-efficient locally repairable codes with and without security constraints.Comment: Submitted to IEEE Transactions on Information Theor