52 research outputs found
Locality and Availability in Distributed Storage
This paper studies the problem of code symbol availability: a code symbol is
said to have -availability if it can be reconstructed from disjoint
groups of other symbols, each of size at most . For example, -replication
supports -availability as each symbol can be read from its other
(disjoint) replicas, i.e., . However, the rate of replication must vanish
like as the availability increases.
This paper shows that it is possible to construct codes that can support a
scaling number of parallel reads while keeping the rate to be an arbitrarily
high constant. It further shows that this is possible with the minimum distance
arbitrarily close to the Singleton bound. This paper also presents a bound
demonstrating a trade-off between minimum distance, availability and locality.
Our codes match the aforementioned bound and their construction relies on
combinatorial objects called resolvable designs.
From a practical standpoint, our codes seem useful for distributed storage
applications involving hot data, i.e., the information which is frequently
accessed by multiple processes in parallel.Comment: Submitted to ISIT 201
Coding for the Clouds: Coding Techniques for Enabling Security, Locality, and Availability in Distributed Storage Systems
Cloud systems have become the backbone of many applications such as multimedia
streaming, e-commerce, and cluster computing. At the foundation of any cloud architecture
lies a large-scale, distributed, data storage system. To accommodate the massive
amount of data being stored on the cloud, these distributed storage systems (DSS) have
been scaled to contain hundreds to thousands of nodes that are connected through a networking
infrastructure. Such data-centers are usually built out of commodity components,
which make failures the norm rather than the exception.
In order to combat node failures, data is typically stored in a redundant fashion. Due to
the exponential data growth rate, many DSS are beginning to resort to error control coding
over conventional replication methods, as coding offers high storage space efficiency. This
paradigm shift from replication to coding, along with the need to guarantee reliability, efficiency,
and security in DSS, has created a new set of challenges and opportunities, opening
up a new area of research. This thesis addresses several of these challenges and opportunities
by broadly making the following contributions. (i) We design practically amenable,
low-complexity coding schemes that guarantee security of cloud systems, ensure quick
recovery from failures, and provide high availability for retrieving partial information; and
(ii) We analyze fundamental performance limits and optimal trade-offs between the key
performance metrics of these coding schemes.
More specifically, we first consider the problem of achieving information-theoretic
security in DSS against an eavesdropper that can observe a limited number of nodes. We
present a framework that enables design of secure repair-efficient codes through a joint
construction of inner and outer codes. Then, we consider a practically appealing notion
of weakly secure coding, and construct coset codes that can weakly secure a wide class of regenerating codes that reduce the amount of data downloaded during node repair.
Second, we consider the problem of meeting repair locality constraints, which specify
the number of nodes participating in the repair process. We propose a notion of unequal
locality, which enables different locality values for different nodes, ensuring quick recovery
for nodes storing important data. We establish tight upper bounds on the minimum
distance of linear codes with unequal locality, and present optimal code constructions.
Next, we extend the notion of locality from the Hamming metric to the rank and subspace
metrics, with the goal of designing codes for efficient data recovery from special types of
correlated failures in DSS.We construct a family of locally recoverable rank-metric codes
with optimal data recovery properties.
Finally, we consider the problem of providing high availability, which is ensured by
enabling node repair from multiple disjoint subsets of nodes of small size. We study
codes with availability from a queuing-theoretical perspective by analyzing the average
time necessary to download a block of data under the Poisson request arrival model when
each node takes a random amount of time to fetch its contents. We compare the delay
performance of the availability codes with several alternatives such as conventional erasure
codes and replication schemes
Coding for the Clouds: Coding Techniques for Enabling Security, Locality, and Availability in Distributed Storage Systems
Cloud systems have become the backbone of many applications such as multimedia
streaming, e-commerce, and cluster computing. At the foundation of any cloud architecture
lies a large-scale, distributed, data storage system. To accommodate the massive
amount of data being stored on the cloud, these distributed storage systems (DSS) have
been scaled to contain hundreds to thousands of nodes that are connected through a networking
infrastructure. Such data-centers are usually built out of commodity components,
which make failures the norm rather than the exception.
In order to combat node failures, data is typically stored in a redundant fashion. Due to
the exponential data growth rate, many DSS are beginning to resort to error control coding
over conventional replication methods, as coding offers high storage space efficiency. This
paradigm shift from replication to coding, along with the need to guarantee reliability, efficiency,
and security in DSS, has created a new set of challenges and opportunities, opening
up a new area of research. This thesis addresses several of these challenges and opportunities
by broadly making the following contributions. (i) We design practically amenable,
low-complexity coding schemes that guarantee security of cloud systems, ensure quick
recovery from failures, and provide high availability for retrieving partial information; and
(ii) We analyze fundamental performance limits and optimal trade-offs between the key
performance metrics of these coding schemes.
More specifically, we first consider the problem of achieving information-theoretic
security in DSS against an eavesdropper that can observe a limited number of nodes. We
present a framework that enables design of secure repair-efficient codes through a joint
construction of inner and outer codes. Then, we consider a practically appealing notion
of weakly secure coding, and construct coset codes that can weakly secure a wide class of regenerating codes that reduce the amount of data downloaded during node repair.
Second, we consider the problem of meeting repair locality constraints, which specify
the number of nodes participating in the repair process. We propose a notion of unequal
locality, which enables different locality values for different nodes, ensuring quick recovery
for nodes storing important data. We establish tight upper bounds on the minimum
distance of linear codes with unequal locality, and present optimal code constructions.
Next, we extend the notion of locality from the Hamming metric to the rank and subspace
metrics, with the goal of designing codes for efficient data recovery from special types of
correlated failures in DSS.We construct a family of locally recoverable rank-metric codes
with optimal data recovery properties.
Finally, we consider the problem of providing high availability, which is ensured by
enabling node repair from multiple disjoint subsets of nodes of small size. We study
codes with availability from a queuing-theoretical perspective by analyzing the average
time necessary to download a block of data under the Poisson request arrival model when
each node takes a random amount of time to fetch its contents. We compare the delay
performance of the availability codes with several alternatives such as conventional erasure
codes and replication schemes
Capacity of Locally Recoverable Codes
Motivated by applications in distributed storage, the notion of a locally
recoverable code (LRC) was introduced a few years back. In an LRC, any
coordinate of a codeword is recoverable by accessing only a small number of
other coordinates. While different properties of LRCs have been well-studied,
their performance on channels with random erasures or errors has been mostly
unexplored. In this note, we analyze the performance of LRCs over such
stochastic channels. In particular, for input-symmetric discrete memoryless
channels, we give a tight characterization of the gap to Shannon capacity when
LRCs are used over the channel.Comment: Invited paper to the Information Theory Workshop (ITW) 201
Optimal Linear and Cyclic Locally Repairable Codes over Small Fields
We consider locally repairable codes over small fields and propose
constructions of optimal cyclic and linear codes in terms of the dimension for
a given distance and length. Four new constructions of optimal linear codes
over small fields with locality properties are developed. The first two
approaches give binary cyclic codes with locality two. While the first
construction has availability one, the second binary code is characterized by
multiple available repair sets based on a binary Simplex code. The third
approach extends the first one to q-ary cyclic codes including (binary)
extension fields, where the locality property is determined by the properties
of a shortened first-order Reed-Muller code. Non-cyclic optimal binary linear
codes with locality greater than two are obtained by the fourth construction.Comment: IEEE Information Theory Workshop (ITW) 2015, Apr 2015, Jerusalem,
Israe
Constructions of Batch Codes via Finite Geometry
A primitive -batch code encodes a string of length into string
of length , such that each multiset of symbols from has mutually
disjoint recovering sets from . We develop new explicit and random coding
constructions of linear primitive batch codes based on finite geometry. In some
parameter regimes, our proposed codes have lower redundancy than previously
known batch codes.Comment: 7 pages, 1 figure, 1 tabl
Optimal Binary Locally Repairable Codes via Anticodes
This paper presents a construction for several families of optimal binary
locally repairable codes (LRCs) with small locality (2 and 3). This
construction is based on various anticodes. It provides binary LRCs which
attain the Cadambe-Mazumdar bound. Moreover, most of these codes are optimal
with respect to the Griesmer bound
- …