3,420 research outputs found
CORE: Augmenting Regenerating-Coding-Based Recovery for Single and Concurrent Failures in Distributed Storage Systems
Data availability is critical in distributed storage systems, especially when
node failures are prevalent in real life. A key requirement is to minimize the
amount of data transferred among nodes when recovering the lost or unavailable
data of failed nodes. This paper explores recovery solutions based on
regenerating codes, which are shown to provide fault-tolerant storage and
minimum recovery bandwidth. Existing optimal regenerating codes are designed
for single node failures. We build a system called CORE, which augments
existing optimal regenerating codes to support a general number of failures
including single and concurrent failures. We theoretically show that CORE
achieves the minimum possible recovery bandwidth for most cases. We implement
CORE and evaluate our prototype atop a Hadoop HDFS cluster testbed with up to
20 storage nodes. We demonstrate that our CORE prototype conforms to our
theoretical findings and achieves recovery bandwidth saving when compared to
the conventional recovery approach based on erasure codes.Comment: 25 page
HFR Code: A Flexible Replication Scheme for Cloud Storage Systems
Fractional repetition (FR) codes are a family of repair-efficient storage
codes that provide exact and uncoded node repair at the minimum bandwidth
regenerating point. The advantageous repair properties are achieved by a
tailor-made two-layer encoding scheme which concatenates an outer
maximum-distance-separable (MDS) code and an inner repetition code. In this
paper, we generalize the application of FR codes and propose heterogeneous
fractional repetition (HFR) code, which is adaptable to the scenario where the
repetition degrees of coded packets are different. We provide explicit code
constructions by utilizing group divisible designs, which allow the design of
HFR codes over a large range of parameters. The constructed codes achieve the
system storage capacity under random access repair and have multiple repair
alternatives for node failures. Further, we take advantage of the systematic
feature of MDS codes and present a novel design framework of HFR codes, in
which storage nodes can be wisely partitioned into clusters such that data
reconstruction time can be reduced when contacting nodes in the same cluster.Comment: Accepted for publication in IET Communications, Jul. 201
Cycle-Consistent Deep Generative Hashing for Cross-Modal Retrieval
In this paper, we propose a novel deep generative approach to cross-modal
retrieval to learn hash functions in the absence of paired training samples
through the cycle consistency loss. Our proposed approach employs adversarial
training scheme to lean a couple of hash functions enabling translation between
modalities while assuming the underlying semantic relationship. To induce the
hash codes with semantics to the input-output pair, cycle consistency loss is
further proposed upon the adversarial training to strengthen the correlations
between inputs and corresponding outputs. Our approach is generative to learn
hash functions such that the learned hash codes can maximally correlate each
input-output correspondence, meanwhile can also regenerate the inputs so as to
minimize the information loss. The learning to hash embedding is thus performed
to jointly optimize the parameters of the hash functions across modalities as
well as the associated generative models. Extensive experiments on a variety of
large-scale cross-modal data sets demonstrate that our proposed method achieves
better retrieval results than the state-of-the-arts.Comment: To appeared on IEEE Trans. Image Processing. arXiv admin note: text
overlap with arXiv:1703.10593 by other author
- …