3,260 research outputs found
Alpha Entanglement Codes: Practical Erasure Codes to Archive Data in Unreliable Environments
Data centres that use consumer-grade disks drives and distributed
peer-to-peer systems are unreliable environments to archive data without enough
redundancy. Most redundancy schemes are not completely effective for providing
high availability, durability and integrity in the long-term. We propose alpha
entanglement codes, a mechanism that creates a virtual layer of highly
interconnected storage devices to propagate redundant information across a
large scale storage system. Our motivation is to design flexible and practical
erasure codes with high fault-tolerance to improve data durability and
availability even in catastrophic scenarios. By flexible and practical, we mean
code settings that can be adapted to future requirements and practical
implementations with reasonable trade-offs between security, resource usage and
performance. The codes have three parameters. Alpha increases storage overhead
linearly but increases the possible paths to recover data exponentially. Two
other parameters increase fault-tolerance even further without the need of
additional storage. As a result, an entangled storage system can provide high
availability, durability and offer additional integrity: it is more difficult
to modify data undetectably. We evaluate how several redundancy schemes perform
in unreliable environments and show that alpha entanglement codes are flexible
and practical codes. Remarkably, they excel at code locality, hence, they
reduce repair costs and become less dependent on storage locations with poor
availability. Our solution outperforms Reed-Solomon codes in many disaster
recovery scenarios.Comment: The publication has 12 pages and 13 figures. This work was partially
supported by Swiss National Science Foundation SNSF Doc.Mobility 162014, 2018
48th Annual IEEE/IFIP International Conference on Dependable Systems and
Networks (DSN
Multiresolutional Fault-Tolerant Sensor Integration and Object Recognition in Images.
This dissertation applies multiresolution methods to two important problems in signal analysis. The problem of fault-tolerant sensor integration in distributed sensor networks is addressed, and an efficient multiresolutional algorithm for estimating the sensors\u27 effective output is proposed. The problem of object/shape recognition in images is addressed in a multiresolutional setting using pyramidal decomposition of images with respect to an orthonormal wavelet basis. A new approach to efficient template matching to detect objects using computational geometric methods is put forward. An efficient paradigm for object recognition is described
Durability and Availability of Erasure-Coded Storage Systems with Concurrent Maintenance
This initial version of this document was written back in 2014 for the sole
purpose of providing fundamentals of reliability theory as well as to identify
the theoretical types of machinery for the prediction of
durability/availability of erasure-coded storage systems. Since the definition
of a "system" is too broad, we specifically focus on warm/cold storage systems
where the data is stored in a distributed fashion across different storage
units with or without continuous operation. The contents of this document are
dedicated to a review of fundamentals, a few major improved stochastic models,
and several contributions of my work relevant to the field. One of the
contributions of this document is the introduction of the most general form of
Markov models for the estimation of mean time to failure. This work was
partially later published in IEEE Transactions on Reliability. Very good
approximations for the closed-form solutions for this general model are also
investigated. Various storage configurations under different policies are
compared using such advanced models. Later in a subsequent chapter, we have
also considered multi-dimensional Markov models to address detached
drive-medium combinations such as those found in optical disk and tape storage
systems. It is not hard to anticipate such a system structure would most likely
be part of future DNA storage libraries. This work is partially published in
Elsevier Reliability and System Safety. Topics that include simulation
modelings for more accurate estimations are included towards the end of the
document by noting the deficiencies of the simplified canonical as well as more
complex Markov models, due mainly to the stationary and static nature of
Markovinity. Throughout the document, we shall focus on concurrently maintained
systems although the discussions will only slightly change for the systems
repaired one device at a time.Comment: 58 pages, 20 figures, 9 tables. arXiv admin note: substantial text
overlap with arXiv:1911.0032
- …