Joint Latency and Cost Optimization for Erasure-coded Data Center Storage

Modern distributed storage systems offer large capacity to satisfy the exponentially increasing need of storage space. They often use erasure codes to protect against disk and node failures to increase reliability, while trying to meet the latency requirements of the applications and clients. This paper provides an insightful upper bound on the average service delay of such erasure-coded storage with arbitrary service time distribution and consisting of multiple heterogeneous files. Not only does the result supersede known delay bounds that only work for a single file or homogeneous files, it also enables a novel problem of joint latency and storage cost minimization over three dimensions: selecting the erasure code, placement of encoded chunks, and optimizing scheduling policy. The problem is efficiently solved via the computation of a sequence of convex approximations with provable convergence. We further prototype our solution in an open-source, cloud storage deployment over three geographically distributed data centers. Experimental results validate our theoretical delay analysis and show significant latency reduction, providing valuable insights into the proposed latency-cost tradeoff in erasure-coded storage.Comment: 14 pages, presented in part at IFIP Performance, Oct 201

Mirrored and Hybrid Disk Arrays: Organization, Scheduling, Reliability, and Performance

Basic mirroring (BM) classified as RAID level 1 replicates data on two disks, thus doubling disk access bandwidth for read requests. RAID1/0 is an array of BM pairs with balanced loads due to striping. When a disk fails the read load on its pair is doubled, which results in halving the maximum attainable bandwidth. We review RAID1 organizations which attain a balanced load upon disk failure, but as shown by reliability analysis tend to be less reliable than RAID1/0. Hybrid disk arrays which store XORed instead of replicated data tend to have a higher reliability than mirrored disks, but incur a higher overhead in updating data. Read request response time can be improved by processing them at a higher priority than writes, since they have a direct effect on application response time. Shortest seek distance and affinity based routing both shorten seek time. Anticipatory arm placement places arms optimally to minimize the seek distance. The analysis of RAID1 in normal, degraded, and rebuild mode is provided to quantify RAID1/0 performance. We compare the reliability of mirrored disk organizations against each other and hybrid disks and erasure coded disk arrays

Distributed Data Collection and Storage Algorithms for Collaborative Learning Vision Sensor Devices with Applications to Pilgrimage

This work presents novel distributed data collection systems and storage algorithms for collaborative learning wireless sensor networks (WSNs). In a large WSN, consider $n$ collaborative sensor devices distributed randomly to acquire information and learn about a certain field. Such sensors have less power, small bandwidth, and short memory, and they might disappear from the network after certain time of operations. The goal of this work is to design efficient strategies to learn about the field by collecting sensed data from these $n$ sensors with less computational overhead and efficient storage encoding operations. In this data collection system, we propose two distributed data storage algorithms (DSA's) to solve this problem with the means of network flooding and connectivity among sensor devices. In the first algorithm denoted, DSA-I, it's assumed that the total number of nodes is known for each node in the network. We show that this algorithm is efficient in terms of the encoding/decoding operations. Furthermore, every node uses network flooding to disseminate its data throughout the network using mixing time approximately O(n). In the second algorithm denoted, DSA-II, it's assumed that the total number of nodes is not known for each learning sensor, hence dissemination of the data does not depend on the value of $n$. In this case we show that the encoding operations take $O(C\mu^2)$, where $\mu$ is the mean degree of the network graph and $C$ is a system parameter. Performance of these two algorithms match the derived theoretical results. Finally, we show how to deploy these algorithms for monitoring and measuring certain phenomenons in American-made camp tents located in Minna field in south-east side of Makkah.Comment: arXiv admin note: substantial text overlap with arXiv:0908.441

Decentralized Coding Algorithms for Distributed Storage in Wireless Sensor Networks

We consider large-scale wireless sensor networks with $n$ nodes, out of which k are in possession, (e.g., have sensed or collected in some other way) k information packets. In the scenarios in which network nodes are vulnerable because of, for example, limited energy or a hostile environment, it is desirable to disseminate the acquired information throughout the network so that each of the n nodes stores one (possibly coded) packet so that the original k source packets can be recovered, locally and in a computationally simple way from any k(1 + \epsilon) nodes for some small \epsilon > 0. We develop decentralized Fountain codes based algorithms to solve this problem. Unlike all previously developed schemes, our algorithms are truly distributed, that is, nodes do not know n, k or connectivity in the network, except in their own neighborhoods, and they do not maintain any routing tables.Comment: Accepted for publication in IEEE JSAC, 201

Multilevel Diversity Coding Systems: Rate Regions, Codes, Computation, & Forbidden Minors

The rate regions of multilevel diversity coding systems (MDCS), a sub-class of the broader family of multi-source multi-sink networks with special structure, are investigated. After showing how to enumerate all non-isomorphic MDCS instances of a given size, the Shannon outer bound and several achievable inner bounds based on linear codes are given for the rate region of each non-isomorphic instance. For thousands of MDCS instances, the bounds match, and hence exact rate regions are proven. Results gained from these computations are summarized in key statistics involving aspects such as the sufficiency of scalar binary codes, the necessary size of vector binary codes, etc. Also, it is shown how to generate computer aided human readable converse proofs, as well as how to construct the codes for an achievability proof. Based on this large repository of rate regions, a series of results about general MDCS cases that they inspired are introduced and proved. In particular, a series of embedding operations that preserve the property of sufficiency of scalar or vector codes are presented. The utility of these operations is demonstrated by boiling the thousands of MDCS instances for which binary scalar codes are insufficient down to 12 forbidden smallest embedded MDCS instances.Comment: Submitted to IEEE Transactions on Information Theory, 52 page

A Survey of Delay Tolerant Networks Routing Protocols

Advances in Micro-Electro-Mechanical Systems (MEMS) have revolutionized the digital age to a point where animate and inanimate objects can be used as a communication channel. In addition, the ubiquity of mobile phones with increasing capabilities and ample resources means people are now effectively mobile sensors that can be used to sense the environment as well as data carriers. These objects, along with their devices, form a new kind of networks that are characterized by frequent disconnections, resource constraints and unpredictable or stochastic mobility patterns. A key underpinning in these networks is routing or data dissemination protocols that are designed specifically to handle the aforementioned characteristics. Therefore, there is a need to review state-of-the-art routing protocols, categorize them, and compare and contrast their approaches in terms of delivery rate, resource consumption and end-to-end delay. To this end, this paper reviews 63 unicast, multicast and coding-based routing protocols that are designed specifically to run in delay tolerant or challenged networks. We provide an extensive qualitative comparison of all protocols, highlight their experimental setup and outline their deficiencies in terms of design and research methodology. Apart from that, we review research that aims to exploit studies on social networks and epidemiology in order to improve routing protocol performance. Lastly, we provide a list of future research directions.Comment: 56 page

A Distributed Data Collection Algorithm for Wireless Sensor Networks with Persistent Storage Nodes

A distributed data collection algorithm to accurately store and forward information obtained by wireless sensor networks is proposed. The proposed algorithm does not depend on the sensor network topology, routing tables, or geographic locations of sensor nodes, but rather makes use of uniformly distributed storage nodes. Analytical and simulation results for this algorithm show that, with high probability, the data disseminated by the sensor nodes can be precisely collected by querying any small set of storage nodes

Optimal Control of Storage Regeneration with Repair Codes

High availability of containerized applications requires to perform robust storage of applications' state. Since basic replication techniques are extremely costly at scale, storage space requirements can be reduced by means of erasure or repairing codes. In this paper we address storage regeneration using repair codes, a robust distributed storage technique with no need to fully restore the whole state in case of failure. In fact, only the lost servers' content is replaced. To do so, new cleanslate storage units are made operational at a cost for activating new storage servers and a cost for the transfer of repair data. Our goal is to guarantee maximal availability of containers' state files by a given deadline. activation of servers and communication cost. Upon a fault occurring at a subset of the storage servers, we aim at ensuring that they are repaired by a given deadline. We introduce a controlled fluid model and derive the optimal activation policy to replace servers under such correlated faults. The solution concept is the optimal control of regeneration via the Pontryagin minimum principle. We characterise feasibility conditions and we prove that the optimal policy is of threshold type. Numerical results describe how to apply the model for system dimensioning and show the tradeoff betweenComment: This research was performed while the first author was visiting Nokia Bell Lab

Application-Driven Near-Data Processing for Similarity Search

Similarity search is a key to a variety of applications including content-based search for images and video, recommendation systems, data deduplication, natural language processing, computer vision, databases, computational biology, and computer graphics. At its core, similarity search manifests as k-nearest neighbors (kNN), a computationally simple primitive consisting of highly parallel distance calculations and a global top-k sort. However, kNN is poorly supported by today's architectures because of its high memory bandwidth requirements. This paper proposes an application-driven near-data processing accelerator for similarity search: the Similarity Search Associative Memory (SSAM). By instantiating compute units close to memory, SSAM benefits from the higher memory bandwidth and density exposed by emerging memory technologies. We evaluate the SSAM design down to layout on top of the Micron hybrid memory cube (HMC), and show that SSAM can achieve up to two orders of magnitude area-normalized throughput and energy efficiency improvement over multicore CPUs; we also show SSAM is faster and more energy efficient than competing GPUs and FPGAs. Finally, we show that SSAM is also useful for other data intensive tasks like kNN index construction, and can be generalized to semantically function as a high capacity content addressable memory.Comment: 15 pages, 8 figures, 7 table

Repairing Multiple Failures with Coordinated and Adaptive Regenerating Codes

Erasure correcting codes are widely used to ensure data persistence in distributed storage systems. This paper addresses the simultaneous repair of multiple failures in such codes. We go beyond existing work (i.e., regenerating codes by Dimakis et al.) by describing (i) coordinated regenerating codes (also known as cooperative regenerating codes) which support the simultaneous repair of multiple devices, and (ii) adaptive regenerating codes which allow adapting the parameters at each repair. Similarly to regenerating codes by Dimakis et al., these codes achieve the optimal tradeoff between storage and the repair bandwidth. Based on these extended regenerating codes, we study the impact of lazy repairs applied to regenerating codes and conclude that lazy repairs cannot reduce the costs in term of network bandwidth but allow reducing the disk-related costs (disk bandwidth and disk I/O).Comment: Update to previous version adding (i) study of lazy repairs, (ii) adaptive codes at the MBR point, and (iii) discussion of related work. Extended from a regular paper at NetCod 2011 available at http://dx.doi.org/10.1109/ISNETCOD.2011.5978920 . First version: "Beyond Regenerating Codes", September 2010 on http://hal.inria.fr/inria-00516647