2 research outputs found
Joint Latency and Cost Optimization for Erasure-coded Data Center Storage
Modern distributed storage systems offer large capacity to satisfy the
exponentially increasing need of storage space. They often use erasure codes to
protect against disk and node failures to increase reliability, while trying to
meet the latency requirements of the applications and clients. This paper
provides an insightful upper bound on the average service delay of such
erasure-coded storage with arbitrary service time distribution and consisting
of multiple heterogeneous files. Not only does the result supersede known delay
bounds that only work for a single file or homogeneous files, it also enables a
novel problem of joint latency and storage cost minimization over three
dimensions: selecting the erasure code, placement of encoded chunks, and
optimizing scheduling policy. The problem is efficiently solved via the
computation of a sequence of convex approximations with provable convergence.
We further prototype our solution in an open-source, cloud storage deployment
over three geographically distributed data centers. Experimental results
validate our theoretical delay analysis and show significant latency reduction,
providing valuable insights into the proposed latency-cost tradeoff in
erasure-coded storage.Comment: 14 pages, presented in part at IFIP Performance, Oct 201
1 Scheduling Memory Access on a Distributed Cloud Storage Network
Abstract—Memory-access speed continues falling behind the growing speeds of network transmission links. High-speed network links provide a means to connect memory placed in hosts, located in different corners of the network. These hosts are called storage system units (SSUs), where data can be stored. Cloud storage provided with a single server can facilitate large amounts of storage to a user, however, at low access speeds. A distributed approach to cloud storage is an attractive solution. In a distributed cloud, small high-speed memories at SSUs can potentially increase the memory access speed for data processing and transmission. However, the latencies of each SSUs may be different. Therefore, the selection of SSUs impacts the overall memory access speed. This paper proposes a latency-aware scheduling scheme to access data from SSUs. This scheme determines the minimum latency requirement for a given dataset and selects available SSUs with the required latencies. Furthermore, because the latencies of some selected SSUs may be large, the proposed scheme notifies SSUs in advance of the expected time to perform data access. The simulation results show that the proposed scheme achieves faster access speeds than a scheme that randomly selects SSUs and another hat greedily selects SSUs with small latencies. I