26 research outputs found
Validation of Large Zoned RAID Systems
Building on our prior work we present an improved model for for large partial stripe following full stripe writes in RAID 5. This was necessary because we observed that our previous model tended to underestimate measured results. To date, we have only validated these models against RAID systems with at most four disks. Here we validate our improved model, and also our existing models for other read and write configurations, against measurements taken from an eight disk RAID array
Modelling and Validation of Response Times in Zoned RAID
We present and validate an enhanced analytical queueing network model of zoned RAID. The model focuses on RAID levels 01 and 5, and yields the distribution of I/O request response time. Whereas our previous work could only support arrival streams of I/O requests of the same type, the model presented here supports heterogeneous streams with a mixture of read and write requests. This improved realism is made possible through multiclass extensions to our existing model. When combined with priority queueing, this development also enables more accurate modelling of the way subtasks of RAID 5 write requests are scheduled. In all cases we derive analytical results for calculating not only the mean but also higher moments and the full distribution of I/O request response time. We validate our mode
Queueing network models of zoned RAID system performance
RAID systems are widely deployed, both as standalone storage solutions and as
the building blocks of modern virtualised storage platforms. An accurate model of
RAID system performance is therefore critical towards fulfilling quality of service
constraints for fast, reliable storage.
This thesis presents techniques and tools that model response times in zoned
RAID systems. The inputs to this analysis are a specified I/O request arrival
rate, an I/O request access profile, a given RAID configuration and physical disk
parameters. The primary output of this analysis is an approximation to the cumulative
distribution function of I/O request response time. From this, it is straightforward
to calculate response time quantiles, as well as the mean, variance and
higher moments of I/O request response time. The model supports RAID levels
0, 01, 10 and 5 and a variety of workload types.
Our RAID model is developed in a bottom-up hierarchical fashion. We begin by
modelling each zoned disk drive in the array as a single M/G/1 queue. The service
time is modelled as the sum of the random variables of seek time, rotational
latency and data transfer time. In doing so, we take into account the properties of
zoned disks. We then abstract a RAID system as a fork-join queueing network.
This comprises several queues, each of which represents one disk drive in the array.
We tailor our basic fork-join approximation to account for the I/O request
patterns associated with particular request types and request sizes under different
RAID levels. We extend the RAID and disk models to support bulk arrivals, requests
of different sizes and scheduling algorithms that reorder queueing requests
to minimise disk head positioning time. Finally, we develop a corresponding simulation
to improve and validate the model. To test the accuracy of all our models,
we validate them against disk drive and RAID device measurements throughout
SIMULATION AND MODELLING OF RAID 0 SYSTEM PERFORMANCE
RAID systems are fundamental components of modern storage infrastructures. It is therefore important to model their performance effectively. This paper describes a simulation model which predicts the cumulative distribution function of I/O request response time in a RAID 0 system consisting of homogeneous zoned disk drives. The model is constructed in a bottom-up manner, starting by abstracting a single disk drive as an M/G/1 queue. This is then extended to model a RAID 0 system using a split-merge queueing network. Simulation results of I/O request response time for RAID 0 systems with various numbers of disks are computed and compared against device measurements
Moment-Generating Algorithm for Response Time in Processor Sharing Queueing Systems
Response times are arguably the most representative and important metric for measuring the performance of modern computer systems. Further, service level agreements (SLAs), ranging from data centres to smartphone users, demand quick and, equally important, predictable response times. Hence, it is necessary to calculate moments, at least, and ideally response time distributions, which is not straightforward. A new moment-generating algorithm for calculating response times analytically is obtained, based on M/M/1 processor sharing (PS) queueing models. This algorithm is compared against existing work on response times in M/M/1-PS queues and extended to M/M/1 discriminatory PS queues. Two real-world case studies are evaluated
Data Management Strategies for Relative Quality of Service in Virtualised Storage Systems
The amount of data managed by organisations continues to grow relentlessly.
Driven by the high costs of maintaining multiple local storage systems, there
is a well established trend towards storage consolidation using multi-tier Virtualised Storage Systems (VSSs). At the same time, storage infrastructures
are increasingly subject to stringent Quality of Service (QoS) demands.
Within a VSS, it is challenging to match desired QoS with delivered QoS,
considering the latter can vary dramatically both across and within tiers.
Manual efforts to achieve this match require extensive and ongoing human
intervention. Automated efforts are based on workload analysis, which ignores
the business importance of infrequently accessed data.
This thesis presents our design, implementation and evaluation of data
maintenance strategies in an enhanced version of the popular Linux Extended
3 Filesystem which features support for the elegant specification
of QoS metadata while maintaining compatibility with stock kernels. Users
and applications specify QoS requirements using a chmod-like interface. System
administrators are provided with a character device kernel interface
that allows for profiling of the QoS delivered by the underlying storage. We
propose a novel score-based metric, together with associated visualisation
resources, to evaluate the degree of QoS matching achieved by any given
data layout. We also design and implement new inode and datablock allocation
and migration strategies which exploit this metric in seeking to match
the QoS attributes set by users and/or applications on files and directories
with the QoS actually delivered by each of the filesystem’s block groups.
To create realistic test filesystems we have included QoS metadata support
in the Impressions benchmarking framework. The effectiveness of the
resulting data layout in terms of QoS matching is evaluated using a special
kernel module that is capable of inspecting detailed filesystem data on-the-fly.
We show that our implementations of the proposed inode and datablock
allocation strategies are capable of dramatically improving data placement
with respect to QoS requirements when compared to the default allocators
Selecting efficient and reliable preservation strategies: modeling long-term information integrity using large-scale hierarchical discrete event simulation
This article addresses the problem of formulating efficient and reliable
operational preservation policies that ensure bit-level information integrity
over long periods, and in the presence of a diverse range of real-world
technical, legal, organizational, and economic threats. We develop a
systematic, quantitative prediction framework that combines formal modeling,
discrete-event-based simulation, hierarchical modeling, and then use
empirically calibrated sensitivity analysis to identify effective strategies.
The framework offers flexibility for the modeling of a wide range of
preservation policies and threats. Since this framework is open source and
easily deployed in a cloud computing environment, it can be used to produce
analysis based on independent estimates of scenario-specific costs,
reliability, and risks.Comment: Fortcoming IDCC 202
Selecting Efficient and Reliable Preservation Strategies
This article addresses the problem of formulating efficient and reliable operational preservation policies that ensure bit-level information integrity over long periods, and in the presence of a diverse range of real-world technical, legal, organizational, and economic threats. We develop a systematic, quantitative prediction framework that combines formal modeling, discrete-event-based simulation, hierarchical modeling, and then use empirically calibrated sensitivity analysis to identify effective strategies.
Specifically, the framework formally defines an objective function for preservation that maps a set of preservation policies and a risk profile to a set of preservation costs, and an expected collection loss distribution. In this framework, a curator’s objective is to select optimal policies that minimize expected loss subject to budget constraints. To estimate preservation loss under different policy conditions optimal policies, we develop a statistical hierarchical risk model that includes four sources of risk: the storage hardware; the physical environment; the curating institution; and the global environment. We then employ a general discrete event-based simulation framework to evaluate the expected loss and the cost of employing varying preservation strategies under specific parameterization of risks.
The framework offers flexibility for the modeling of a wide range of preservation policies and threats. Since this framework is open source and easily deployed in a cloud computing environment, it can be used to produce analysis based on independent estimates of scenario-specific costs, reliability, and risks.
We present results summarizing hundreds of thousands of simulations using this framework. This exploratory analysis points to a number of robust and broadly applicable preservation strategies, provides novel insights into specific preservation tactics, and provides evidence that challenges received wisdom
Selecting Efficient and Reliable Preservation Strategies:
This article addresses the problem of formulating efficient and reliable operational preservation policies that ensure bit-level information integrity over long periods, and in the presence of a diverse range of real-world technical, legal, organizational, and economic threats. We develop a systematic, quantitative prediction framework that combines formal modelling, discrete-event-based simulation, hierarchical modelling, and then use empirically calibrated sensitivity analysis to identify effective strategies. Specifically, the framework formally defines an objective function for preservation that maps a set of preservation policies and a risk profile to a set of preservation costs, and an expected collection loss distribution. In this framework, a curator’s objective is to select optimal policies that minimize expected loss subject to budget constraints. To estimate preservation loss under different policy conditions optimal policies, we develop a statistical hierarchical risk model that includes four sources of risk: the storage hardware; the physical environment; the curating institution; and the global environment. We then employ a general discrete event-based simulation framework to evaluate the expected loss and the cost of employing varying preservation strategies under specific parameterization of risks. Source code is available at:https://github.com/MIT-Informatics/PreservationSimulation
The framework offers flexibility for the modeling of a wide range of preservation policies and threats. Since this framework is open source and easily deployed in a cloud computing environment, it can be used to produce analysis based on independent estimates of scenario-specific costs, reliability, and risk. We present results summarizing hundreds of thousands of simulations using this framework. This exploratory analysis points to a number of robust and broadly applicable preservation strategies, provides novel insights into specific preservation tactics, and provides evidence that challenges received wisdom.
An earlier version of this paper was published previously in IJDC 15(1) 202