Tools for modelling and simulating migration-based preservation

This report describes two tools for modelling and simulating the costs and risks of using IT storage systems for the long-term archiving of file-based AV assets. The tools include a model of storage costs, the ingest and access of files, the possibility of data corruption and loss from a range of mechanisms, and the impact of having limited resources with which to fulfill access requests and preservation actions. Applications include archive planning, development of a technology strategy, cost estimation for business planning, operational decision support, staff training and generally promoting awareness of the issues and challenges archives face in digital preservation

Digital preservation strategies for AV content

The mass digitisation of analogue archive holdings plus the transition to tapeless production for new content means AV archives inevitably face the prospect of file-based archiving solutions using IT storage technology. But what is the long-term Total Cost of Ownership (TCO) of these systems, which file formats should be used, what storage technologies make sense, what are the risks involved, what is the additional cost of managing these risks, and what new software approaches can be applied? These are all issues being explored by major broadcasters, national archives and technology specialists in the European PrestoPrime project and the UK AVATAR-m project

Selecting efficient and reliable preservation strategies: modeling long-term information integrity using large-scale hierarchical discrete event simulation

This article addresses the problem of formulating efficient and reliable operational preservation policies that ensure bit-level information integrity over long periods, and in the presence of a diverse range of real-world technical, legal, organizational, and economic threats. We develop a systematic, quantitative prediction framework that combines formal modeling, discrete-event-based simulation, hierarchical modeling, and then use empirically calibrated sensitivity analysis to identify effective strategies. The framework offers flexibility for the modeling of a wide range of preservation policies and threats. Since this framework is open source and easily deployed in a cloud computing environment, it can be used to produce analysis based on independent estimates of scenario-specific costs, reliability, and risks.Comment: Fortcoming IDCC 202

Selecting Efficient and Reliable Preservation Strategies

This article addresses the problem of formulating efficient and reliable operational preservation policies that ensure bit-level information integrity over long periods, and in the presence of a diverse range of real-world technical, legal, organizational, and economic threats. We develop a systematic, quantitative prediction framework that combines formal modeling, discrete-event-based simulation, hierarchical modeling, and then use empirically calibrated sensitivity analysis to identify effective strategies. Specifically, the framework formally defines an objective function for preservation that maps a set of preservation policies and a risk profile to a set of preservation costs, and an expected collection loss distribution. In this framework, a curator’s objective is to select optimal policies that minimize expected loss subject to budget constraints. To estimate preservation loss under different policy conditions optimal policies, we develop a statistical hierarchical risk model that includes four sources of risk: the storage hardware; the physical environment; the curating institution; and the global environment. We then employ a general discrete event-based simulation framework to evaluate the expected loss and the cost of employing varying preservation strategies under specific parameterization of risks. The framework offers flexibility for the modeling of a wide range of preservation policies and threats. Since this framework is open source and easily deployed in a cloud computing environment, it can be used to produce analysis based on independent estimates of scenario-specific costs, reliability, and risks. We present results summarizing hundreds of thousands of simulations using this framework. This exploratory analysis points to a number of robust and broadly applicable preservation strategies, provides novel insights into specific preservation tactics, and provides evidence that challenges received wisdom

Architectural Techniques to Enable Reliable and Scalable Memory Systems

High capacity and scalable memory systems play a vital role in enabling our desktops, smartphones, and pervasive technologies like Internet of Things (IoT). Unfortunately, memory systems are becoming increasingly prone to faults. This is because we rely on technology scaling to improve memory density, and at small feature sizes, memory cells tend to break easily. Today, memory reliability is seen as the key impediment towards using high-density devices, adopting new technologies, and even building the next Exascale supercomputer. To ensure even a bare-minimum level of reliability, present-day solutions tend to have high performance, power and area overheads. Ideally, we would like memory systems to remain robust, scalable, and implementable while keeping the overheads to a minimum. This dissertation describes how simple cross-layer architectural techniques can provide orders of magnitude higher reliability and enable seamless scalability for memory systems while incurring negligible overheads.Comment: PhD thesis, Georgia Institute of Technology (May 2017

CloudSkulk: Design of a Nested Virtual Machine Based Rootkit-in-the-Middle Attack

Virtualized cloud computing services are a crucial facet in the software industry today, with clear evidence of its usage quickly accelerating. Market research forecasts an increase in cloud workloads by more than triple, 3.3-fold, from 2014 to 2019 [33]. Integrating system security is then an intrinsic concern of cloud platform system administrators that with the growth of cloud usage, is becoming increasingly relevant. People working in the cloud demand security more than ever. In this paper, we take an offensive, malicious approach at targeting such cloud environments as we hope both cloud platform system administrators and software developers of these infrastructures can advance their system securities. A vulnerability could exist in any layer of a computer system. It is commonly believed in the security community that the battle between attackers and defenders is determined by which side can exploit these vulnerabilities and then gain control at the lower layer of a system [22]. Because of this perception, kernel level defense is proposed to defend against user-level malware [25], hypervisor-level defense is proposed to detect kernel-level malware or rootkits [36, 47, 41], hardware-level defense is proposed to defend or protect hypervisors [4, 51, 45]. Once attackers find a way to exploit a particular vulnerability and obtain a certain level of control over the victim system, retaining that control and avoiding detection becomes their top priority. To achieve this goal, various rootkits have been developed. However, existing rootkits have a common weakness: they are still detectable as long as defenders can gain control at a lower-level, such as the operating system level, the hypervisor level, or the hardware level. In this paper, we present a new type of rootkit called CloudSkulk, which is a nested virtual machine (VM) based rootkit. While nested virtualization has attracted sufficient attention from the security and cloud community, to the best of our knowledge, we are the first to reveal and demonstrate nested virtualization can be used by attackers for developing malicious rootkits. By impersonating the original hypervisor to communicate with the original guest operating system (OS) and impersonating the original guest OS to communicate with the hypervisor, CloudSkulk is hard to detect, regardless of whether defenders are at the lower-level (e.g., in the original hypervisor) or at the higher-level (e.g., in the original guest OS). We perform a variety of performance experiments to evaluate how stealthy the proposed rootkit is at remaining unnoticed as introducing one more layer of virtualization inevitably incurs extra overhead. Our performance characterization data shows that an installation of our novel rootkit on a targeted nested virtualization environment is likely to remain undetected unless the guest user performs IO intensive-type workloads

RELIABILITY MODEL AND ASSESSMENT OF REDUNDANT ARRAYS OF INEXPENSIVE DISKS (RAID) INCORPORATING LATENT DEFECTS AND NON-HOMOGENEOUS POISSON PROCESS EVENTS.

Today's most reliable data storage systems are made of redundant arrays of inexpensive disks (RAID). The quantification of RAID system reliability is often based on models that omit critical hard disk drive failure modes, assume all failure and restoration rates are constant (exponential distributions), and assume the RAID group times to failure follow a homogeneous Poisson process (HPP). This paper presents a comprehensive reliability model that accounts for numerous failure causes for today's hard disk drives, allows proper representation of repair and restoration, and does not rely on the assumption of a HPP for the RAID group. The model does not assume hard disk drives have constant transition rates, but allows each hard disk drive "slot" in the RAID group to have its own set of distributions, closed form or user defined. Hard disk drive (HDD) failure distributions derived from field usage are presented, showing that failure distributions are commonly non-homogeneous, frequently having increasing hazard rates from time zero. Hard disks drive failure modes and causes are presented and used to develop a model that reflects not only complete failure, but also degraded conditions due to undetected, but corrupted data (latent defects). The model can represent user defined distributions for completion of "background scrubbing" to correct (remove) corrupted data. Sequential Monte Carlo simulation is used to determine the number of double disk failures expected as a function of time. RAID group can be any size up to 25. The results are presented as mean cumulative failure distributions for the RAID group. Results estimate the number of double disk failures can be as much as 5000 times greater than that predicted over 10 years when using the mean time to data loss method or Markov models when the characteristic lives of the input distributions is the same. Model results are compared to actual field data for two HDD families and two different RAID group sizes and show good correlation. Results show the rate of occurrence of failure for the RAID group may be increasing, decreasing or constant depending on the parameters used for the four input distributions