1,877 research outputs found
Alpha Entanglement Codes: Practical Erasure Codes to Archive Data in Unreliable Environments
Data centres that use consumer-grade disks drives and distributed
peer-to-peer systems are unreliable environments to archive data without enough
redundancy. Most redundancy schemes are not completely effective for providing
high availability, durability and integrity in the long-term. We propose alpha
entanglement codes, a mechanism that creates a virtual layer of highly
interconnected storage devices to propagate redundant information across a
large scale storage system. Our motivation is to design flexible and practical
erasure codes with high fault-tolerance to improve data durability and
availability even in catastrophic scenarios. By flexible and practical, we mean
code settings that can be adapted to future requirements and practical
implementations with reasonable trade-offs between security, resource usage and
performance. The codes have three parameters. Alpha increases storage overhead
linearly but increases the possible paths to recover data exponentially. Two
other parameters increase fault-tolerance even further without the need of
additional storage. As a result, an entangled storage system can provide high
availability, durability and offer additional integrity: it is more difficult
to modify data undetectably. We evaluate how several redundancy schemes perform
in unreliable environments and show that alpha entanglement codes are flexible
and practical codes. Remarkably, they excel at code locality, hence, they
reduce repair costs and become less dependent on storage locations with poor
availability. Our solution outperforms Reed-Solomon codes in many disaster
recovery scenarios.Comment: The publication has 12 pages and 13 figures. This work was partially
supported by Swiss National Science Foundation SNSF Doc.Mobility 162014, 2018
48th Annual IEEE/IFIP International Conference on Dependable Systems and
Networks (DSN
Robo-line storage: Low latency, high capacity storage systems over geographically distributed networks
Rapid advances in high performance computing are making possible more complete and accurate computer-based modeling of complex physical phenomena, such as weather front interactions, dynamics of chemical reactions, numerical aerodynamic analysis of airframes, and ocean-land-atmosphere interactions. Many of these 'grand challenge' applications are as demanding of the underlying storage system, in terms of their capacity and bandwidth requirements, as they are on the computational power of the processor. A global view of the Earth's ocean chlorophyll and land vegetation requires over 2 terabytes of raw satellite image data. In this paper, we describe our planned research program in high capacity, high bandwidth storage systems. The project has four overall goals. First, we will examine new methods for high capacity storage systems, made possible by low cost, small form factor magnetic and optical tape systems. Second, access to the storage system will be low latency and high bandwidth. To achieve this, we must interleave data transfer at all levels of the storage system, including devices, controllers, servers, and communications links. Latency will be reduced by extensive caching throughout the storage hierarchy. Third, we will provide effective management of a storage hierarchy, extending the techniques already developed for the Log Structured File System. Finally, we will construct a protototype high capacity file server, suitable for use on the National Research and Education Network (NREN). Such research must be a Cornerstone of any coherent program in high performance computing and communications
High availability using virtualization
High availability has always been one of the main problems for a data center.
Till now high availability was achieved by host per host redundancy, a highly
expensive method in terms of hardware and human costs. A new approach to the
problem can be offered by virtualization. Using virtualization, it is possible
to achieve a redundancy system for all the services running on a data center.
This new approach to high availability allows to share the running virtual
machines over the servers up and running, by exploiting the features of the
virtualization layer: start, stop and move virtual machines between physical
hosts. The system (3RC) is based on a finite state machine with hysteresis,
providing the possibility to restart each virtual machine over any physical
host, or reinstall it from scratch. A complete infrastructure has been
developed to install operating system and middleware in a few minutes. To
virtualize the main servers of a data center, a new procedure has been
developed to migrate physical to virtual hosts. The whole Grid data center
SNS-PISA is running at the moment in virtual environment under the high
availability system. As extension of the 3RC architecture, several storage
solutions have been tested to store and centralize all the virtual disks, from
NAS to SAN, to grant data safety and access from everywhere. Exploiting
virtualization and ability to automatically reinstall a host, we provide a sort
of host on-demand, where the action on a virtual machine is performed only when
a disaster occurs.Comment: PhD Thesis in Information Technology Engineering: Electronics,
Computer Science, Telecommunications, pp. 94, University of Pisa [Italy
Operating-system support for distributed multimedia
Multimedia applications place new demands upon processors, networks and operating systems. While some network designers, through ATM for example, have considered revolutionary approaches to supporting multimedia, the same cannot be said for operating systems designers. Most work is evolutionary in nature, attempting to identify additional features that can be added to existing systems to support multimedia. Here we describe the Pegasus project's attempt to build an integrated hardware and operating system environment from\ud
the ground up specifically targeted towards multimedia
- …