CRAID: Online RAID upgrades using dynamic hot data reorganization

Current algorithms used to upgrade RAID arrays typically require large amounts of data to be migrated, even those that move only the minimum amount of data required to keep a balanced data load. This paper presents CRAID, a self-optimizing RAID array that performs an online block reorganization of frequently used, long-term accessed data in order to reduce this migration even further. To achieve this objective, CRAID tracks frequently used, long-term data blocks and copies them to a dedicated partition spread across all the disks in the array. When new disks are added, CRAID only needs to extend this process to the new devices to redistribute this partition, thus greatly reducing the overhead of the upgrade process. In addition, the reorganized access patterns within this partition improve the array’s performance, amortizing the copy overhead and allowing CRAID to offer a performance competitive with traditional RAIDs. We describe CRAID’s motivation and design and we evaluate it by replaying seven real-world workloads including a file server, a web server and a user share. Our experiments show that CRAID can successfully detect hot data variations and begin using new disks as soon as they are added to the array. Also, the usage of a dedicated partition improves the sequentiality of relevant data access, which amortizes the cost of reorganizations. Finally, we prove that a full-HDD CRAID array with a small distributed partition (<1.28% per disk) can compete in performance with an ideally restriped RAID-5 and a hybrid RAID-5 with a small SSD cache.Peer ReviewedPostprint (published version

An effective data placement strategy for XML documents

As XML is increasingly being used in Web applications, new technologies need to be investigated for processing XML documents with high performance. Parallelism is a promising solution for structured document processing and data placement is a major factor for system performance improvement in parallel processing. This paper describes an effective XML document data placement strategy. The new strategy is based on a multilevel graph partitioning algorithm with the consideration of the unique features of XML documents and query distributions. A new algorithm, which is based on XML query schemas to derive the weighted graph from the labelled directed graph presentation of XML documents, is also proposed. Performance analysis on the algorithm presented in the paper shows that the new data placement strategy exhibits low workload skew and a high degree of parallelism

Data partitioning and load balancing in parallel disk systems

Parallel disk systems provide opportunities for exploiting I/O parallelism in two possible ways, namely via inter-request and intra-request parallelism. In this paper we discuss the main issues in performance tuning of such systems, namely striping and load balancing, and show their relationship to response time and throughput. We outline the main components of an intelligent file system that optimizes striping by taking into account the requirements of the applications, and performs load balancing by judicious file allocation and dynamic redistributions of the data when access patterns change. Our system uses simple but effective heuristics that incur only little overhead. We present performance experiments based on synthetic workloads and real-life traces

Micro-Macro relations for flow through random arrays of cylinders

The transverse permeability for creeping flow through unidirectional random arrays of fibers with various structures is revisited theoretically and numerically using the finite element method (FEM). The microstructure at various porosities has a strong effect on the transport properties, like permeability, of fibrous materials. We compare different microstructures (due to four random generator algorithms) as well as the effect of boundary conditions, finite size, homogeneity and isotropy of the structure on the macroscopic permeability of the fibrous medium. Permeability data for different minimal distances collapse when their minimal value is subtracted, which yields an empirical macroscopic permeability master function of porosity. Furthermore, as main result, a microstructural model is developed based on the lubrication effect in the narrow channels between neighboring fibers. The numerical experiments suggest a unique, scaling power law relationship between the permeability obtained from fluid flow simulations and the mean value of the shortest Delaunay triangulation edges (constructed using the centers of the fibers), which is identical to the averaged second nearest neighbor fiber distances. This universal lubrication relation, as valid in a wide range of porosities, accounts for the microstructure, e.g. hexagonally ordered or disordered fibrous media. It is complemented by a closure relation that relates the effective microscopic length to the packing fraction