HEC: Collaborative Research: SAM^2 Toolkit: Scalable and Adaptive Metadata Management for High-End Computing

The increasing demand for Exa-byte-scale storage capacity by high end computing applications requires a higher level of scalability and dependability than that provided by current file and storage systems. The proposal deals with file systems research for metadata management of scalable cluster-based parallel and distributed file storage systems in the HEC environment. It aims to develop a scalable and adaptive metadata management (SAM2) toolkit to extend features of and fully leverage the peak performance promised by state-of-the-art cluster-based parallel and distributed file storage systems used by the high performance computing community. There is a large body of research on data movement and management scaling, however, the need to scale up the attributes of cluster-based file systems and I/O, that is, metadata, has been underestimated. An understanding of the characteristics of metadata traffic, and an application of proper load-balancing, caching, prefetching and grouping mechanisms to perform metadata management correspondingly, will lead to a high scalability. It is anticipated that by appropriately plugging the scalable and adaptive metadata management components into the state-of-the-art cluster-based parallel and distributed file storage systems one could potentially increase the performance of applications and file systems, and help translate the promise and potential of high peak performance of such systems to real application performance improvements. The project involves the following components: 1. Develop multi-variable forecasting models to analyze and predict file metadata access patterns. 2. Develop scalable and adaptive file name mapping schemes using the duplicative Bloom filter array technique to enforce load balance and increase scalability 3. Develop decentralized, locality-aware metadata grouping schemes to facilitate the bulk metadata operations such as prefetching. 4. Develop an adaptive cache coherence protocol using a distributed shared object model for client-side and server-side metadata caching. 5. Prototype the SAM2 components into the state-of-the-art parallel virtual file system PVFS2 and a distributed storage data caching system, set up an experimental framework for a DOE CMS Tier 2 site at University of Nebraska-Lincoln and conduct benchmark, evaluation and validation studies

Improving Energy Effeciency and Reliability of Disk Storage Systems

Numerous energy saving techniques have been developed to aggressively reduce energy dissipation in parallel disks. However, many existing energy conservation schemes have substantial adverse impacts on disk reliability. To remedy this deficiency, in this paper we address the problem of making tradeoffs between energy efficiency and reliability in parallel disk systems. Among several factors affecting disk reliability, the two most important factors - disk utilization and ages - are the focus of this study. We built a mathematical reliability model to quantify the impacts of disk age and utilization on failure probabilities of mirrored disk systems. In light of the reliability model, we proposed a novel concept of safe utilization zone, within which energy dissipation in disks can be reduced without degrading reliability. We developed two approaches to improving both reliability and energy efficiency of disk systems through disk mirroring and utilization control, enforcing disk drives to be operated in safe utilization zones. Our utilization-based control schemes seamlessly integrate reliability with energy saving techniques in the context of fault-tolerant systems. Experimental results show that our approaches can significantly improve reliable while achieving high-energy efficiency for disk systems under a wide range of workload situations

Energy efficiency of error correction on wireless systems

Since high error rates are inevitable to the wireless environment, energy-efficient error-control is an important issue for mobile computing systems. We have studied the energy efficiency of two different error correction mechanisms and have measured the efficiency of an implementation in software. We show that it is not sufficient to concentrate on the energy efficiency of error control mechanisms only, but the required extra energy consumed by the wireless interface should be incorporated as well. A model is presented that can be used to determine an energy-efficient error correction scheme of a minimal system consisting of a general purpose processor and a wireless interface. As an example we have determined these error correction parameters on two systems with a WaveLAN interfac