HEC: Collaborative Research: SAM^2 Toolkit: Scalable and Adaptive Metadata Management for High-End Computing

The increasing demand for Exa-byte-scale storage capacity by high end computing applications requires a higher level of scalability and dependability than that provided by current file and storage systems. The proposal deals with file systems research for metadata management of scalable cluster-based parallel and distributed file storage systems in the HEC environment. It aims to develop a scalable and adaptive metadata management (SAM2) toolkit to extend features of and fully leverage the peak performance promised by state-of-the-art cluster-based parallel and distributed file storage systems used by the high performance computing community. There is a large body of research on data movement and management scaling, however, the need to scale up the attributes of cluster-based file systems and I/O, that is, metadata, has been underestimated. An understanding of the characteristics of metadata traffic, and an application of proper load-balancing, caching, prefetching and grouping mechanisms to perform metadata management correspondingly, will lead to a high scalability. It is anticipated that by appropriately plugging the scalable and adaptive metadata management components into the state-of-the-art cluster-based parallel and distributed file storage systems one could potentially increase the performance of applications and file systems, and help translate the promise and potential of high peak performance of such systems to real application performance improvements. The project involves the following components: 1. Develop multi-variable forecasting models to analyze and predict file metadata access patterns. 2. Develop scalable and adaptive file name mapping schemes using the duplicative Bloom filter array technique to enforce load balance and increase scalability 3. Develop decentralized, locality-aware metadata grouping schemes to facilitate the bulk metadata operations such as prefetching. 4. Develop an adaptive cache coherence protocol using a distributed shared object model for client-side and server-side metadata caching. 5. Prototype the SAM2 components into the state-of-the-art parallel virtual file system PVFS2 and a distributed storage data caching system, set up an experimental framework for a DOE CMS Tier 2 site at University of Nebraska-Lincoln and conduct benchmark, evaluation and validation studies

Content storage and retrieval mechanisms for vehicular delay-tolerant networks

Vehicular delay-tolerant networks (VDTNs) were proposed as a novel disruptive network concept based on the delay tolerant networking (DTN) paradigm. VDTN architecture uses vehicles to relay messages, enabling network connectivity in challenging scenarios. Due to intermittent connectivity, network nodes carry messages in their buffers, relaying them only when a proper contact opportunity occurs. Thus, the storage capacity and message retrieving of intermediate nodes directly affects the network performance. Therefore, efficient and robust caching and forwarding mechanisms are needed. This dissertation proposes a content storage and retrieval (CSR) solution for VDTN networks. This solution consists on storage and retrieval control labels, attached to every data bundle of aggregated network traffic. These labels define cacheable contents, and apply cachecontrol and forwarding restrictions on data bundles. The presented mechanisms gathered several contributions from cache based technologies such as Web cache schemes, ad-hoc and DTN networks. This solution is fully automated, providing a fast, safe, and reliable data transfer and storage management, while improves the applicability and performance of VDTN networks significantly. This work presents the performance evaluation and validation of CSR mechanisms through a VDTN testbed. Furthermore it presents several network performance evaluations and results using the well-known DTN routing protocols, Epidemic and Spray and Wait (including its binary variant). The comparison of the network behavior and performance on both protocols, with and without CSR mechanisms, proves that CSR mechanisms improve significantly the overall network performance