NFSv4 and High Performance File Systems: Positioning to Scale

The avant-garde of high performance computing is building petabyte storage systems. At CITI, we are investigating the use of NFSv4 as a standard for fast and secure access to this data, both across a WAN and within a (potentially massive) cluster. An NFSv4 server manages much state information, which hampers exporting objects via multiple servers and allows the NFSv4 server to become a bottleneck as load increases. This paper introduces Parallel NFSv4, extending the NFSv4 protocol with a new server-to-server protocol and a new file description and location mechanism for increased scalability.http://deepblue.lib.umich.edu/bitstream/2027.42/107881/1/citi-tr-04-2.pd

Direct-pNFS: Scalable, transparent, and versatile access to parallel file systems

Grid computations require global access to massive data stores. To meet this need, the GridNFS project aims to provide scalable, high-performance, transparent, and secure wide-area data management as well as a scalable and agile name space. While parallel file systems give high I/O throughput, they are highly specialized, have limited operating system and hardware platform support, and often lack strong security mechanisms. Remote data access tools such as NFS and GridFTP overcome some of these limitations, but fail to provide universal, transparent, and scalable remote data access. As part of GridNFS, this paper introduces Direct-pNFS, which builds on the NFSv4.1 protocol to meet a key challenge in accessing remote parallel file systems: high-performance and scalable data access without sacrificing transparency, security, orportability. Experiments with Direct-pNFS demonstrate I/O throughput that equals or out performs the exported parallel file system across a range of workloads.http://deepblue.lib.umich.edu/bitstream/2027.42/107917/1/citi-tr-07-2.pd

Benchmarking of IP-based Network Storage Systems

Mobile platforms with access to high speed wireless network have become ubiquitous. Advancements in network technology and consumer electronics have brought traditional storage systems into offices and homes. Services based on cloud technologies, including object based storage, have gained popularity among both private users and enterprises. However, there is still a lack of systematic evaluation of both traditional storage systems and cloud based object storage in a mobile and wireless context. In this thesis, we evaluate the performance of three drastically different storage systems, namely NFS, iSCSI, and OpenStack Swift, which can potentially be used by mobile platforms over wireless network. We build a testbed and an in house, ad hoc microbenchmark to study the impact of various network complexities and different access behaviours of application. In addition, we employ two widely used macrobenchmarks -- PostMark and FileBench -- to simulate the workloads of typical applications. We find that: (1) iSCSI excels in networks whose condition is as good as LAN; (2) NFS and Swift are more suitable for complex networks such as wireless network and WAN; (3) Swift is a viable replacement for NFS in all scenarios; and (4) System configuration on the client side impacts storage performance significantly and deserve adequate attention. Furthermore, we make several recommendations to practitioners and point out numerous future research directions

Optimización de clusters como plataformas multimedia utilizando clientes predictivios multihilo

En esta tesis doctoral se pretende hacer un estudio de los problemas principales de los clusters de computadoras utilizando aplicaciones multimedia. Enfrentamos las dos partes, arquitectura y aplicaciones y, además, lo hacemos de una forma "real", sin simulaciones. Se va a partir de una plataforma real, formada por un cluster de alta velocidad, y de una aplicación multimedia totalmente flexible como para emular el patrón de tráfico de otras. Esta aplicación va a inyectar la carga multimedia al cluster y va a exigir unos requisitos particulares. El gran reto actual de los cluster es la entrada/salida de alto rendimiento, ya que la computación de alto rendimiento está conseguida. Expondremos los principales problemas y daremos una solución que mejore los resultados obtenidos inicialmente. En entrada/salida existen numerosos e interesantes trabajos. Todos han intentado aportar alguna novedad, con el propósito de mejorar algún punto negro de la entrada/salida. En nuestro trabajo se ha hecho un estudio exhaustivo de todos esos trabajos con el fin de plantear un nuevo método híbrido de adelantamiento de datos para arquitecturas cliente-servidor, en uno de los sistemas de ficheros en red más utilizado actualmente, NFS (Network Fle System) . Pero no solo va a ser un planteamiento algorítmico y teórico, sino que se va a implementar en el mismo núcleo del sistema operativo, donde NFS aparece como modulo, y se realizaran los experimentos para confirmar las mejoras de la implementación desarrollada. Inicialmente, nos planteamos la posibilidad de cambiar la forma de trabajar del servidor, pero después realizamos el traslado al cliente, mucho más manejable y abierto a mejoras de este tipo. Planteamos el diseño de la técnica hibrida de prefetching, una técnica basada en grafos de acceso, con el propósito de adelantar datos no solo del fichero actual que se está leyendo (que eso ya se hacía) sino, también, a través de distintos ficheros. Presentamos también los resultados con este nuevo cliente predictivo y obtenemos una importante reducción en los tiempos de lectura, y unos valores importantes en la ganancia conseguida. Por lo tanto, queda demostrada la utilidad de técnicas e este tipo para el sistema NFS

Cooperative Data Backup for Mobile Devices

Les dispositifs informatiques mobiles tels que les ordinateurs portables, assistants personnels et téléphones portables sont de plus en plus utilisés. Cependant, bien qu'ils soient utilisés dans des contextes où ils sont sujets à des endommagements, à la perte, voire au vol, peu de mécanismes permettent d'éviter la perte des données qui y sont stockées. Dans cette thèse, nous proposons un service de sauvegarde de données coopératif pour répondre à ce problème. Cette approche tire parti de communications spontanées entre de tels dispositifs, chaque dispositif stockant une partie des données des dispositifs rencontrés. Une étude analytique des gains de cette approche en termes de sûreté de fonctionnement est proposée. Nous étudions également des mécanismes de stockage réparti adaptés. Les problèmes de coopération entre individus mutuellement suspicieux sont également abordés. Enfin, nous décrivons notre mise en oeuvre du service de sauvegarde coopérative. ABSTRACT : Mobile devices such as laptops, PDAs and cell phones are increasingly relied on but are used in contexts that put them at risk of physical damage, loss or theft. However, few mechanisms are available to reduce the risk of losing the data stored on these devices. In this dissertation, we try to address this concern by designing a cooperative backup service for mobile devices. The service leverages encounters and spontaneous interactions among participating devices, such that each device stores data on behalf of other devices. We first provide an analytical evaluation of the dependability gains of the proposed service. Distributed storage mechanisms are explored and evaluated. Security concerns arising from thecooperation among mutually suspicious principals are identified, and core mechanisms are proposed to allow them to be addressed. Finally, we present our prototype implementation of the cooperative backup servic