Distributed paged Hash tables

In this paper we present the design and implementation of DPH, a storage layer for cluster environments. DPH is a Distributed Data Structure (DDS) based on the distribution of a paged hash table. It combines main memory with file system resources across the cluster in order to implement a distributed dictionary that can be used for the storage of very large data sets with key based addressing techniques. The DPH storage layer is supported by a collection of cluster-aware utilities and services. Access to the DPH interface is provided by a user-level API. A preliminary performance evaluation shows promising results.Supported by PRODEP III (grant 5.3/N/199.006/00) and SAPIENS (grant 41739/CHS/2001

Hare: a file system for non-cache-coherent multicores

Hare is a new file system that provides a POSIX-like interface on multicore processors without cache coherence. Hare allows applications on different cores to share files, directories, and file descriptors. The challenge in designing Hare is to support the shared abstractions faithfully enough to run applications that run on traditional shared-memory operating systems, with few modifications, and to do so while scaling with an increasing number of cores. To achieve this goal, Hare must support features (such as shared file descriptors) that traditional network file systems don't support, as well as implement them in a way that scales (e.g., shard a directory across servers to allow concurrent operations in that directory). Hare achieves this goal through a combination of new protocols (including a 3-phase commit protocol to implement directory operations correctly and scalably) and leveraging properties of non-cache-coherent multiprocessors (e.g., atomic low-latency message delivery and shared DRAM). An evaluation on a 40-core machine demonstrates that Hare can run many challenging Linux applications (including a mail server and a Linux kernel build) with minimal or no modifications. The results also show these applications achieve good scalability on Hare, and that Hare's techniques are important to achieving scalability.Quanta Computer (Firm

A Persistent Storage Model for Extreme Computing

The continuing technological progress resulted in a dramatic growth in aggregate computational performance of the largest supercomputing systems. Unfortunately, these advances did not translate to the required extent into accompanying I/O systems and little more in terms of architecture or effective access latency. New classes of algorithms developed for massively parallel applications, that gracefully handle the challenges of asynchrony, heavily multi-threaded distributed codes, and message-driven computation, must be matched by similar advances in I/O methods and algorithms to produce a well performing and balanced supercomputing system. This dissertation proposes PXFS, a storage model for persistent objects inspired by the ParalleX model of execution that addresses many of these challenges. The PXFS model is designed to be asynchronous in nature to comply with ParalleX model and proposes an active TupleSpace concept to hold all kinds of metadata/meta-object for either storage objects or runtime objects. The new active TupleSpace can also register ParalleX actions to be triggered under certain tuple operations. An first implementation of PXFS utilizing a well-known Orange parallel file system as its back-end via asynchronous I/O layer and the implementation of TupleSpace component in HPX, the implementation of ParalleX. These details are also described along with the preliminary performance data. A house-made micro benchmark is developed to measure the disk I/O throughput of the PXFS asynchronous interface. The results show perfect scalability and 3x to 20x times speedup of I/O throughput performance comparing to OrangeFS synchronous user interface. Use cases of TupleSpace components are discussed for real-world applications including micro check-pointing. By utilizing TupleSpace in HPX applications for I/O, global barrier can be replaced with fine-grained parallelism to overlap more computation with communication and greatly boost the performance and efficiency. Also the dissertation showcases the distributed directory service in Orange file system which process directory entries in parallel and effectively improves the directory metada operations

A 64-bit, Shared Disk File System for Linux

In computer systems today, speed and responsiveness is often determined by network and storage subsystem performance. Faster, more scalable networking interfaces like Fibre Channel and Gigabit Ethernet provide the scaffolding from which higher performance implementations may be constructed, but new thinking is required about how machines interact with network-enabled storage devices. We have developed a Linux file system called GFS (the Global File System) that allows multiple Linux machines to access and share disk and tape devices on a Fibre Channel or SCSI storage network. We plan to extend GFS by transporting packetized SCSI commands over IP so that any GFS-enabled Linux machine can access shared network devices. GFS will perform well as a local file system, as a traditional network file system running over IP, and as a high-performance cluster file system running over storage networks like Fibre Channel. GFS device sharing provides a key cluster-enabling technology for Linux, helping to bring the availability, scalability, and load balancing benefits of clustering to Linux

The global intelligent file system framework.

"Since its inception the Internet has grown rapidly in both size and importance in our everyday lives. The Internet today is the preliminary model of what is commonly called the global information infrastructure. However, at the moment this "infrastructure" is considered to be an addition to our computer, and is not an integrated part of a file system which is essentially a "local information infrastructure" of a computer. Advancements in the sizes of disks in computers, network bandwidth and the types of media available mean users now keep large amounts of files in their personal data storage spaces, with little or no additional support for the organisation, searching or sharing of this data. The hierarchical model of file system storage is no longer the most effective way of organising and categorising files and information. Relying largely on the user, rather than the computer, being efficient and organised its inflexible nature renders it unsuitable for the meaningful coordination of an increasing bulk of divergent file types that users deal with on a daily basis. The work presented in this thesis describes a new paradigm for file storage, management and retrieval. Providing globally integrated document emplacement and administration, the GIFS (Global Intelligent File System) framework offers the necessary architecture for transparently directing the storage, access, sharing, manipulation, and security of files across interconnected computers. To address the discrepancy between user actions and computer actions, GIFS provides each user with a "Virtual Secretary" to reduce the cognitive workload and remove the time-consuming task of information organisation from the user. The Secretary is supported by a knowledge base and a collection of intelligent agents, which are programs that manage and process the data collected, and work behind the scenes aiding gradual proliferation of knowledge. The Virtual Secretary is responsible for providing fast and accurate assistance to aid users who wish to create, store, retrieve, share, secure and collaborate on their files. Through both system prototyping and performance simulation it is demonstrated that it is desirable as well as feasible to deploy a knowledge base in supporting an intelligent user interface that acts like a human assistant who handles paperwork, looks after filing, security and so on. This work provides the contribution of a new framework and architecture to the field of files systems and document management as well as focusing on reducing the burden placed upon users through everyday usage of computer systems. Such a framework has the potential to be evolved into a highly intelligent assistant to a user over a period of service and the introduction of additional agents, and provides the basis for advancements in file system and organisational technologies.

Arquitectura multiagente para E/S de alto rendimiento en clusters

La E/S constituye en la actualidad uno de los principales cuellos de botella de los sistemas distribuidos de propósito general, debido al desequilibrio existente entre el tiempo de cómputo y de E/S. Una de las soluciones propuestas para este problema ha sido el uso de la E/S paralela. En esta área, se han originado un gran número de bibliotecas de E/S paralela y sistemas de ficheros paralelos. Este tipo de sistemas adolecen de algunos defectos y carencias. Muchos de ellos están concebidos para máquinas paralelas y no se integran adecuadamente en entornos distribuidos y clusters. El uso intensivo de clusters de estaciones de trabajo durante estos últimos años hace que este tipo de sistemas no sean adecuados en el escenario de computación actual. Otros sistemas, que se adaptan a este tipo de entornos, no incluyen capacidades de reconfiguración dinámica, por lo que tienen una funcionalidad limitada. Por último, la mayoría de los sistemas de E/S que utilizan diferentes optimizaciones de E/S, no ofrecen flexibilidad a las aplicaciones para hacer uso de las mismas, intentando ocultar al usuario este tipo de técnicas. No obstante, a fin de optimizar las operaciones de E/S, es importante que las aplicaciones sean capaces de describir sus patrones de acceso, interactuando con el sistema de E/S. En otro ámbito, dentro del área de los sistemas distribuidos se encuentra el paradigma de agentes, que permite dotar a las aplicaciones de un conjunto de propiedades muy adecuadas para su adaptación a entornos complejos y dinámicos. Las características de este paradigma lo hacen a priori prometedor para abordar algunos de los problemas existentes en el campo de la E/S paralela. Esta tesis propone una solución a la problemática actual de E/S a través de tres líneas principales: (i) el uso de la teoría de agentes en sistemas de E/S de alto rendimiento, (ii) la definición de un formalismo que permita la reconfiguración dinámica de nodos de almacenamiento en un cluster y (iii) el uso de técnicas de optimización de E/S configurables y orientadas a las aplicaciones

Co-operação de Tabelas de Hash Distribuídas em clusters heterogéneos

Tese de Doutoramento em Informática - Área de Conhecimento de Engenharia de ComputadoresAs Estruturas de Dados Distribuídas (DDSs) constituem uma abordagem ao Armazenamento Distribuído adequada a aplicações com requisitos de elevada capacidade de armazenamento, escalabilidade e disponibilidade. Ao mesmo tempo, a disponibilização de interfaces simples e familiares permite encurtar o ciclo de desenvolvimento de aplicações. Num ambiente de cluster, caracterizado pela coexistência de grande variedade de aplicações e utilizadores com requisitos dinâmicos e de gerações de equipamento com desempenhos diferenciados (tanto ao nível do poder de calculo, como das comunicações e do armazenamento), a necessidade de operação conjunta e eficiente de múltiplas DDSs pelas aplicações constituiu um desafio científico e tecnológico que esta tese se propôs superar. Esta tese apresenta as propostas e os resultados alcançados durante o estudo e desenvolvimento dos modelos e da plataforma de suporte à arquitectura Domus para a co-operação de múltiplas Tabelas de Hash Distribuídas (DHTs) em clusters partilhados e heterogéneos. A plataforma contempla várias classes de atributos e operações sobre DHTs permitindo, entre outras possibilidades, a definição administrativa de limite à expansão/contracção, o recurso a diferentes tecnologias de armazenamento e a suspensão/retoma dinâmica da operação. Numa outra vertente, para responder aos requisitos impostos pela execução simultânea de múltiplas e diferenciadas aplicações, foram formulados e desenvolvidos mecanismos de balanceamento dinâmico de carga que visam a optimização e rentabilização dos recursos computacionais/comunicacionais e de armazenamento disponíveis no cluster. Na base da abordagem estão modelos que dão uma resposta qualificada aos problemas do particionamento de uma DHT e da localização das suas entradas. Em relação ao particionamento, são propostos modelos para definição do número de entradas de cada nó de uma DHT, suportando variação dinâmica do número total de nós; os modelos asseguram uma distribuição óptima do número de entradas, nos quatro cenários que resultam da combinação de Hashing Estático ou Dinâmico, com Distribuição Homogénea ou Heterogénea; com Hashing Dinâmico, a qualidade da distribuição é parametrizável e, com Distribuição Homogénea, o esforço de re-particionamento ´e de ordem O(1) face ao total de nós da DHT. No que toca à localização, definiram-se algoritmos de encaminhamento acelerado para localização distribuída em topologias DeBruijn e Chord, que respondem `a descontinuidade das partições das DHTs geradas pelos modelos de particionamento. Para o efeito, explorou-se a possível coexistência em cada nó de uma DHT de múltiplas tabelas de encaminhamento, para tentar aproximar do custo, de referência, sobre grafos esparsos em que os vértices são nós da DHT, ao custo da localização sobre grafos completos, em que os vértices são entradas da DHT, tendo-se obtido valores de 70% a 90% do custo de referência.Distributed Data Structures (DDSs) as an approach to Distributed Storage are adequate to applications that require high storage capacity, scalability and availability. At the same time, DDSs present simple and familiar interfaces that allow shorter development cycles. In a cluster environment, shared by multiple users and applications with dynamic requisites, and built on different hardware generations (with different computing, communication and storage power), the need to simultaneously and efficiently operate several DDSs by user applications presents the scientific and technological challenge embraced by this thesis. This thesis presents the proposals and the results achieved during the study of the models and development of the platform that supports the Domus architecture for the co-operation of multiple Distributed Hash Tables (DHTs) in shared and heterogeneous clusters. The platform supports several classes of attributes and operations on DHTs allowing, among other possibilities, the administrative definition of per DHT node boundaries and storage technologies, or the deactivation/reactivation of DHTs. In addition, Domus DHTs are managed by dynamic load balancing mechanisms which ensure certain service levels (like storage space or access performance) and optimize the utilization of cluster resources. The architecture builds on specific models for the partitioning and lookup of a DHT address space. The partitioning models ensure optimal distribution of the number of DHT buckets per node, for a dynamic number of DHT nodes, and support the different scenarios that arise from the combination of Static/Dynamic Hashing, with Homogeneous/Heterogeneous Distributions. The quality of the distribution is tuneable under Dynamic Hashing, and repartitioning involves O(1) nodes of the DHT under Homogeneous Distributions. With regard to the lookup problem, a set of accelerated routing algorithms were designed for distributed lookup, both for DeBruijn and Chord topologies, in order to deal with the discontinuous DHT partitions generated by our partitioning models. The algorithms explore the availability of multiple routing tables per DHT node with the aim of reducing the lookup cost with full graphs, in which the vertexes are all the DHT buckets, towards the reference lookup cost with sparse graphs, in which the vertexes are all the DHT nodes. The best lookup algorithms ensure a lookup cost of 70% to 90% of the reference cost