Using TCP/IP traffic shaping to achieve iSCSI service predictability

This thesis reproduces the properties of load interference common in many storage devices using resource sharing for flexibility and maximum hardware utilization. The nature of resource sharing and load is studied and compared to assumptions and models used in previous work. The results are used to design a method for throttling iSCSI initiators, attached to an iSCSI target server, using a packet delay module in Linux Traffic Control. The packet delay throttle enables close-to-linear rate reduction for both read and write operations. Iptables and Ipset are used to add dynamic packet matching needed for rapidly changing throttling values. All throttling is achieved without triggering TCP retransmit timeout and subsequent slow start caused by packet loss. A control mechanism for dynamically adapting throttling values to rapidly changing workloads is implemented using a modified proportional integral derivative (PID) controller. Using experiments, control engineering filtering techniques and results from previous research, a suitable per resource saturation indicator was found. The indicator is an exponential moving average of the wait time of active resource consumers. It is used as input value to the PID controller managing the packet rates of resource consumers, creating a closed control loop managed by the PID controller. Finally a prototype of an autonomic resource prioritization framework is designed. The framework identifies and maintains information about resources, their consumers, their average wait time for active consumers and their set of throttleable consumers. The information is kept in shared memory and a PID controller is spawned for each resource, thus safeguarding read response times by throttling writers on a per-resource basis. The framework is exposed to extreme workload changes and demonstrates high ability to keep read response time below a predefined threshold. Using moderate tuning efforts the framework exhibits low overhead and resource consumption, promising suitability for large scale operation in production environments

Design and implementation of an object storage system

Master'sMASTER OF ENGINEERIN

Service-oriented models for audiovisual content storage

What are the important topics to understand if involved with storage services to hold digital audiovisual content? This report takes a look at how content is created and moves into and out of storage; the storage service value networks and architectures found now and expected in the future; what sort of data transfer is expected to and from an audiovisual archive; what transfer protocols to use; and a summary of security and interface issues

Virtualization services: scalable methods for virtualizing multicore systems

Multi-core technology is bringing parallel processing capabilities from servers to laptops and even handheld devices. At the same time, platform support for system virtualization is making it easier to consolidate server and client resources, when and as needed by applications. This consolidation is achieved by dynamically mapping the virtual machines on which applications run to underlying physical machines and their processing cores. Low cost processor and I/O virtualization methods efficiently scaled to different numbers of processing cores and I/O devices are key enablers of such consolidation. This dissertation develops and evaluates new methods for scaling virtualization functionality to multi-core and future many-core systems. Specifically, it re-architects virtualization functionality to improve scalability and better exploit multi-core system resources. Results from this work include a self-virtualized I/O abstraction, which virtualizes I/O so as to flexibly use different platforms' processing and I/O resources. Flexibility affords improved performance and resource usage and most importantly, better scalability than that offered by current I/O virtualization solutions. Further, by describing system virtualization as a service provided to virtual machines and the underlying computing platform, this service can be enhanced to provide new and innovative functionality. For example, a virtual device may provide obfuscated data to guest operating systems to maintain data privacy; it could mask differences in device APIs or properties to deal with heterogeneous underlying resources; or it could control access to data based on the ``trust' properties of the guest VM. This thesis demonstrates that extended virtualization services are superior to existing operating system or user-level implementations of such functionality, for multiple reasons. First, this solution technique makes more efficient use of key performance-limiting resource in multi-core systems, which are memory and I/O bandwidth. Second, this solution technique better exploits the parallelism inherent in multi-core architectures and exhibits good scalability properties, in part because at the hypervisor level, there is greater control in precisely which and how resources are used to realize extended virtualization services. Improved control over resource usage makes it possible to provide value-added functionalities for both guest VMs and the platform. Specific instances of virtualization services described in this thesis are the network virtualization service that exploits heterogeneous processing cores, a storage virtualization service that provides location transparent access to block devices by extending the functionality provided by network virtualization service, a multimedia virtualization service that allows efficient media device sharing based on semantic information, and an object-based storage service with enhanced access control.Ph.D.Committee Chair: Schwan, Karsten; Committee Member: Ahamad, Mustaq; Committee Member: Fujimoto, Richard; Committee Member: Gavrilovska, Ada; Committee Member: Owen, Henry; Committee Member: Xenidis, Jim

Evaluation and Deployment of a Private Cloud Framework at DI-FCT-NOVA

In today’s technological landscape, there is an ever-increasing demand for computing resources for simulations, machine learning, or other use-cases. This demand can be seen across the business world, with the success of Amazon’s AWS and Microsoft’s Azure offer- ings, which provide a cloud of on-demand computing resources to any paying customer. The necessity for computing resources is no less felt in the academic world, where departments are forced to assign researchers and teachers to time-consuming system administrator roles, to allocate resources to users, leading to delays and wasted potential. Allowing researchers to request computing resources and then get them, on-demand, with minimal input from any administrative staff, is a great boon. Not only does it increase productivity of the administrators themselves, but it also allows users (teachers, researchers and students) to get the resources they need faster, and more securely. This goal is attainable through the use of a cloud management framework to assist in the administration of a department’s computing infrastructure. This dissertation aims to provide a critical evaluation on the adequacy of three cloud management frameworks, evaluating the requirements for a private cloud at the DI- FCT-NOVA, as well as which features of the selected cloud framework may be used in the fulfilment of the department’s needs. The final goal is to architect and deploy the selected framework to DI-FCT-NOVA, which will give the department a maintainable state-of-the-art private cloud deployment, capable of adequately responding to the needs of its users.No cenário tecnológico atual, existe uma necessidade crescente por recursos computaci- onais quer para simulações, aprendizagem automática, ou outros fins. Essa necessidade pode ser vista no mundo dos negócios, traduzindo-se no sucesso da Amazon AWS e a da Microsoft Azure, entre outras, que oferecem clouds de recursos computacionais a qualquer cliente, sujeito a diferentes formas de pagamento. A necessidade de recursos computacionais não é menos sentida no mundo académico, onde departamentos são forçados a atribuir a investigadores e professores tarefas onerosas que desperdiçam o seu potencial, como administração de sistemas computacionais com o fim de alocar recursos quem deles necessita (docentes, investigadores e estudantes). Permitir que se peçam recursos computacionais, e estes sejam alocados com o mínimo de interacção de uma equipa administrativa, é um grande benefício. Isto não só aumenta a produtividade dos próprios administradores, como também permite que se obtenham os recursos mais depressa, e de forma mais segura. Esta meta é alcançável através do uso de uma framework de gestão de cloud, cujo objectivo é assistir na administração da infraestrutura computacional de um departamento. Esta dissertação tem como objectivo fornecer uma avaliação crítica da adequação de três frameworks de gestão de cloud, avaliar os requisitos necessários para uma cloud privada no DI-FCT-NOVA, e identificar que funcionalidades da framework selecionada podem ser utilizadas para a satisfação dos requisitos indicados. O objectivo final é dese- nhar e instalar a framework selecionada no DI-FCT-NOVA, oferecendo assim uma cloud privada de última geração, capaz de responder adequadamente às necessidades dos seus utilizadores - docentes, investigadores e estudantes

Optimizations for Energy-Aware, High-Performance and Reliable Distributed Storage Systems

With the decreasing cost and wide-spread use of commodity hard drives, it has become possible to create very large-scale storage systems with less expense. However, as we approach exabyte-scale storage systems, maintaining important features such as energy-efficiency, performance, reliability and usability became increasingly difficult. Despite the decreasing cost of storage systems, the energy consumption of these systems still needs to be addressed in order to retain cost-effectiveness. Any improvements in a storage system can be outweighed by high energy costs. On the other hand, large-scale storage systems can benefit more from the object storage features for improved performance and usability. One area of concern is metadata performance bottleneck of applications reading large directories or creating a large number of files. Similarly, computation on big data where data needs to be transferred between compute and storage clusters adversely affects I/O performance. As the storage systems become more complex and larger, transferring data between remote compute and storage tiers becomes impractical. Furthermore, storage systems implement reliability typically at the file system or client level. This approach might not always be practical in terms of performance. Lastly, object storage features are usually tailored to specific use cases that makes it harder to use them in various contexts. In this thesis, we are presenting several approaches to enhance energy-efficiency, performance, reliability and usability of large-scale storage systems. To begin with, we improve the energy-efficiency of storage systems by moving I/O load to a subset of the storage nodes with energy-aware node allocation methods and turn off the unused nodes, while preserving load balance on demand. To address the metadata performance issue associated with large creates and directory reads, we represent directories with object storage collections and implement lazy creation of objects. Similarly, in-situ computation on large-scale data is enabled by using object storage features to integrate a computational framework with the existing object storage layer to eliminate the need to transfer data between compute and storage silos for better performance. We then present parity-based redundancy using object storage features to achieve reliability with less performance impact. Finally, unified storage brings together the object storage features to meet the needs of distinct use cases; such as cloud storage, big data or high-performance computing to alleviate the unnecessary fragmentation of storage resources. We evaluate each proposed approach thoroughly and validate their effectiveness in terms of improving energy-efficiency, performance, reliability and usability of a large-scale storage system

Estrategias de optimización y análisis de performance en sistemas de almacenamiento distribuido

La complejidad de los sistemas de almacenamiento es creciente dada la cantidad y simultaneidad de clientes conectados, los datos que son accedidos de manera concurrente, los usuarios distribuidos geográficamente, con tiempos de respuesta acotados y un volumen de información transferida que se amplía de forma exponencial. Respondiendo a estas necesidades, surgen soluciones de Almacenamiento Definido por Software (Software Defined Storage), en las que distintos dispositivos conectados a través de una red de datos forman un cluster que ofrece un conjunto de interfaces a las aplicaciones y clientes, y proveen un complejo sistema de gestión, mantenimiento y monitoreo de los distintos componentes. La tesis centra su análisis en Ceph, un sistema de almacenamiento distribuido de código abierto que corre sobre hardware genérico, diseñado para proveer escalabilidad, confiabilidad y alta performance. Se determinan metodología y métricas para el análisis de performance y se lleva adelante un procedimiento de optimización capa por capa, desde la interfaz de red y los discos, hasta las configuraciones en el cliente para lograr minimizar latencia y maximizar el througput. Finalmente, se estabece un modelo de performance a modo de "linea base", con el fin de monitorear el funcionamiento del cluster y utilizar como referencia en ajustes futuros.Facultad de Informátic

A shared-disk parallel cluster file system

Dissertação apresentada para obtenção do Grau de Doutor em Informática Pela Universidade Nova de Lisboa, Faculdade de Ciências e TecnologiaToday, clusters are the de facto cost effective platform both for high performance computing (HPC) as well as IT environments. HPC and IT are quite different environments and differences include, among others, their choices on file systems and storage: HPC favours parallel file systems geared towards maximum I/O bandwidth, but which are not fully POSIX-compliant and were devised to run on top of (fault prone) partitioned storage; conversely, IT data centres favour both external disk arrays (to provide highly available storage) and POSIX compliant file systems, (either general purpose or shared-disk cluster file systems, CFSs). These specialised file systems do perform very well in their target environments provided that applications do not require some lateral features, e.g., no file locking on parallel file systems, and no high performance writes over cluster-wide shared files on CFSs. In brief, we can say that none of the above approaches solves the problem of providing high levels of reliability and performance to both worlds. Our pCFS proposal makes a contribution to change this situation: the rationale is to take advantage on the best of both – the reliability of cluster file systems and the high performance of parallel file systems. We don’t claim to provide the absolute best of each, but we aim at full POSIX compliance, a rich feature set, and levels of reliability and performance good enough for broad usage – e.g., traditional as well as HPC applications, support of clustered DBMS engines that may run over regular files, and video streaming. pCFS’ main ideas include: · Cooperative caching, a technique that has been used in file systems for distributed disks but, as far as we know, was never used either in SAN based cluster file systems or in parallel file systems. As a result, pCFS may use all infrastructures (LAN and SAN) to move data. · Fine-grain locking, whereby processes running across distinct nodes may define nonoverlapping byte-range regions in a file (instead of the whole file) and access them in parallel, reading and writing over those regions at the infrastructure’s full speed (provided that no major metadata changes are required). A prototype was built on top of GFS (a Red Hat shared disk CFS): GFS’ kernel code was slightly modified, and two kernel modules and a user-level daemon were added. In the prototype, fine grain locking is fully implemented and a cluster-wide coherent cache is maintained through data (page fragments) movement over the LAN. Our benchmarks for non-overlapping writers over a single file shared among processes running on different nodes show that pCFS’ bandwidth is 2 times greater than NFS’ while being comparable to that of the Parallel Virtual File System (PVFS), both requiring about 10 times more CPU. And pCFS’ bandwidth also surpasses GFS’ (600 times for small record sizes, e.g., 4 KB, decreasing down to 2 times for large record sizes, e.g., 4 MB), at about the same CPU usage.Lusitania, Companhia de Seguros S.A, Programa IBM Shared University Research (SUR

Matching distributed file systems with application workloads

Modern storage systems have a large number of configurable parameters, distributed over many layers of abstraction. The number of combinations of these parameters, that can be altered to create an instance of such a system, is enormous. In practise, many of these parameters are never altered; instead default values, intended to support generic workloads and access patterns, are used. As systems become larger and evolve to support different workloads, the appropriateness of using default parameters in this way comes into question. This thesis examines the implications of changing some of these parameters and explores the effects these changes have on performance. As part of that work multiple contributions have been made, including the creation of a structured method to create and evaluate different storage configurations, choosing appropriate access sizes for the evaluation, picking representative cloud workloads and capturing storage traces for further analysis, extraction of the workload storage characteristics, creating logical partitions of the distributed file system used for the optimization, the creation of heterogeneous storage pools within the homogeneous system and the mapping and evaluation of the chosen workloads to the examined configurations