A Survey on the Integration of NAND Flash Storage in the Design of File Systems and the Host Storage Software Stack

With the ever-increasing amount of data generate in the world, estimated to reach over 200 Zettabytes by 2025, pressure on efficient data storage systems is intensifying. The shift from HDD to flash-based SSD provides one of the most fundamental shifts in storage technology, increasing performance capabilities significantly. However, flash storage comes with different characteristics than prior HDD storage technology. Therefore, storage software was unsuitable for leveraging the capabilities of flash storage. As a result, a plethora of storage applications have been design to better integrate with flash storage and align with flash characteristics. In this literature study we evaluate the effect the introduction of flash storage has had on the design of file systems, which providing one of the most essential mechanisms for managing persistent storage. We analyze the mechanisms for effectively managing flash storage, managing overheads of introduced design requirements, and leverage the capabilities of flash storage. Numerous methods have been adopted in file systems, however prominently revolve around similar design decisions, adhering to the flash hardware constrains, and limiting software intervention. Future design of storage software remains prominent with the constant growth in flash-based storage devices and interfaces, providing an increasing possibility to enhance flash integration in the host storage software stack

A Survey on the Integration of NAND Flash Storage in the Design of File Systems and the Host Storage Software Stack

With the ever-increasing amount of data generate in the world, estimated to reach over 200 Zettabytes by 2025, pressure on efficient data storage systems is intensifying. The shift from HDD to flash-based SSD provides one of the most fundamental shifts in storage technology, increasing performance capabilities significantly. However, flash storage comes with different characteristics than prior HDD storage technology. Therefore, storage software was unsuitable for leveraging the capabilities of flash storage. As a result, a plethora of storage applications have been design to better integrate with flash storage and align with flash characteristics. In this literature study we evaluate the effect the introduction of flash storage has had on the design of file systems, which providing one of the most essential mechanisms for managing persistent storage. We analyze the mechanisms for effectively managing flash storage, managing overheads of introduced design requirements, and leverage the capabilities of flash storage. Numerous methods have been adopted in file systems, however prominently revolve around similar design decisions, adhering to the flash hardware constrains, and limiting software intervention. Future design of storage software remains prominent with the constant growth in flash-based storage devices and interfaces, providing an increasing possibility to enhance flash integration in the host storage software stack

A sidecar object for the optimized communication between edge and cloud in internet of things applications

The internet of things (IoT) is one of the most disrupting revolutions that is characterizing the technology ecosystem. In the near future, the IoT will have a significant impact on people's lives and on the design and developments of new paradigms and architectures coping with a completely new set of challenges and service categories. The IoT can be described as an ecosystem where a massive number of constrained devices (denoted as smart objects) will be deployed and connected to cooperate for multiple purposes, such a data collection, actuation, and interaction with people. In order to meet the specific requirements, IoT services may be deployed leveraging a hybrid architecture that will involve services deployed on the edge and the cloud. In this context, one of the challenges is to create an infrastructure of objects and microservices operating between both the edge and in the cloud that can be easily updated and extended with new features and functionalities without the need of updating or re-deploying smart objects. This work introduces a new concept for extending smart objects' support for cloud services, denoted as a sidecar object. A sidecar object serves the purpose of being deployed as additional component of a preexisting object without interfering with the mechanisms and behaviors that have already been implemented. In particular, the sidecar object implementation developed in this work focuses on the communication with existing IoT cloud services (namely, AWS IoT and Google Cloud IoT) to provide a transparent and seamless synchronization of data, states, and commands between the object on the edge and the cloud. The proposed sidecar object implementation has been extensively evaluated through a detailed set of tests, in order to analyze the performances and behaviors in real- world scenarios

A reliability model for dependent and distributed MDS disk array units

Archiving and systematic backup of large digital data generates a quick demand for multi-petabyte scale storage systems. As drive capacities continue to grow beyond the few terabytes range to address the demands of today’s cloud, the likelihood of having multiple/simultaneous disk failures became a reality. Among the main factors causing catastrophic system failures, correlated disk failures and the network bandwidth are reported to be the two common source of performance degradation. The emerging trend is to use efficient/sophisticated erasure codes (EC) equipped with multiple parities and efficient repairs in order to meet the reliability/bandwidth requirements. It is known that mean time to failure and repair rates reported by the disk manufacturers cannot capture life-cycle patterns of distributed storage systems. In this study, we develop failure models based on generalized Markov chains that can accurately capture correlated performance degradations with multiparity protection schemes based on modern maximum distance separable EC. Furthermore, we use the proposed model in a distributed storage scenario to quantify two example use cases: Primarily, the common sense that adding more parity disks are only meaningful if we have a decent decorrelation between the failure domains of storage systems and the reliability of generic multiple single-dimensional EC protected storage systems.WOS:000460728600008Scopus - Affiliation ID: 60105072Science Citation Index ExpandedQ1 - Q2ArticleUluslararası işbirliği ile yapılmayan - HAYIRMart2019YÖK - 2018-1

On I/O Performance and Cost Efficiency of Cloud Storage: A Client\u27s Perspective

Cloud storage has gained increasing popularity in the past few years. In cloud storage, data are stored in the service provider’s data centers; users access data via the network and pay the fees based on the service usage. For such a new storage model, our prior wisdom and optimization schemes on conventional storage may not remain valid nor applicable to the emerging cloud storage. In this dissertation, we focus on understanding and optimizing the I/O performance and cost efficiency of cloud storage from a client’s perspective. We first conduct a comprehensive study to gain insight into the I/O performance behaviors of cloud storage from the client side. Through extensive experiments, we have obtained several critical findings and useful implications for system optimization. We then design a client cache framework, called Pacaca, to further improve end-to-end performance of cloud storage. Pacaca seamlessly integrates parallelized prefetching and cost-aware caching by utilizing the parallelism potential and object correlations of cloud storage. In addition to improving system performance, we have also made efforts to reduce the monetary cost of using cloud storage services by proposing a latency- and cost-aware client caching scheme, called GDS-LC, which can achieve two optimization goals for using cloud storage services: low access latency and low monetary cost. Our experimental results show that our proposed client-side solutions significantly outperform traditional methods. Our study contributes to inspiring the community to reconsider system optimization methods in the cloud environment, especially for the purpose of integrating cloud storage into the current storage stack as a primary storage layer

A Comprehensive Survey of In-Band Control in SDN: Challenges and Opportunities

Software-Defined Networking (SDN) is a thriving networking architecture that has gained popularity in recent years, particularly as an enabling technology to foster paradigms like edge computing. SDN separates the control and data planes, which are later on synchronised via a control protocol such as OpenFlow. In-band control is a type of SDN control plane deployment in which the control and data planes share the same physical network. It poses several challenges, such as security vulnerabilities, network congestion, or data loss. Nevertheless, despite these challenges, in-band control also presents significant opportunities, including improved network flexibility and programmability, reduced costs, and increased reliability. Benefiting from the previous advantages, diverse in-band control designs exist in the literature, with the objective of improving the operation of SDN networks. This paper surveys the different approaches that have been proposed so far towards the advance in in-band SDN control, based on four main categories: automatic routing, fast failure recovery, network bootstrapping, and distributed control. Across these categories, detailed summary tables and comparisons are presented, followed by a discussion on current trends a challenges in the field. Our conclusion is that the use of in-band control in SDN networks is expected to drive innovation and growth in the networking industry, but efforts for holistic and full-fledged proposals are still needed

Caches collaboratifs noyau adaptés aux environnements virtualisés

With the advent of cloud architectures, virtualization has become a key mechanism for ensuring isolation and flexibility. However, a drawback of using virtual machines (VMs) is the fragmentation of physical resources. As operating systems leverage free memory for I/O caching, memory fragmentation is particularly problematic for I/O-intensive applications, which suffer a significant performance drop. In this context, providing the ability to dynamically adjust the resources allocated among the VMs is a primary concern.To address this issue, this thesis proposes a distributed cache mechanism called Puma. Puma pools together the free memory left unused by VMs: it enables a VM to entrust clean page-cache pages to other VMs. Puma extends the Linux kernel page cache, and thus remains transparent, to both applications and the rest of the operating system. Puma adjusts itself dynamically to the caching activity of a VM, which Puma evaluates by means of metrics derived from existing Linux kernel memory management mechanisms. Our experiments show that Puma significantly improves the performance of I/O-intensive applications and that it adapts well to dynamically changing conditions.Avec l'avènement du cloud computing, la virtualisation est devenue aujourd'hui incontournable. Elle offre isolation et flexibilité, en revanche elle implique une fragmentation des ressources, et notamment de la mémoire. Les performances des applications qui effectuent beaucoup d'entrées/sorties (E/S) en sont particulièrement impactées. En effet, celles-ci reposent en grande partie sur la présence de mémoire libre, utilisée par le système pour faire du cache et ainsi accélérer les E/S. Ajuster dynamiquement les ressources d'une machine virtuelle devient donc un enjeu majeur. Dans cette thèse nous nous intéressons à ce problème, et nous proposons Puma, un cache réparti permettant de mutualiser la mémoire inutilisée des machines virtuelles pour améliorer les performances des applications qui effectuent beaucoup d'E/S. Contrairement aux solutions existantes, notre approche noyau permet à Puma de fonctionner avec les applications sans adaptation ni système de fichiers spécifique. Nous proposons plusieurs métriques, reposant sur des mécanismes existants du noyau Linux, qui permettent de définir le niveau d'activité « cache » du système. Ces métriques sont utilisées par Puma pour automatiser le niveau de contribution d'un noeud au cache réparti. Nos évaluations de Puma montrent qu'il est capable d'améliorer significativement les performances d'applications qui effectuent beaucoup d'E/S et de s'adapter dynamiquement afin de ne pas dégrader leurs performances