Improving OpenStack Swift interaction with the I/O stack to enable software defined storage

This paper analyses how OpenStack Swift, a distributed object storage service for a globally used middleware, interacts with the I/O subsystem through the Operating System. This interaction, which seems organised and clean on the middleware side, becomes disordered on the device side when using mechanical disk drives, due to the way threads are used internally to request data. We will show that only modifying the Swift threading model we achieve an 18% mean improvement in performance with objects larger than 512 KiB and obtain a similar performance with smaller objects. Compared to the original scenario, the performance obtained on both scenarios is obtained in a fair way: the bandwidth is shared equally between concurrently accessed objects. Moreover, this threading model allows us to apply techniques for Software Defined Storage (SDS). We show an implementation of a Bandwidth Differentiation technique that can control each data stream and that guarantees a high utilization of the device.The research leading to these results has received funding from the European Community under the IOStack (H2020-ICT-2014-7-1) project, by the Spanish Ministry of Economy and Competitiveness under the TIN2015-65316-P grant and by the Catalan Government under the 2014-SGR-1051 grant. To learn more about the IOStack H2020 project, please visit http:nnwww.iostack.eu.Peer ReviewedPostprint (author's final draft

Experimental Performance Evaluation of Cloud-Based Analytics-as-a-Service

An increasing number of Analytics-as-a-Service solutions has recently seen the light, in the landscape of cloud-based services. These services allow flexible composition of compute and storage components, that create powerful data ingestion and processing pipelines. This work is a first attempt at an experimental evaluation of analytic application performance executed using a wide range of storage service configurations. We present an intuitive notion of data locality, that we use as a proxy to rank different service compositions in terms of expected performance. Through an empirical analysis, we dissect the performance achieved by analytic workloads and unveil problems due to the impedance mismatch that arise in some configurations. Our work paves the way to a better understanding of modern cloud-based analytic services and their performance, both for its end-users and their providers.Comment: Longer version of the paper in Submission at IEEE CLOUD'1

A digital vault solution for banking institutions

Trabalho de projecto de mestrado, Segurança Informática, Universidade de Lisboa, Faculdade de Ciências, 2019Este projeto surgiu no âmbito da necessidade que a empresa Securibox tem em fornecer um produto de armazenamento seguro compatível com o funcionamento na nuvem, para as instituições bancárias que operam no mercado francês. Com o aparecimento da banca on-line e o intuito de atrair mais clientes, as instituições bancárias começaram a oferecer serviços que vão para além dos serviços convencionais deste setor. Muitas vezes esses serviços tratam ou armazenam dados sensíveis dos seus clientes e podem até incluir informação e documentos pessoais dos utilizadores que estão hospedados noutras entidades, tais como faturas eletrónicas, transações bancárias de outras instituições financeiras e recibos de vencimento. No entanto, sempre que for necessário armazenar informação dos clientes, este processo tem de respeitar um conjunto de boas práticas e normas do país onde a instituição opera, utilizando para o efeito um cofre digital. No caso do mercado francês, existem poucas soluções que satisfazem, parcialmente ou totalmente, as normas e a legislação respeitante aos cofres digitais e que sejam tecnicamente eficientes e competitivas. O objetivo deste trabalho visou desenvolver uma versão inicial de uma solução que colmata a necessidade atual do mercado bancário francês relativo à área de armazenamento e manuseamento inteligente de dados. Para satisfazer as normas da União Europeia e da França em particular, é necessário armazenar os ficheiros de forma cifrada, registar o seu formato, como, quando e por quem estes formas acedidos e os seus meta-dados de modo a garantir a sua preservação mesmo após a eliminação dos mesmos. Este desafio foi resolvido, e para se destacar das soluções atualmente existentes, foi construída a base para no futuro integrar esta solução com o serviço Securibox ParseXtract, que tem a capacidade de analisar e extrair informação importante do conteúdo dos documentos, de uma forma estruturada e precisa, recorrendo a aprendizagem automática. Para o armazenamento dos documentos a solução adotada foi o OpenStack Swift – um software de código aberto, compatível com nuvens pública e privadas. Uma vez que os documentos podem ser eliminados do sistema pelo utilizador, é necessário a existência de uma plataforma, separada do OpenStack, para armazenar os dados relativos aos meta-dados dos documentos e acessos ao sistema. A solução encontrada para o armazenamento destes dados, consiste no seu registo, através de logs, numa base de dados não relacional – o MongoDB, que é compatível com tecnologias em nuvem e é eficiente com grandes volumes de dados. Para realizar a comunicação entre os vários componentes do cofre digital, foi criado um serviço que oferece uma REST API, o núcleo da solução. Nesta camada, os documentos são cifrados garantindo também a integridade, confidencialidade e o não-repúdio dos dados. Por último, um servidor Web que comunica com a REST API foi criado para demonstrar todas as funcionalidades do cofre digital. As principais vantagens desta solução consistem na utilização de tecnologias código aberto, na compatibilidade com o funcionamento na nuvem, na escalabilidade de todas as suas camadas, tais como o armazenamento de dados, logs e serviço web API, e numa melhor integração com outros produtos da Securibox, que deste modo reduzem o custo da solução para o cliente final. Do ponto de vista conceptual, esta solução pode ser utilizada não apenas pelo sector bancário, mas também por qualquer outra área empresarial onde é necessário armazenar grandes volumes de dados em nuvem privada e pública em simultâneo, tendo como base uma solução facilmente escalável e onde todas as ações dos seus utilizadores são rastreáveis em conformidade com a legislação.This project is a result of the Securibox need to provide a digital vault storage solution for some of their bank clients, operating in the French market. Since electronic banking has emerged, banking institutions began to provide online services that go beyond conventional bank services to attract more users. Sometimes those services involve operations with personal data of their customers which can include data and documents from other services, entities and companies. All this information must be stored on the banking institution side, using a secure digital vault storage, while respecting the legislation of the country where the institution is located. The goal of this work was to develop an initial solution, that would address the current needs of the French banking market, regarding intelligent data handling and storage. To be compliant with the European Union and the French legislation it is necessary to ensure the security and the privacy of the costumers documents and data. To address those requirements a REST API solution was developed using .Net technology. This solution is divided in 3 layers. The document storage layer, the metadata and log storage layer and the core layer. The documents are encrypted and stored at the OpenStack Swift environment, while metadata is stored at the MongoDB database as journal log entries. The information processing and the communication between OpenStack and MongoDB occurs at the core layer. This solution relies on open-source technologies, is easily scalable and compatible with other Securibox products. Conceptually it can be used, not only by banking institutions, but also by any organization or company that have to store and deal with large amounts of information

OpenStack Swift: An Ideal Bit-Level Object Storage System for Digital Preservation

A bit-level object storage system is a foundational building block of long-term digital preservation (LTDP). To achieve the purposes of LTDP, the system must be able to: preserve the authenticity and integrity of the original digital objects; scale up with dramatically increasing demands for preservation storage; mitigate the impact of hardware obsolescence and software ephemerality; replicate digital objects among distributed data centers at different geographical locations; and to constantly audit and automatically recover from compromised states. A realistic and daunting challenge to satisfy these requirements is not only to overcome technological difficulties but also to maintain economic sustainability by implementing and continuously operating such systems in a cost-effective way. In this paper, we present OpenStack Swift, an open-source, mature and widely accepted cloud platform, as a practical and proven solution with a case study at the University of Alberta Library. We emphasize the implementation, application, cost analysis and maintenance of the system, with the purpose of contributing to the community with an exceedingly robust, highly scalable, self-healing and comparatively cost-effective bit-level object storage system for long-term digital preservation.&nbsp

Assured deletion in the cloud:requirements, challenges and future directions

Inadvertent exposure of sensitive data is a major concern for potential cloud customers. Much focus has been on other data leakage vectors, such as side channel attacks, while issues of data disposal and assured deletion have not received enough attention to date. However, data that is not properly destroyed may lead to unintended disclosures, in turn, resulting in heavy financial penalties and reputational damage. In non-cloud contexts, issues of incomplete deletion are well understood. To the best of our knowledge, to date, there has been no systematic analysis of assured deletion challenges in public clouds. In this paper, we aim to address this gap by analysing assured deletion requirements for the cloud, identifying cloud features that pose a threat to assured deletion, and describing various assured deletion challenges. Based on this discussion, we identify future challenges for research in this area and propose an initial assured deletion architecture for cloud settings. Altogether, our work offers a systematization of requirements and challenges of assured deletion in the cloud, and a well-founded reference point for future research in developing new solutions to assured deletion

Evaluation of containers as a virtualisation alternative for HEP workloads

In this paper the emerging technology of Linux containers is examined and evaluated for use in the High Energy Physics (HEP) community. Key technologies required to enable containerisation will be discussed along with emerging technologies used to manage container images. An evaluation of the requirements for containers within HEP will be made and benchmarking will be carried out to asses performance over a range of HEP workflows. The use of containers will be placed in a broader context and recommendations on future work will be given

Implementation and Deployment of a Server at the Edge Using OpenStack Components

As the 5th telecommunication Generation (5G) deployments are spreading around via various mobile operators, the capabilities behind 5G are becoming more and more understandable. Infrastructure vendors, operators, and end users now have a clear picture of the 5G potential and, for that reason, the research and the development of 5G are surely continuing. The one-to-one mapping between 5G and Software Defined Network - Network Function Virtualization (SDN-NFV) architecture is not in discussion, but the impact of porting SDN-NFV into the Radio Access Network (RAN) is still under investigation. Sometimes, the RAN requirements set strong limitations even in the basic hardware and software setup. For example, the most complete and very well integrated SDN-NFV infrastructure distributions require specific hardware capabilities in terms of available nodes, in contrast with the RAN requirement to be economic, power consumption limited and with limited overhead due to operating system and middleware cost. For that reason, this study uses only a minimal set of OpenStack components in order to evaluate what is the minimal hardware capability needed to set up a basic, but fully working environment for NFV, highlighting the pros and cons of embracing a solution solely based on standard OpenStack components