Managing data sharing in OpenStack Swift with Over-Encryption

The sharing of large amounts of data is greatly facilitated by the adoption of cloud storage solutions. In many scenarios, this adoption could be hampered by possible concerns about data confidentiality, as cloud providers are not trusted to know the content of the data they store. Especially when the data are organized in objects, the application of an encryption layer is an interesting solution to this problem, because it offers strong confidentiality guarantees with a limited performance overhead. In a data sharing scenario, the management of access privileges then requires an adequate support for key derivation and for managing policy evolution. We present a solution that provides transparent support for the encryption of objects stored on Swift. Our system offers an efficient management of the updates to the access control policy, including revocation of authorizations from some of the sharing users. We explore several alternatives for the architecture, associated with distinct levels of transparency for the applications, and integrate different options for the management of policy updates. Our implementation and experiments demonstrate the easy integration of the approach with existing cloud storage solutions

A digital vault solution for banking institutions

Trabalho de projecto de mestrado, Segurança Informática, Universidade de Lisboa, Faculdade de Ciências, 2019Este projeto surgiu no âmbito da necessidade que a empresa Securibox tem em fornecer um produto de armazenamento seguro compatível com o funcionamento na nuvem, para as instituições bancárias que operam no mercado francês. Com o aparecimento da banca on-line e o intuito de atrair mais clientes, as instituições bancárias começaram a oferecer serviços que vão para além dos serviços convencionais deste setor. Muitas vezes esses serviços tratam ou armazenam dados sensíveis dos seus clientes e podem até incluir informação e documentos pessoais dos utilizadores que estão hospedados noutras entidades, tais como faturas eletrónicas, transações bancárias de outras instituições financeiras e recibos de vencimento. No entanto, sempre que for necessário armazenar informação dos clientes, este processo tem de respeitar um conjunto de boas práticas e normas do país onde a instituição opera, utilizando para o efeito um cofre digital. No caso do mercado francês, existem poucas soluções que satisfazem, parcialmente ou totalmente, as normas e a legislação respeitante aos cofres digitais e que sejam tecnicamente eficientes e competitivas. O objetivo deste trabalho visou desenvolver uma versão inicial de uma solução que colmata a necessidade atual do mercado bancário francês relativo à área de armazenamento e manuseamento inteligente de dados. Para satisfazer as normas da União Europeia e da França em particular, é necessário armazenar os ficheiros de forma cifrada, registar o seu formato, como, quando e por quem estes formas acedidos e os seus meta-dados de modo a garantir a sua preservação mesmo após a eliminação dos mesmos. Este desafio foi resolvido, e para se destacar das soluções atualmente existentes, foi construída a base para no futuro integrar esta solução com o serviço Securibox ParseXtract, que tem a capacidade de analisar e extrair informação importante do conteúdo dos documentos, de uma forma estruturada e precisa, recorrendo a aprendizagem automática. Para o armazenamento dos documentos a solução adotada foi o OpenStack Swift – um software de código aberto, compatível com nuvens pública e privadas. Uma vez que os documentos podem ser eliminados do sistema pelo utilizador, é necessário a existência de uma plataforma, separada do OpenStack, para armazenar os dados relativos aos meta-dados dos documentos e acessos ao sistema. A solução encontrada para o armazenamento destes dados, consiste no seu registo, através de logs, numa base de dados não relacional – o MongoDB, que é compatível com tecnologias em nuvem e é eficiente com grandes volumes de dados. Para realizar a comunicação entre os vários componentes do cofre digital, foi criado um serviço que oferece uma REST API, o núcleo da solução. Nesta camada, os documentos são cifrados garantindo também a integridade, confidencialidade e o não-repúdio dos dados. Por último, um servidor Web que comunica com a REST API foi criado para demonstrar todas as funcionalidades do cofre digital. As principais vantagens desta solução consistem na utilização de tecnologias código aberto, na compatibilidade com o funcionamento na nuvem, na escalabilidade de todas as suas camadas, tais como o armazenamento de dados, logs e serviço web API, e numa melhor integração com outros produtos da Securibox, que deste modo reduzem o custo da solução para o cliente final. Do ponto de vista conceptual, esta solução pode ser utilizada não apenas pelo sector bancário, mas também por qualquer outra área empresarial onde é necessário armazenar grandes volumes de dados em nuvem privada e pública em simultâneo, tendo como base uma solução facilmente escalável e onde todas as ações dos seus utilizadores são rastreáveis em conformidade com a legislação.This project is a result of the Securibox need to provide a digital vault storage solution for some of their bank clients, operating in the French market. Since electronic banking has emerged, banking institutions began to provide online services that go beyond conventional bank services to attract more users. Sometimes those services involve operations with personal data of their customers which can include data and documents from other services, entities and companies. All this information must be stored on the banking institution side, using a secure digital vault storage, while respecting the legislation of the country where the institution is located. The goal of this work was to develop an initial solution, that would address the current needs of the French banking market, regarding intelligent data handling and storage. To be compliant with the European Union and the French legislation it is necessary to ensure the security and the privacy of the costumers documents and data. To address those requirements a REST API solution was developed using .Net technology. This solution is divided in 3 layers. The document storage layer, the metadata and log storage layer and the core layer. The documents are encrypted and stored at the OpenStack Swift environment, while metadata is stored at the MongoDB database as journal log entries. The information processing and the communication between OpenStack and MongoDB occurs at the core layer. This solution relies on open-source technologies, is easily scalable and compatible with other Securibox products. Conceptually it can be used, not only by banking institutions, but also by any organization or company that have to store and deal with large amounts of information

Assured deletion in the cloud:requirements, challenges and future directions

Inadvertent exposure of sensitive data is a major concern for potential cloud customers. Much focus has been on other data leakage vectors, such as side channel attacks, while issues of data disposal and assured deletion have not received enough attention to date. However, data that is not properly destroyed may lead to unintended disclosures, in turn, resulting in heavy financial penalties and reputational damage. In non-cloud contexts, issues of incomplete deletion are well understood. To the best of our knowledge, to date, there has been no systematic analysis of assured deletion challenges in public clouds. In this paper, we aim to address this gap by analysing assured deletion requirements for the cloud, identifying cloud features that pose a threat to assured deletion, and describing various assured deletion challenges. Based on this discussion, we identify future challenges for research in this area and propose an initial assured deletion architecture for cloud settings. Altogether, our work offers a systematization of requirements and challenges of assured deletion in the cloud, and a well-founded reference point for future research in developing new solutions to assured deletion

Protecting resources and regulating access in cloud-based object storage

Cloud storage services offer a variety of benefits that make them extremely attractive for the management of large amounts of data. These services, however, raise some concerns related to the proper protection of data that, being stored on servers of third party cloud providers, are no more under the data owner control. The research and development community has addressed these concerns by proposing solutions where encryption is adopted not only for protecting data but also for regulating accesses. Depending on the trust assumption on the cloud provider offering the storage service, encryption can be applied at the server side, client side, or through an hybrid approach. The goal of this chapter is to survey these encryption-based solutions and to provide a description of some representative systems that adopt such solutions

Analysis of security in cloud platforms using OpenStack as case study

In the last few years, cloud computing has grown from being a promising business concept to one of the fastest growing segments of the IT industry. Big companies like Amazon, Google, Microsoft etc., expand their market by adopting Cloud Computing systems which enhance their services provided to a large number of users. However, security and privacy issues present a strong barrier for users to adapt into the Cloud.This research investigates the security features and issues of cloud platforms using OpenStack as a case study. The goal was to identify security weakness in terms of Authentication and Identity Management(IAM), and Data Management. Base on the findings, specific recommendations on security standards and management models have been proffered in order to address these problems. These Recommendations if implemented, will lead to trust in cloud computing systems, which in turn would encourage more companies to adopt cloud computing, as a means of providing better IT services

Access Control Management for Secure Cloud Storage

With the widespread success and adoption of cloud-based solutions, we are witnessing an ever increasing reliance on external providers for storing and managing data. This evolution is greatly facilitated by the availability of solutions - typically based on encryption - ensuring the confidentiality of externally outsourced data against the storing provider itself. Selective application of encryption (i.e., with different keys depending on the authorizations holding on data) provides a convenient approach to access control policy enforcement. Effective realization of such policy-based encryption entails addressing several problems related to key management, access control enforcement, and authorization revocation, while ensuring efficiency of access and deployment with current technology. We present the design and implementation of an approach to realize policy-based encryption for enforcing access control in OpenStack Swift. We also report experimental results evaluating and comparing different implementation choices of our approach

Enhancing the Programmability of Cloud Object Storage

En un món que depèn cada vegada més de la tecnologia, les dades digitals es generen a una escala sense precedents. Això fa que empreses que requereixen d'un gran espai d'emmagatzematge, com Netflix o Dropbox, utilitzin solucions d'emmagatzematge al núvol. Mes concretament, l'emmagatzematge d'objectes, donada la seva simplicitat, escalabilitat i alta disponibilitat. No obstant això, aquests magatzems s'enfronten a tres desafiaments principals: 1) Gestió flexible de càrregues de treball de múltiples usuaris. Normalment, els magatzems d'objectes són sistemes multi-usuari, la qual cosa significa que tots ells comparteixen els mateixos recursos, el que podria ocasionar problemes d'interferència. A més, és complex administrar polítiques d'emmagatzematge heterogènies a gran escala en ells. 2) Autogestió de dades. Els magatzems d'objectes no ofereixen molta flexibilitat pel que fa a l'autogestió de dades per part dels usuaris. Típicament, són sistemes rígids, la qual cosa impedeix gestionar els requisits específics dels objectes. 3) Còmput elàstic prop de les dades. Situar els càlculs prop de les dades pot ser útil per reduir la transferència de dades. Però, el desafiament aquí és com aconseguir la seva elasticitat sense provocar contenció de recursos i interferències en la capa d'emmagatzematge. En aquesta tesi presentem tres contribucions innovadores que resolen aquests desafiaments. En primer lloc, presentem la primera arquitectura d'emmagatzematge definida per programari (SDS) per a magatzems d'objectes que separa les capes de control i de dades. Això permet gestionar les càrregues de treball de múltiples usuaris d'una manera flexible i dinàmica. En segon lloc, hem dissenyat una nova abstracció de polítiques anomenada "microcontrolador" que transforma els objectes comuns en objectes intel·ligents, permetent als usuaris programar el seu comportament. Finalment, presentem la primera plataforma informàtica "serverless" guiada per dades i elàstica, que mitiga els problemes de col·locar el càlcul prop de les dades.En un mundo que depende cada vez más de la tecnología, los datos digitales se generan a una escala sin precedentes. Esto hace que empresas que requieren de un gran espacio de almacenamiento, como Netflix o Dropbox, usen soluciones de almacenamiento en la nube. Mas concretamente, el almacenamiento de objectos, dada su escalabilidad y alta disponibilidad. Sin embargo, estos almacenes se enfrentan a tres desafíos principales: 1) Gestión flexible de cargas de trabajo de múltiples usuarios. Normalmente, los almacenes de objetos son sistemas multi-usuario, lo que significa que todos ellos comparten los mismos recursos, lo que podría ocasionar problemas de interferencia. Además, es complejo administrar políticas de almacenamiento heterogéneas a gran escala en ellos. 2) Autogestión de datos. Los almacenes de objetos no ofrecen mucha flexibilidad con respecto a la autogestión de datos por parte de los usuarios. Típicamente, son sistemas rígidos, lo que impide gestionar los requisitos específicos de los objetos. 3) Cómputo elástico cerca de los datos. Situar los cálculos cerca de los datos puede ser útil para reducir la transferencia de datos. Pero, el desafío aquí es cómo lograr su elasticidad sin provocar contención de recursos e interferencias en la capa de almacenamiento. En esta tesis presentamos tres contribuciones que resuelven estos desafíos. En primer lugar, presentamos la primera arquitectura de almacenamiento definida por software (SDS) para almacenes de objetos que separa las capas de control y de datos. Esto permite gestionar las cargas de trabajo de múltiples usuarios de una manera flexible y dinámica. En segundo lugar, hemos diseñado una nueva abstracción de políticas llamada "microcontrolador" que transforma los objetos comunes en objetos inteligentes, permitiendo a los usuarios programar su comportamiento. Finalmente, presentamos la primera plataforma informática "serverless" guiada por datos y elástica, que mitiga los problemas de colocar el cálculo cerca de los datos.In a world that is increasingly dependent on technology, digital data is generated in an unprecedented way. This makes companies that require large storage space, such as Netflix or Dropbox, use cloud object storage solutions. This is mainly thanks to their built-in characteristics, such as simplicity, scalability and high-availability. However, cloud object stores face three main challenges: 1) Flexible management of multi-tenant workloads. Commonly, cloud object stores are multi-tenant systems, meaning that all tenants share the same system resources, which could lead to interference problems. Furthermore, it is now complex to manage heterogeneous storage policies in a massive scale. 2) Data self-management. Cloud object stores themselves do not offer much flexibility regarding data self-management by tenants. Typically, they are rigid, which prevent tenants to handle the specific requirements of their objects. 3) Elastic computation close to the data. Placing computations close to the data can be useful to reduce data transfers. But, the challenge here is how to achieve elasticity in those computations without provoking resource contention and interferences in the storage layer. In this thesis, we present three novel research contributions that solve the aforementioned challenges. Firstly, we introduce the first Software-defined Storage (SDS) architecture for cloud object stores that separates the control plane from the data plane, allowing to manage multi-tenant workloads in a flexible and dynamic way. For example, by applying different service levels of bandwidth to different tenants. Secondly, we designed a novel policy abstraction called microcontroller that transforms common objects into smart objects, enabling tenants to programmatically manage their behavior. For example, a content-level access control microcontroller attached to an specific object to filter its content depending on who is accessing it. Finally, we present the first elastic data-driven serverless computing platform that mitigates the resource contention problem of placing computation close to the data

Building and Configuring a Custom Private Cloud Using Consumer Hardware

Pilve kasutamine on tänapäeval moodne trend erinevatele organisatsioonidele. Selle ajendiks on efektiivsus, sest pilv lubab kasutada olemasolevaid ressursse kõige paindlikumal ja efektiivseimal viisil. Bakalaureusetöö aluseks oli kapis seisev madala astme riistvara, mida keegi ei kasutanud. Töö põhipanus on see, et olemasolev riistvara ehitati üles privaatpilveks. Kuna privaatpilve sobib pigem kõrgklassi serveri tasemel riistvara, siis tekitab madala taseme riistvara mõningaid probleeme, kuid töö näitab, et nendest võib üle vaadata, kuna boonuseid on rohkem kui negatiivseid aspekte. Töö lõpptulemuseks on töötav OpenStacki implementatsioon, mida on kerge kasutada igaühel. See on igalt poolt kättesaadav ja piisavalt paindlik täitmaks erinevaid Tartu Ülikooli hajussüsteemide uurimisrühma vajadusi. Saadud infrastruktuur on kergelt skaleeruv ning füüsiliste masinate lisamine võtab vähem kui 30 minutit. Lisaks tagab OpenStack selle, et projektide ja kasutajate haldus on väga kerge ning teadlased saavad teha virtuaalmasinaid vähem kui minutiga.Moving into the cloud is a common trend for organizations to use existing hardware in an efficient way. organizations specifically use private or hybrid clouds. Existing unused hardware was the main problem in this thesis. Contribution of this thesis was a description of how to build and configure OpenStack using consumer grade hardware. Using low end hardware to build a working private cloud does have some disadvantages, but in our case, it was not that critical. In the end, there were more advantages than disadvantages. The end result is a working implementation of OpenStack, which is easy to use, accessible from anywhere in the world and flexible enough to fill the needs of Distributed Systems in University of Tartu platforms. This thesis and configurations in appendix provide a scalable solution - additional compute nodes can be setup with less than 30 minutes. Creation of virtual machines takes less than a minute by using a web interface, which is very easy to understand and use