Secure Remote Storage of Logs with Search Capabilities

Dissertação de Mestrado em Engenharia InformáticaAlong side with the use of cloud-based services, infrastructure and storage, the use of application logs in business critical applications is a standard practice nowadays. Such application logs must be stored in an accessible manner in order to used whenever needed. The debugging of these applications is a common situation where such access is required. Frequently, part of the information contained in logs records is sensitive. This work proposes a new approach of storing critical logs in a cloud-based storage recurring to searchable encryption, inverted indexing and hash chaining techniques to achieve, in a unified way, the needed privacy, integrity and authenticity while maintaining server side searching capabilities by the logs owner. The designed search algorithm enables conjunctive keywords queries plus a fine-grained search supported by field searching and nested queries, which are essential in the referred use case. To the best of our knowledge, the proposed solution is also the first to introduce a query language that enables complex conjunctive keywords and a fine-grained search backed by field searching and sub queries.A gerac¸ ˜ao de logs em aplicac¸ ˜oes e a sua posterior consulta s˜ao fulcrais para o funcionamento de qualquer neg´ocio ou empresa. Estes logs podem ser usados para eventuais ac¸ ˜oes de auditoria, uma vez que estabelecem uma baseline das operac¸ ˜oes realizadas. Servem igualmente o prop´ osito de identificar erros, facilitar ac¸ ˜oes de debugging e diagnosticar bottlennecks de performance. Tipicamente, a maioria da informac¸ ˜ao contida nesses logs ´e considerada sens´ıvel. Quando estes logs s˜ao armazenados in-house, as considerac¸ ˜oes relacionadas com anonimizac¸ ˜ao, confidencialidade e integridade s˜ao geralmente descartadas. Contudo, com o advento das plataformas cloud e a transic¸ ˜ao quer das aplicac¸ ˜oes quer dos seus logs para estes ecossistemas, processos de logging remotos, seguros e confidenciais surgem como um novo desafio. Adicionalmente, regulac¸ ˜ao como a RGPD, imp˜oe que as instituic¸ ˜oes e empresas garantam o armazenamento seguro dos dados. A forma mais comum de garantir a confidencialidade consiste na utilizac¸ ˜ao de t ´ecnicas criptogr ´aficas para cifrar a totalidade dos dados anteriormente `a sua transfer ˆencia para o servidor remoto. Caso sejam necess´ arias capacidades de pesquisa, a abordagem mais simples ´e a transfer ˆencia de todos os dados cifrados para o lado do cliente, que proceder´a `a sua decifra e pesquisa sobre os dados decifrados. Embora esta abordagem garanta a confidencialidade e privacidade dos dados, rapidamente se torna impratic ´avel com o crescimento normal dos registos de log. Adicionalmente, esta abordagem n˜ao faz uso do potencial total que a cloud tem para oferecer. Com base nesta tem´ atica, esta tese prop˜oe o desenvolvimento de uma soluc¸ ˜ao de armazenamento de logs operacionais de forma confidencial, integra e autˆ entica, fazendo uso das capacidades de armazenamento e computac¸ ˜ao das plataformas cloud. Adicionalmente, a possibilidade de pesquisa sobre os dados ´e mantida. Essa pesquisa ´e realizada server-side diretamente sobre os dados cifrados e sem acesso em momento algum a dados n˜ao cifrados por parte do servidor..

POPE: Partial Order Preserving Encoding

Recently there has been much interest in performing search queries over encrypted data to enable functionality while protecting sensitive data. One particularly efficient mechanism for executing such queries is order-preserving encryption/encoding (OPE) which results in ciphertexts that preserve the relative order of the underlying plaintexts thus allowing range and comparison queries to be performed directly on ciphertexts. In this paper, we propose an alternative approach to range queries over encrypted data that is optimized to support insert-heavy workloads as are common in "big data" applications while still maintaining search functionality and achieving stronger security. Specifically, we propose a new primitive called partial order preserving encoding (POPE) that achieves ideal OPE security with frequency hiding and also leaves a sizable fraction of the data pairwise incomparable. Using only O(1) persistent and $O(n^\epsilon)$ non-persistent client storage for $0<\epsilon<1$, our POPE scheme provides extremely fast batch insertion consisting of a single round, and efficient search with O(1) amortized cost for up to $O(n^{1-\epsilon})$ search queries. This improved security and performance makes our scheme better suited for today's insert-heavy databases.Comment: Appears in ACM CCS 2016 Proceeding

Controlled and Secure Sharing Threat Intelligence

Cyber threat information sharing platforms have become a useful weapon for dealing with cyberattacks, proactively mitigating them and thus reducing risk exposure. These allow multiple agencies to connect with each other, forming a community, and share that same intrusion information regarding cyberattacks or threats with each other. The Malware Information Sharing Platform (MISP) is particularly developed to promote the open dissemination of information such as intrusion indicators within a community. This exchange of information related to threats or incidents is treated as a data synchronisation procedure between di erent MISP instances, which may belong to one or more communities, companies or organisations. However, this platform presents limitations if its information is considered as classi ed or shared only for a certain period of time. This implies that this information should be treated only in encrypted form. One solution is to use MISP with searchable encryption techniques to impose greater control over information sharing. In this document, it is present a system that guarantees a controlled synchronisation of information between entities through the use of encrypted search techniques to guarantee the con dentiality of the information present in the MISP platform and also the use of synchronisation policies to control the way information is exchanged

Towards a secure and efficient search over encrypted cloud data

Includes bibliographical references.2016 Summer.Cloud computing enables new types of services where the computational and network resources are available online through the Internet. One of the most popular services of cloud computing is data outsourcing. For reasons of cost and convenience, public as well as private organizations can now outsource their large amounts of data to the cloud and enjoy the benefits of remote storage and management. At the same time, confidentiality of remotely stored data on untrusted cloud server is a big concern. In order to reduce these concerns, sensitive data, such as, personal health records, emails, income tax and financial reports, are usually outsourced in encrypted form using well-known cryptographic techniques. Although encrypted data storage protects remote data from unauthorized access, it complicates some basic, yet essential data utilization services such as plaintext keyword search. A simple solution of downloading the data, decrypting and searching locally is clearly inefficient since storing data in the cloud is meaningless unless it can be easily searched and utilized. Thus, cloud services should enable efficient search on encrypted data to provide the benefits of a first-class cloud computing environment. This dissertation is concerned with developing novel searchable encryption techniques that allow the cloud server to perform multi-keyword ranked search as well as substring search incorporating position information. We present results that we have accomplished in this area, including a comprehensive evaluation of existing solutions and searchable encryption schemes for ranked search and substring position search

Searchable Encryption for Cloud and Distributed Systems

The vast development in information and communication technologies has spawned many new computing and storage architectures in the last two decades. Famous for its powerful computation ability and massive storage capacity, cloud services, including storage and computing, replace personal computers and software systems in many industrial applications. Another famous and influential computing and storage architecture is the distributed system, which refers to an array of machines or components geographically dispersed but jointly contributes to a common task, bringing premium scalability, reliability, and efficiency. Recently, the distributed cloud concept has also been proposed to benefit both cloud and distributed computing. Despite the benefits of these new technologies, data security and privacy are among the main concerns that hinder the wide adoption of these attractive architectures since data and computation are not under the control of the end-users in such systems. The traditional security mechanisms, e.g., encryption, cannot fit these new architectures since they would disable the fast access and retrieval of remote storage servers. Thus, an urgent question turns to be how to enable refined and efficient data retrieval on encrypted data among numerous records (i.e., searchable encryption) in the cloud and distributed systems, which forms the topic of this thesis. Searchable encryption technologies can be divided into Searchable Symmetric Encryption (SSE) and Public-key Encryption with Keyword Search (PEKS). The intrinsical symmetric key hinders data sharing since it is problematic and insecure to reveal one’s key to others. However, SSE outperforms PEKS due to its premium efficiency and is thus is prefered in a number of keyword search applications. Then multi-user SSE with rigorous and fine access control undoubtedly renders a satisfactory solution of both efficiency and security, which is the first problem worthy of our much attention. Second, functions and versatility play an essential role in a cloud storage application but it is still tricky to realize keyword search and deduplication in the cloud simultaneously. Large-scale data usually renders significant data redundancy and saving cloud storage resources turns to be inevitable. Existing schemes only facilitate data retrieval due to keywords but rarely consider other demands like deduplication. To be noted, trivially and hastily affiliating a separate deduplication scheme to the searchable encryption leads to disordered system architecture and security threats. Therefore, attention should be paid to versatile solutions supporting both keyword search and deduplication in the cloud. The third problem to be addressed is implementing multi-reader access for PEKS. As we know, PEKS was born to support multi-writers but enabling multi-readers in PEKS is challenging. Repeatedly encrypting the same keyword with different readers’ keys is not an elegant solution. In addition to keyword privacy, user anonymity coming with a multi-reader setting should also be formulated and preserved. Last but not least, existing schemes targeting centralized storage have not taken full advantage of distributed computation, which is considerable efficiency and fast response. Specifically, all testing tasks between searchable ciphertexts and trapdoor/token are fully undertaken by the only centralized cloud server, resulting in a busy system and slow response. With the help of distributed techniques, we may now look forward to a new turnaround, i.e., multiple servers jointly work to perform the testing with better efficiency and scalability. Then the intractable multi-writer/multi-reader mode supporting multi-keyword queries may also come true as a by-product. This thesis investigates searchable encryption technologies in cloud storage and distributed systems and spares effort to address the problems mentioned above. Our first work can be classified into SSE. We formulate the Multi-user Verifiable Searchable Symmetric Encryption (MVSSE) and propose a concrete scheme for multi-user access. It not only offers multi-user access and verifiability but also supports extension on updates as well as a non-single keyword index. Moreover, revocable access control is obtained that the search authority is validated each time a query is launched, different from existing mechanisms that once the search authority is granted, users can search forever. We give simulation-based proof, demonstrating our proposal possesses Universally Composable (UC)-security. Second, we come up with a redundancy elimination solution on top of searchable encryption. Following the keyword comparison approach of SSE, we formulate a hybrid primitive called Message-Locked Searchable Encryption (MLSE) derived in the way of SSE’s keyword search supporting keyword search and deduplication and present a concrete construction that enables multi-keyword query and negative keyword query as well as deduplication at a considerable small cost, i.e., the tokens are used for both search and deduplication. And it can further support Proof of Storage (PoS), testifying the content integrity in cloud storage. The semantic security is proved in Random Oracle Model using the game-based methodology. Third, as the branch of PEKS, the Broadcast Authenticated Encryption with Keyword Search (BAEKS) is proposed to bridge the gap of multi-reader access for PEKS, followed by a scheme. It not only resists Keyword Guessing Attacks (KGA) but also fills in the blank of anonymity. The scheme is proved secure under Decisional Bilinear Diffie-Hellman (DBDH) assumption in the Random Oracle Model. For distributed systems, we present a Searchable Encryption based on Efficient Privacy-preserving Outsourced calculation framework with Multiple keys (SE-EPOM) enjoying desirable features, which can be classified into PEKS. Instead of merely deploying a single server, multiple servers are employed to execute the test algorithm in our scheme jointly. The refined search, i.e., multi-keyword query, data confidentiality, and search pattern hiding, are realized. Besides, the multi-writer/multi-reader mode comes true. It is shown that under the distributed circumstance, much efficiency can be substantially achieved by our construction. With simulation-based proof, the security of our scheme is elaborated. All constructions proposed in this thesis are formally proven according to their corresponding security definitions and requirements. In addition, for each cryptographic primitive designed in this thesis, concrete schemes are initiated to demonstrate the availability and practicality of our proposal

A Practical Framework for Storing and Searching Encrypted Data on Cloud Storage

Security has become a significant concern with the increased popularity of cloud storage services. It comes with the vulnerability of being accessed by third parties. Security is one of the major hurdles in the cloud server for the user when the user data that reside in local storage is outsourced to the cloud. It has given rise to security concerns involved in data confidentiality even after the deletion of data from cloud storage. Though, it raises a serious problem when the encrypted data needs to be shared with more people than the data owner initially designated. However, searching on encrypted data is a fundamental issue in cloud storage. The method of searching over encrypted data represents a significant challenge in the cloud. Searchable encryption allows a cloud server to conduct a search over encrypted data on behalf of the data users without learning the underlying plaintexts. While many academic SE schemes show provable security, they usually expose some query information, making them less practical, weak in usability, and challenging to deploy. Also, sharing encrypted data with other authorized users must provide each document's secret key. However, this way has many limitations due to the difficulty of key management and distribution. We have designed the system using the existing cryptographic approaches, ensuring the search on encrypted data over the cloud. The primary focus of our proposed model is to ensure user privacy and security through a less computationally intensive, user-friendly system with a trusted third party entity. To demonstrate our proposed model, we have implemented a web application called CryptoSearch as an overlay system on top of a well-known cloud storage domain. It exhibits secure search on encrypted data with no compromise to the user-friendliness and the scheme's functional performance in real-world applications.Comment: 146 Pages, Master's Thesis, 6 Chapters, 96 Figures, 11 Table

Protection of access privacy in un-trusted storage

With the emergence of cloud computing, outsourcing data services to cloud servers is becoming more and more prevalent. Along with this arise also security and privacy concerns. Particularly, it is an important concern to the user that the service provider itself may be malicious and breach the secrecy and privacy of users. Although encrypting data content has been a common practice, it does not relieve the concerns, because users data access pattern is not preserved and researchers have found that a wide range of private information could be conveniently revealed by observing the access pattern. It is, therefore, critical to investigate the problem of protecting users access pattern privacy in untrusted storage. Existing solutions that provide strict protection to the privacy of access pattern incur very high overhead, such as high bandwidth cost, long round-trip delay and/or large user side storage. The high overhead is a major barrier that hampers the adoption of these solutions in practice. Although strict protection of access pattern privacy is attractive, less strict protection, such as protecting the privacy of long-term access pattern, is also very useful in practice. Based on these considerations, we investigate the problem of protecting the long-term access pattern privacy in un-trusted storage and propose two light-weight schemes to preserve the privacy of long-term access pattern. We conduct rigorous proofs and extensive evaluations to demonstrate that the proposed schemes can hide the data access pattern in the long run, and the number of accesses required to preserve the access pattern privacy is reasonable in many situations. With outsourced data storage, keyword based query is a critical and primitive function for the users to access the data of their interest. Similar to access pattern, exposure of query pattern also leads to the leakage of sensitive information about the queried keyword and data. We show that without proper protection of both the query trapdoor and the access pattern, the query pattern may be exposed readily. Based on the framework of our access pattern privacy protection schemes, we propose a novel defense solution that protects the query privacy in a light-weight manner. One important benefit offered by cloud storage is its convenience for data sharing among multiple users. But when not all users are trusted, it is important to hold malicious users accountable for their misconduct. Due to conflicting goals of user accountability and access pattern privacy preservation, existing user accountability solutions cannot be readily integrated with ORAM constructions. As the last part of this dissertation, we investigate the problem of introducing support of user accountability into hash-based ORAM.We propose a scheme that can detect misconduct by malicious users and identify the attackers, while not interfering with the access pattern preservation mechanisms inherent from the underlying ORAM. Security and overhead analysis shows that the proposed scheme has achieved the design goals of providing accountability support to ORAM and preservation of data access pattern privacy, at the cost of slightly increased storage, communication, and computational overheads