Optimizations for Energy-Aware, High-Performance and Reliable Distributed Storage Systems

With the decreasing cost and wide-spread use of commodity hard drives, it has become possible to create very large-scale storage systems with less expense. However, as we approach exabyte-scale storage systems, maintaining important features such as energy-efficiency, performance, reliability and usability became increasingly difficult. Despite the decreasing cost of storage systems, the energy consumption of these systems still needs to be addressed in order to retain cost-effectiveness. Any improvements in a storage system can be outweighed by high energy costs. On the other hand, large-scale storage systems can benefit more from the object storage features for improved performance and usability. One area of concern is metadata performance bottleneck of applications reading large directories or creating a large number of files. Similarly, computation on big data where data needs to be transferred between compute and storage clusters adversely affects I/O performance. As the storage systems become more complex and larger, transferring data between remote compute and storage tiers becomes impractical. Furthermore, storage systems implement reliability typically at the file system or client level. This approach might not always be practical in terms of performance. Lastly, object storage features are usually tailored to specific use cases that makes it harder to use them in various contexts. In this thesis, we are presenting several approaches to enhance energy-efficiency, performance, reliability and usability of large-scale storage systems. To begin with, we improve the energy-efficiency of storage systems by moving I/O load to a subset of the storage nodes with energy-aware node allocation methods and turn off the unused nodes, while preserving load balance on demand. To address the metadata performance issue associated with large creates and directory reads, we represent directories with object storage collections and implement lazy creation of objects. Similarly, in-situ computation on large-scale data is enabled by using object storage features to integrate a computational framework with the existing object storage layer to eliminate the need to transfer data between compute and storage silos for better performance. We then present parity-based redundancy using object storage features to achieve reliability with less performance impact. Finally, unified storage brings together the object storage features to meet the needs of distinct use cases; such as cloud storage, big data or high-performance computing to alleviate the unnecessary fragmentation of storage resources. We evaluate each proposed approach thoroughly and validate their effectiveness in terms of improving energy-efficiency, performance, reliability and usability of a large-scale storage system

Service Provisioning in Edge-Cloud Continuum Emerging Applications for Mobile Devices

Disruptive applications for mobile devices can be enhanced by Edge computing facilities. In this context, Edge Computing (EC) is a proposed architecture to meet the mobility requirements imposed by these applications in a wide range of domains, such as the Internet of Things, Immersive Media, and Connected and Autonomous Vehicles. EC architecture aims to introduce computing capabilities in the path between the user and the Cloud to execute tasks closer to where they are consumed, thus mitigating issues related to latency, context awareness, and mobility support. In this survey, we describe which are the leading technologies to support the deployment of EC infrastructure. Thereafter, we discuss the applications that can take advantage of EC and how they were proposed in the literature. Finally, after examining enabling technologies and related applications, we identify some open challenges to fully achieve the potential of EC, and also research opportunities on upcoming paradigms for service provisioning. This survey is a guide to comprehend the recent advances on the provisioning of mobile applications, as well as foresee the expected next stages of evolution for these applications

A monitoring and threat detection system using stream processing as a virtual function for big data

The late detection of security threats causes a significant increase in the risk of irreparable damages, disabling any defense attempt. As a consequence, fast realtime threat detection is mandatory for security guarantees. In addition, Network Function Virtualization (NFV) provides new opportunities for efficient and low-cost security solutions. We propose a fast and efficient threat detection system based on stream processing and machine learning algorithms. The main contributions of this work are i) a novel monitoring threat detection system based on stream processing; ii) two datasets, first a dataset of synthetic security data containing both legitimate and malicious traffic, and the second, a week of real traffic of a telecommunications operator in Rio de Janeiro, Brazil; iii) a data pre-processing algorithm, a normalizing algorithm and an algorithm for fast feature selection based on the correlation between variables; iv) a virtualized network function in an open-source platform for providing a real-time threat detection service; v) near-optimal placement of sensors through a proposed heuristic for strategically positioning sensors in the network infrastructure, with a minimum number of sensors; and, finally, vi) a greedy algorithm that allocates on demand a sequence of virtual network functions.A detecção tardia de ameaças de segurança causa um significante aumento no risco de danos irreparáveis, impossibilitando qualquer tentativa de defesa. Como consequência, a detecção rápida de ameaças em tempo real é essencial para a administração de segurança. Além disso, A tecnologia de virtualização de funções de rede (Network Function Virtualization - NFV) oferece novas oportunidades para soluções de segurança eficazes e de baixo custo. Propomos um sistema de detecção de ameaças rápido e eficiente, baseado em algoritmos de processamento de fluxo e de aprendizado de máquina. As principais contribuições deste trabalho são: i) um novo sistema de monitoramento e detecção de ameaças baseado no processamento de fluxo; ii) dois conjuntos de dados, o primeiro ´e um conjunto de dados sintético de segurança contendo tráfego suspeito e malicioso, e o segundo corresponde a uma semana de tráfego real de um operador de telecomunicações no Rio de Janeiro, Brasil; iii) um algoritmo de pré-processamento de dados composto por um algoritmo de normalização e um algoritmo para seleção rápida de características com base na correlação entre variáveis; iv) uma função de rede virtualizada em uma plataforma de código aberto para fornecer um serviço de detecção de ameaças em tempo real; v) posicionamento quase perfeito de sensores através de uma heurística proposta para posicionamento estratégico de sensores na infraestrutura de rede, com um número mínimo de sensores; e, finalmente, vi) um algoritmo guloso que aloca sob demanda uma sequencia de funções de rede virtual

Mobility-aware Software-Defined Service-Centric Networking for Service Provisioning in Urban Environments

Disruptive applications for mobile devices, such as the Internet of Things, Connected and Autonomous Vehicles, Immersive Media, and others, have requirements that the current Cloud Computing paradigm cannot meet. These unmet requirements bring the necessity to deploy geographically distributed computing architectures, such as Fog and Mobile Edge Computing. However, bringing computing close to users has its costs. One example of cost is the complexity introduced by the management of the mobility of the devices at the edge. This mobility may lead to issues, such as interruption of the communication with service instances hosted at the edge or an increase in communication latency during mobility events, e.g., handover. These issues, caused by the lack of mobility-aware service management solutions, result in degradation in service provisioning. The present thesis proposes a series of protocols and algorithms to handle user and service mobility at the edge of the network. User mobility is characterized when user change access points of wireless networks, while service mobility happens when services have to be provisioned from different hosts. It assembles them in a solution for mobility-aware service orchestration based on Information-Centric Networking (ICN) and runs on top of Software-Defined Networking (SDN). This solution addresses three issues related to handling user mobility at the edge: (i) proactive support for user mobility events, (ii) service instance addressing management, and (iii) distributed application state data management. For (i), we propose a proactive SDN-based handover scheme. For (ii), we propose an ICN addressing strategy to remove the necessity of updating addresses after service mobility events. For (iii), we propose a graph-based framework for state data placement in the network nodes that accounts for user mobility and latency requirements. The protocols and algorithms proposed in this thesis were compared with different approaches from the literature through simulation. Our results show that the proposed solution can reduce service interruption and latency in the presence of user and service mobility events while maintaining reasonable overhead costs regarding control messages sent in the network by the SDN controller