On the design and optimization of heterogeneous distributed storage systems

Durant la última dècada, la demanda d’emmagatzematge de dades ha anat creixent exponencialment any rere any. Apart de demanar més capacitat d’emmagatzematge, el usuaris actualment també demanen poder accedir a les seves dades des de qualsevol lloc i des de qualsevol dispositiu. Degut a aquests nous requeriments, els usuaris estan actualment movent les seves dades personals (correus electrònics, documents, fotografies, etc.) cap a serveis d’emmagatzematge en línia com ara Gmail, Facebook, Flickr o Dropbox. Malauradament, aquests serveis d’emmagatzematge en línia estan sostinguts per unes grans infraestructures informàtiques que poques empreses poden finançar. Per tal de reduir el costs d’aquestes grans infraestructures informàtiques, ha sorgit una nova onada de serveis d’emmagatzematge en línia que obtenen grans infraestructures d’emmagatzematge a base d’integrar els recursos petits centres de dades, o fins i tot a base d’integrar els recursos d’emmagatzematge del usuaris finals. No obstant això, els recursos que formen aquestes noves infraestructures d’emmagatzematge són molt heterogenis, cosa que planteja un repte per al dissenyadors d’aquests sistemes: Com es poden dissenyar sistemes d’emmagatzematge en línia, fiables i eficients, quan la infraestructura emprada és tan heterogènia? Aquesta tesis presenta un estudi dels principals problemes que sorgeixen quan un vol respondre a aquesta pregunta. A més proporciona diferents eines per tal d’optimitzar el disseny de sistemes d’emmagatzematge distribuïts i heterogenis. Les principals contribucions són: Primer, creem un marc d’anàlisis per estudiar els efectes de la redundància de dades en el cost dels sistemes d’emmagatzematge distribuïts. Donat un esquema de redundància específic, el marc d’anàlisis presentat permet predir el cost mitjà d’emmagatzematge i el cost mitjà de comunicació d’un sistema d’emmagatzematge implementat sobre qualsevol infraestructura informàtica distribuïda. Segon, analitzem els impactes que la redundància de dades té en la disponibilitat de les dades, i en els temps de recuperació. Donada una redundància, i donat un sistema d’emmagatzematge heterogeni, creem un grup d’algorismes per a determinar la disponibilitat de les dades esperada, i els temps de recuperació esperats. Tercer, dissenyem diferents polítiques d’assignació de dades per a diferents sistemes d’emmagatzematge. Diferenciem entre aquells escenaris on la totalitat de la infraestructura està administrada per una sola organització, i els escenaris on diferents parts auto administrades contribueixen els seus recursos. Els objectius de les nostres polítiques d’assignació de dades són: (i) minimitzar la redundància necessària, (ii) garantir la equitat entre totes les parts que participen al sistema, i (iii) incentivar a les parts perquè contribueixin els seus recursos al sistema.Over the last decade, users’ storage demands have been growing exponentially year over year. Besides demanding more storage capacity and more data reliability, today users also demand the possibility to access their data from any location and from any device. These new needs encourage users to move their personal data (e.g., E-mails, documents, pictures, etc.) to online storage services such as Gmail, Facebook, Flickr or Dropbox. Unfortunately, these online storage services are built upon expensive large datacenters that only a few big enterprises can afford. To reduce the costs of these large datacenters, a new wave of online storage services has recently emerged integrating storage resources from different small datacenters, or even integrating user storage resources into the provider’s storage infrastructure. However, the storage resources that compose these new storage infrastructures are highly heterogeneous, which poses a challenging problem to storage systems designers: How to design reliable and efficient distributed storage systems over heterogeneous storage infrastructures? This thesis provides an analysis of the main problems that arise when one aims to answer this question. Besides that, this thesis provides different tools to optimize the design of heterogeneous distributed storage systems. The contribution of this thesis is threefold: First, we provide a novel framework to analyze the effects that data redundancy has on the storage and communication costs of distributed storage systems. Given a generic redundancy scheme, the presented framework can predict the average storage costs and the average communication costs of a storage system deployed over a specific storage infrastructure. Second, we analyze the impacts that data redundancy has on data availability and retrieval times. For a given redundancy and a heterogeneous storage infrastructure, we provide a set of algorithms that allow to determine the expected data availability and expected retrieval times. Third, we design different data assignment policies for different storage scenarios. We differentiate between scenarios where the entire storage infrastructure is managed by the same organization, and scenarios where different parties contribute their storage resources. The aims of our assignment policies are: (i) to minimize the required redundancy, (ii) to guarantee fairness among all parties, and (iii) to encourage different parties to contribute their local storage resources to the system

Contributions to routing scalability and QoS assurance in cloud data transport networks based on the recursive internetwork architecture

With an increasing number of devices and heterogeneous distributed applications, it is becoming evident that service delivered by the current Internet fall short to supply the actual Quality of Service (QoS) requirements of applications. In addition, the global scope of the IP layer causes large scalability problems on the network. Multiple solutions aim to overcome the limitations of the model (BGP, NAT, etc.), but all end being constrained by the same networking model that they try to improve, ending simply breaking and patching the stack itself of TCP/IP. In contrast, RINA proposes a new clean-slate Internet architecture based on a recursive networking stack with focus on inter process communication, where each layer, or DIF, performs the same set of tasks. DIFs are fully configurable by mean of programmable policies, and provide complete support for QoS services. RINA is capable to provide a standardized way to express the capabilities of each layer, the QoS Cubes. With those, RINA allows for applications and upper processes to express their requirements in terms of latency, losses, etc. The contributions in this thesis take profit from the recursive stack of RINA and the use of policies to propose and analyse old and new solutions which would not be compatible with the current TCP/IP Internet. Improving the QoS services, this work takes profit from the information on requirements provided by the applications themselves to improve the assurance of QoS. With the use of Q-based scheduling policies, improved QoS assurances are provided, aiming to provide “good enough” service for all flows in the network, resulting in a more appropriate sharing of resources. These policies have been tested in backbone-like networks, showing interesting improvements with respect to commonly used solutions like MPLS-based VPNs. In addition the provisioning of QoS services to end-users is also considered. In order to allow that, it is required to impose some limits on what end-users can send to the network, limiting the amount of priority traffic that potentially greedy users can send. In that regard, while enforcing strict rate-limits per QoS would be trivial in RINA, a new △Q-based rate-limiting policy that aims to limit the amount of priority traffic in a more user-friendly way is also explored. In terms of scalability, this work also considers different measures to improve forwarding and routing within large-scale networks. As for the use of policies that could profit from specific network topologies, a new forwarding policy, that mix both topological rules and exceptions, is proposed. With this policy, forwarding table lookups in large tables are replaced with fast and simple forwarding rules based on the location of nodes and their neighbourhood. Given the common topologies used in large data centres, the proposed policy is found to be a perfect match for those scenarios. Test for different data centre topology showed clear improvements, requiring only a small fraction of all forwarding information despite the large size of such networks, depending that in the number of concurrent failures in the network rather than on the size of it. In addition, this work also considers the use of topological routing policies to populate exceptions upon failures. The use of topological routing solutions resulted in reduced complexity for computing paths and less routing messages. In addition to topological solutions, the use other routing solution, not well suited for the IP environment are also investigated. Specifically, it is shown how a Landmark routing solution could be implemented within RINA. Finally, efforts are also devoted to analyse the importance of path selection for ensuring QoS requirements and how it is not required to reach extremes solutions, like the use of connections, to provide the required services.Con un número cada vez mayor de dispositivos y aplicaciones distribuidas, se está volviendo evidente que el servicio best-effort ofrecido por la actual Internet TCP/IP no satisface los requisitos de calidad de servicio (QoS) de las aplicaciones. No solo eso, sino que el alcance global de la capa de IP se convierte en la causa de grandes problemas de escalabilidad, requiriendo costes cada vez más altos para ser resueltos. Desde la implantación de TCP/IP, han aparecido múltiples soluciones que tienen como objetivo superar las limitaciones del modelo (BGP, NAT, LISP, etc.). Aun así, todas estas soluciones terminan restringidas por el mismo modelo de red que intentan mejorar. Dado esto, la mayoría de las soluciones terminan simplemente rompiendo y parcheando la pila misma de TCP/IP. Con el objetivo de resolver esos problemas, la Recursive InterNetwork Architecture (RINA) propone una nueva arquitectura de Internet que vuelve a las raíces de la comunicación en red. En lugar de parchear la pila actual de TCP/IP, RINA propone una pila de red recursiva con enfoque en la comunicación entre procesos, donde cada capa, llamada Distributed IPC Facility (DIF), realiza el mismo conjunto de tareas. Mientras realizan las mismas tareas, las DIF de RINA son completamente configurables por medio de políticas programables, definiciones de cómo realizar tales tareas. Además, RINA brinda soporte completo para servicios de QoS por medio de los Cubos QoS, o clases de QoS que definen las capacidades de cada DIF. Con el uso de los Cubos QoS, RINA es capaz de proporcionar una forma estandarizada de expresar las capacidades de cada capa. Además, dada esa información, RINA también permite que las aplicaciones y los procesos de capas superiores expresen sus requisitos de QoS en términos de latencia aceptada, pérdidas, uso promedio, etc. Las contribuciones en esta tesis sacan provecho de la pila recursiva de RINA y el uso de políticas para proponer y analizar soluciones, antiguas y nuevas, para QoS y escalabilidad, que no serán compatibles con la Internet TCP/IP actual. En términos de mejoras de los servicios de QoS, el trabajo en esta tesis aprovecha la información sobre los requisitos de flujo, proporcionados por las propias aplicaciones, para mejorar las garantías de QoS proporcionadas por la red. Propone el uso de políticas basadas en △Q, proporcionando garantías de QoS mejoradas, que coinciden mejor con los requisitos de los flujos. A diferencia de las soluciones de diferenciación de QoS más simples, donde los servicios de QoS se proporcionan en orden de prioridad, △Q pretende proporcionar un servicio “suficientemente bueno" para todos los flujos en la red, lo que resulta en una repartición de recursos más apropiada. En este trabajo, estas políticas se han probado en redes tipo backbone, que muestran mejoras interesantes con respecto a las soluciones comunes de diferenciación de QoS, como las VPN basadas en MPLS. Además del uso de las políticas de △Q en el núcleo de la red, esta tesis también considera el suministro de servicios de QoS a los usuarios finales, siendo ese el objetivo final de las redes. Para permitir eso, se requiere imponer algunos límites a lo que los usuarios finales pueden enviar a la red, con el fin de limitar la cantidad de tráfico prioritario que usuarios codiciosos puedan enviar. En ese sentido, aunque imponer límites de velocidad estrictos por QoS sería trivial en RINA, también se explora una nueva política de limitación de tasas basada en △Q que pretende limitar la cantidad de tráfico prioritario de una manera más beneficiosa para los usuarios. En términos de escalabilidad, esta tesis también considera diferentes medidas para mejorar el reenvío y el enrutamiento dentro de redes de gran escala. Primero, en cuanto al uso de políticas que podrán beneficiarse de topologías de red específicas, se propone una nueva política de forwarding que combina reglas topológicas, es decir decisiones basadas en la ubicación de nodos, y excepciones, es decir entradas que sobrescriben reglas en caso de error. Con esta política, las costosas búsquedas en tablas grandes se reemplazan con reglas de rápidas y simples basadas en la ubicación de los nodos y su vecindad. Dadas las topologías específicas más comúnmente utilizadas en los grandes centros de datos hoy en día, se encuentra que el uso de la política propuesta es la combinación perfecta para esos escenarios. Pruebas en varias topologías comunes para centros de datos mostraron mejoras claras, que requieren solo una pequeña fracción de toda la información sobre la red, a pesar del gran tamaño de dichas redes, dependiendo esta de la cantidad de fallas concurrentes en la red y no del tamaño de la misma. Además, esta tesis también considera el uso de políticas de enrutamiento topológico para poblar tales excepciones en caso de fallas. El uso de soluciones de enrutamiento topológico dio como resultado la reducción de la complejidad en el cálculo de rutas, junto con un menor número de mensajes de enrutamiento. Además de las soluciones topológicas, también se investiga el uso de otra solución de enrutamiento, no adecuada para el entorno de IP. Específicamente, se muestra como una solución de enrutamiento Landmark, una solución de enrutamiento de la familia de enrutamiento compacto, podría implementarse dentro de RINA. Finalmente, también se dedican esfuerzos a analizar la importancia de la selección de rutas para garantizar los requisitos de QoS y como no se requiere llegar a soluciones extremas, como el uso de conexiones, para proporcionar los servicios requeridos.Postprint (published version

A differentiated quality of service oriented multimedia multicast protocol

Les systèmes de communication multimédia modernes aspirent à fournir de nouveaux services tels que des communications multipoints. Néanmoins, l'apparition de dispositifs multimédias très diversifiés et le nombre croissant de clients ont révélé de nouveaux besoins pour les mécanismes et les protocoles. Dans une communication multimédia, les flux présentent des contraintes différentes et la QdS requise pour chaque flux n'est pas la même. De plus, dans une communication multipoint, tous les utilisateurs ne peuvent pas ou ne sont pas capables de recevoir la même QdS ; cette contrainte implique que les nouveaux mécanismes de communication doivent prendre en compte les besoins des utilisateurs pour fournir un service adéquat à chaque utilisateur, surtout pour éviter le gaspillage des ressources réseau. Cette thèse propose une architecture multipoint à QdS différentiée appelée M-FPTP. Basée sur des proxies client/serveur, elle relie plusieurs LANs multipoints à travers des liens point-à-point partiellement fiables. Cette architecture fournit une QdS différente à chaque LAN dépendant des besoins des utilisateurs. Pour ce faire, nous proposons un modèle du réseau appelé Arbre Hiérarchisé (AH) qui représente en même temps les performances du réseau et les contraintes de QdS des utilisateurs. Nonobstant, l'application de méthodes standard pour la création d'arbres sur un AH peut conduire à des problèmes de surcharge du degré de sortie dans la source. Pour résoudre ce problème, nous proposons alors un nouvel algorithme appelé Arbre de Plus Courts Chemins à Degré de Sortie Limité. Le déploiement de ce service nécessite, pour gérer les utilisateurs et le déploiement correct des proxies, un nouveau protocole appelé Protocole Simple de Session pour QdS multipoint. L'ensemble des solutions proposées a été modélisé, vérifié, validé et testé en utilisant UML 2.0 et l'outil TAU G2. ABSTRACT : Modern multimedia (MM) communication systems aim to provide new services such as multicast (MC) communication. But the rising of new very different MM capable devices and the growing number of clients drive to new requirements for mechanisms and protocols. In a MM communication, there are some flows that have constraints different from others and the required QoS for each flow is not the same. Furthermore, in MC communications, all the users do not want or are not able to receive the same QoS. These constraints imply that new communication mechanisms have to take into account the user requirements in order to provide an ad hoc service to each user and to avoid wasting the network resources. This dissertation proposes a new differentiated QoS multicast architecture, based on client/server proxies, called M-FPTP, which relays many MC LANs by single partially reliable links. This architecture provides a different QoS to each LAN depending on the users requirements. For doing so, it is also provided a network model called Hierarchized Graph (HG) which represents at the same time the network performances and the users QoS constraints. Nevertheless, the application of standard tree creation methods on an HG can lead to source overloading problems. It is then proposed a new algorithm called Degree-Bounded Shortest-Path-Tree (DgB-SPT) which solves this problem. However, the deployment of such a service needs a new protocol in order to collect users requirements and correctly deploy the proxies. This protocol is called Simple Session Protocol for QoS MC (SSP-QoM). The proposed solutions have been modeled, verified, validated and tested by using UML 2.0 and TAU G2 CASE tool

Economically sustainable public security and emergency network exploiting a broadband communications satellite

The research contributes to work in Rapid Deployment of a National Public Security and Emergency Communications Network using Communication Satellite Broadband. Although studies in Public Security Communication networks have examined the use of communications satellite as an integral part of the Communication Infrastructure, there has not been an in-depth design analysis of an optimized regional broadband-based communication satellite in relation to the envisaged service coverage area, with little or no terrestrial last-mile telecommunications infrastructure for delivery of satellite solutions, applications and services. As such, the research provides a case study of a Nigerian Public Safety Security Communications Pilot project deployed in regions of the African continent with inadequate terrestrial last mile infrastructure and thus requiring a robust regional Communications Satellite complemented with variants of terrestrial wireless technologies to bridge the digital hiatus as a short and medium term measure apart from other strategic needs. The research not only addresses the pivotal role of a secured integrated communications Public safety network for security agencies and emergency service organizations with its potential to foster efficient information symmetry amongst their operations including during emergency and crisis management in a timely manner but demonstrates a working model of how analogue spectrum meant for Push-to-Talk (PTT) services can be re-farmed and digitalized as a “dedicated” broadband-based public communications system. The network’s sustainability can be secured by using excess capacity for the strategic commercial telecommunication needs of the state and its citizens. Utilization of scarce spectrum has been deployed for Nigeria’s Cashless policy pilot project for financial and digital inclusion. This effectively drives the universal access goals, without exclusivity, in a continent, which still remains the least wired in the world

Service level agreement specification for IoT application workflow activity deployment, configuration and monitoring

PhD ThesisCurrently, we see the use of the Internet of Things (IoT) within various domains such as healthcare, smart homes, smart cars, smart-x applications, and smart cities. The number of applications based on IoT and cloud computing is projected to increase rapidly over the next few years. IoT-based services must meet the guaranteed levels of quality of service (QoS) to match users’ expectations. Ensuring QoS through specifying the QoS constraints using service level agreements (SLAs) is crucial. Also because of the potentially highly complex nature of multi-layered IoT applications, lifecycle management (deployment, dynamic reconfiguration, and monitoring) needs to be automated. To achieve this it is essential to be able to specify SLAs in a machine-readable format. currently available SLA specification languages are unable to accommodate the unique characteristics (interdependency of its multi-layers) of the IoT domain. Therefore, in this research, we propose a grammar for a syntactical structure of an SLA specification for IoT. The grammar is based on a proposed conceptual model that considers the main concepts that can be used to express the requirements for most common hardware and software components of an IoT application on an end-to-end basis. We follow the Goal Question Metric (GQM) approach to evaluate the generality and expressiveness of the proposed grammar by reviewing its concepts and their predefined lists of vocabularies against two use-cases with a number of participants whose research interests are mainly related to IoT. The results of the analysis show that the proposed grammar achieved 91.70% of its generality goal and 93.43% of its expressiveness goal. To enhance the process of specifying SLA terms, We then developed a toolkit for creating SLA specifications for IoT applications. The toolkit is used to simplify the process of capturing the requirements of IoT applications. We demonstrate the effectiveness of the toolkit using a remote health monitoring service (RHMS) use-case as well as applying a user experience measure to evaluate the tool by applying a questionnaire-oriented approach. We discussed the applicability of our tool by including it as a core component of two different applications: 1) a contextaware recommender system for IoT configuration across layers; and 2) a tool for automatically translating an SLA from JSON to a smart contract, deploying it on different peer nodes that represent the contractual parties. The smart contract is able to monitor the created SLA using Blockchain technology. These two applications are utilized within our proposed SLA management framework for IoT. Furthermore, we propose a greedy heuristic algorithm to decentralize workflow activities of an IoT application across Edge and Cloud resources to enhance response time, cost, energy consumption and network usage. We evaluated the efficiency of our proposed approach using iFogSim simulator. The performance analysis shows that the proposed algorithm minimized cost, execution time, networking, and Cloud energy consumption compared to Cloud-only and edge-ward placement approaches

Game Theory Relaunched

The game is on. Do you know how to play? Game theory sets out to explore what can be said about making decisions which go beyond accepting the rules of a game. Since 1942, a well elaborated mathematical apparatus has been developed to do so; but there is more. During the last three decades game theoretic reasoning has popped up in many other fields as well - from engineering to biology and psychology. New simulation tools and network analysis have made game theory omnipresent these days. This book collects recent research papers in game theory, which come from diverse scientific communities all across the world; they combine many different fields like economics, politics, history, engineering, mathematics, physics, and psychology. All of them have as a common denominator some method of game theory. Enjoy

Gerenciamento de nuvem computacional usando critérios de segurança

Orientador: Paulo Lício de GeusTese (doutorado) - Universidade Estadual de Campinas, Instituto de ComputaçãoResumo: A nuvem computacional introduziu novas tecnologias e arquiteturas, mudando a computação empresarial. Atualmente, um grande número de organizações optam por utilizar arquiteturas computacionais tradicionais por considerarem esta tecnologia não confiável, devido a problemas não resolvidos relacionados a segurança e privacidade. Em particular, quanto á contratação de um serviço na nuvem, um aspecto importante é a forma como as políticas de segurança serão aplicadas neste ambiente caracterizado pela virtualização e serviços em grande escala de multi-locação. Métricas de segurança podem ser vistas como ferramentas para fornecer informações sobre o estado do ambiente. Com o objetivo de melhorar a segurança na nuvem computacional, este trabalho apresenta uma metodologia para a gestão da nuvem computacional usando a segurança como um critério, através de uma arquitetura para monitoramento da segurança com base em acordos de níveis de serviço de segurança Security-SLA para serviços de IaaS, PaaS e SaaS, que usa métricas de segurançaAbstract: Cloud Computing has introduced new technology and architectures that changed enterprise computing. Currently, there is a large number of organizations that choose to stick to traditional architectures, since this technology is considered unreliable due to yet unsolved problems related to security and privacy. In particular, when hiring a service in the cloud, an important aspect is how security policies will be applied in this environment characterized by both virtualization and large-scale multi-tenancy service. Security metrics can be seen as tools to provide information about the status of the environment. Aimed at improving security in the Cloud Computing, this work presents a methodology for Cloud Computing management using security as a criterion, across an architecture for security monitoring based on Security-SLA for IaaS, PaaS and SaaS services using security metricsDoutoradoCiência da ComputaçãoDoutor em Ciência da Computação23/200.308/2009FUNDEC

Enabling Technology in Optical Fiber Communications: From Device, System to Networking

This book explores the enabling technology in optical fiber communications. It focuses on the state-of-the-art advances from fundamental theories, devices, and subsystems to networking applications as well as future perspectives of optical fiber communications. The topics cover include integrated photonics, fiber optics, fiber and free-space optical communications, and optical networking