Scheduling of data-intensive workloads in a brokered virtualized environment

Providing performance predictability guarantees is increasingly important in cloud platforms, especially for data-intensive applications, for which performance depends greatly on the available rates of data transfer between the various computing/storage hosts underlying the virtualized resources assigned to the application. With the increased prevalence of brokerage services in cloud platforms, there is a need for resource management solutions that consider the brokered nature of these workloads, as well as the special demands of their intra-dependent components. In this paper, we present an offline mechanism for scheduling batches of brokered data-intensive workloads, which can be extended to an online setting. The objective of the mechanism is to decide on a packing of the workloads in a batch that minimizes the broker's incurred costs, Moreover, considering the brokered nature of such workloads, we define a payment model that provides incentives to these workloads to be scheduled as part of a batch, which we analyze theoretically. Finally, we evaluate the proposed scheduling algorithm, and exemplify the fairness of the payment model in practical settings via trace-based experiments

A Taxonomy of Data Grids for Distributed Data Sharing, Management and Processing

Data Grids have been adopted as the platform for scientific communities that need to share, access, transport, process and manage large data collections distributed worldwide. They combine high-end computing technologies with high-performance networking and wide-area storage management techniques. In this paper, we discuss the key concepts behind Data Grids and compare them with other data sharing and distribution paradigms such as content delivery networks, peer-to-peer networks and distributed databases. We then provide comprehensive taxonomies that cover various aspects of architecture, data transportation, data replication and resource allocation and scheduling. Finally, we map the proposed taxonomy to various Data Grid systems not only to validate the taxonomy but also to identify areas for future exploration. Through this taxonomy, we aim to categorise existing systems to better understand their goals and their methodology. This would help evaluate their applicability for solving similar problems. This taxonomy also provides a "gap analysis" of this area through which researchers can potentially identify new issues for investigation. Finally, we hope that the proposed taxonomy and mapping also helps to provide an easy way for new practitioners to understand this complex area of research.Comment: 46 pages, 16 figures, Technical Repor

Publish/subscribe protocol in wireless sensor networks: improved reliability and timeliness

The rapidly-evolving demand of applications using wireless sensor networks in several areas such as building and industrial automation or smart cities, among other, makes it necessary to determine and provide QoS support mechanisms which can satisfy the requirements of applications. In this paper we propose a mechanism that establishes different QoS levels, based on Publish/Subscribe model for wireless networks to meet application requirements, to provide reliable delivery of packet and timeliness. The first level delivers packets in a best effort way. The second one intends to provide reliable packet delivery with a novel approach for Retransmission Timeout (RTO) calculation, which adjusts the RTO depending on the subscriber Packet Delivery Ratio (PDR). The third one provides the same reliable packet delivery as the second one, but in addition, it provides data aggregation trying to be efficient in terms of energy consumption and the use of network bandwidth. The last one provides timeliness in the packet delivery. We evaluate each QoS Level with several performance metrics such as PDR, Message Delivery Ratio, Duplicated and Retransmitted Packet Ratio and Packet Timeliness Ratio to demonstrate that our proposal provides significant improvements based on the increase of the PDR obtained.Peer ReviewedPostprint (author's final draft

New Challenges in Quality of Services Control Architectures in Next Generation Networks

A mesura que Internet i les xarxes IP s'han anat integrant dins la societat i les corporacions, han anat creixent les expectatives de nous serveis convergents així com les expectatives de qualitat en les comunicacions. Les Next Generation Networks (NGN) donen resposta a les noves necessitats i representen el nou paradigma d'Internet a partir de la convergència IP. Un dels aspectes menys desenvolupats de les NGN és el control de la Qualitat del Servei (QoS), especialment crític en les comunicacions multimèdia a través de xarxes heterogènies i/o de diferents operadors. A més a més, les NGN incorporen nativament el protocol IPv6 que, malgrat les deficiències i esgotament d'adreces IPv4, encara no ha tingut l'impuls definitiu.Aquesta tesi està enfocada des d'un punt de vista pràctic. Així doncs, per tal de poder fer recerca sobre xarxes de proves (o testbeds) que suportin IPv6 amb garanties de funcionament, es fa un estudi en profunditat del protocol IPv6, del seu grau d'implementació i dels tests de conformància i interoperabilitat existents que avaluen la qualitat d'aquestes implementacions. A continuació s'avalua la qualitat de cinc sistemes operatius que suporten IPv6 mitjançant un test de conformància i s'implementa el testbed IPv6 bàsic, a partir del qual es farà la recerca, amb la implementació que ofereix més garanties.El QoS Broker és l'aportació principal d'aquesta tesi: un marc integrat que inclou un sistema automatitzat per gestionar el control de la QoS a través de sistemes multi-domini/multi-operador seguint les recomanacions de les NGN. El sistema automatitza els mecanismes associats a la configuració de la QoS dins d'un mateix domini (sistema autònom) mitjançant la gestió basada en polítiques de QoS i automatitza la negociació dinàmica de QoS entre QoS Brokers de diferents dominis, de forma que permet garantir QoS extrem-extrem sense fissures. Aquesta arquitectura es valida sobre un testbed de proves multi-domini que utilitza el mecanisme DiffServ de QoS i suporta IPv6.L'arquitectura definida en les NGN permet gestionar la QoS tant a nivell 3 (IP) com a nivell 2 (Ethernet, WiFi, etc.) de forma que permet gestionar també xarxes PLC. Aquesta tesi proposa una aproximació teòrica per aplicar aquesta arquitectura de control, mitjançant un QoS Broker, a les noves xarxes PLC que s'estan acabant d'estandarditzar, i discuteix les possibilitats d'aplicació sobre les futures xarxes de comunicació de les Smart Grids.Finalment, s'integra en el QoS Broker un mòdul per gestionar l'enginyeria del tràfic optimitzant els dominis mitjançant tècniques de intel·ligència artificial. La validació en simulacions i sobre un testbed amb routers Cisco demostra que els algorismes genètics híbrids són una opció eficaç en aquest camp.En general, les observacions i avenços assolits en aquesta tesi contribueixen a augmentar la comprensió del funcionament de la QoS en les NGN i a preparar aquests sistemes per afrontar problemes del món real de gran complexitat.A medida que Internet y las redes IP se han ido integrando dentro de la sociedad y las corporaciones, han ido creciendo las expectativas de nuevos servicios convergentes así como las expectativas de calidad en las comunicaciones. Las Next Generation Networks (NGN) dan respuesta a las nuevas necesidades y representan el nuevo paradigma de Internet a partir de la convergencia IP. Uno de los aspectos menos desarrollados de las NGN es el control de la Calidad del Servicio (QoS), especialmente crítico en las comunicaciones multimedia a través de redes heterogéneas y/o de diferentes operadores. Además, las NGN incorporan nativamente el protocolo IPv6 que, a pesar de las deficiencias y agotamiento de direcciones IPv4, aún no ha tenido el impulso definitivo.Esta tesis está enfocada desde un punto de vista práctico. Así pues, con tal de poder hacer investigación sobre redes de prueba (o testbeds) que suporten IPv6 con garantías de funcionamiento, se hace un estudio en profundidad del protocolo IPv6, de su grado de implementación y de los tests de conformancia e interoperabilidad existentes que evalúan la calidad de estas implementaciones. A continuación se evalua la calidad de cinco sistemas operativos que soportan IPv6 mediante un test de conformancia y se implementa el testbed IPv6 básico, a partir del cual se realizará la investigación, con la implementación que ofrece más garantías.El QoS Broker es la aportación principal de esta tesis: un marco integrado que incluye un sistema automatitzado para gestionar el control de la QoS a través de sistemas multi-dominio/multi-operador siguiendo las recomendaciones de las NGN. El sistema automatiza los mecanismos asociados a la configuración de la QoS dentro de un mismo dominio (sistema autónomo) mediante la gestión basada en políticas de QoS y automatiza la negociación dinámica de QoS entre QoS brokers de diferentes dominios, de forma que permite garantizar QoS extremo-extremo sin fisuras. Esta arquitectura se valida sobre un testbed de pruebas multi-dominio que utiliza el mecanismo DiffServ de QoS y soporta IPv6. La arquitectura definida en las NGN permite gestionar la QoS tanto a nivel 3 (IP) o como a nivel 2 (Ethernet, WiFi, etc.) de forma que permite gestionar también redes PLC. Esta tesis propone una aproximación teórica para aplicar esta arquitectura de control, mediante un QoS Broker, a las noves redes PLC que se están acabando de estandardizar, y discute las posibilidades de aplicación sobre las futuras redes de comunicación de las Smart Grids.Finalmente, se integra en el QoS Broker un módulo para gestionar la ingeniería del tráfico optimizando los dominios mediante técnicas de inteligencia artificial. La validación en simulaciones y sobre un testbed con routers Cisco demuestra que los algoritmos genéticos híbridos son una opción eficaz en este campo.En general, las observaciones y avances i avances alcanzados en esta tesis contribuyen a augmentar la comprensión del funcionamiento de la QoS en las NGN y en preparar estos sistemas para afrontar problemas del mundo real de gran complejidad.The steady growth of Internet along with the IP networks and their integration into society and corporations has brought with it increased expectations of new converged services as well as greater demands on quality in communications. The Next Generation Networks (NGNs) respond to these new needs and represent the new Internet paradigm from the IP convergence. One of the least developed aspects in the NGNs is the Quality of Service (QoS) control, which is especially critical in the multimedia communication through heterogeneous networks and/or different operators. Furthermore, the NGNs natively incorporate the IPv6 protocol which, despite its shortcomings and the depletion of IPv4 addresses has not been boosted yet.This thesis has been developed with a practical focus. Therefore, with the aim of carrying out research over testbeds supporting the IPv6 with performance guarantees, an in-depth study of the IPv6 protocol development has been conducted and its degree of implementation and the existing conformance and interoperability tests that evaluate these implementations have been studied. Next, the quality of five implementations has been evaluated through a conformance test and the basic IPv6 testbed has been implemented, from which the research will be carried out. The QoS Broker is the main contribution to this thesis: an integrated framework including an automated system for QoS control management through multi-domain/multi-operator systems according to NGN recommendations. The system automates the mechanisms associated to the QoS configuration inside the same domain (autonomous system) through policy-based management and automates the QoS dynamic negotiation between peer QoS Brokers belonging to different domains, so it allows the guarantee of seamless end-to-end QoS. This architecture is validated over a multi-domain testbed which uses the QoS DiffServ mechanism and supports IPv6.The architecture defined in the NGN allows QoS management at level 3 (IP) as well as at level 2 (e.g. Ethernet, WiFi) so it also facilitates the management of PLC networks. Through the use of a QoS Broker, this thesis proposes a theoretical approach for applying this control architecture to the newly standardized PLC networks, and discusses the possibilities of applying it over the future communication networks of the Smart Grids.Finally, a module for managing traffic engineering which optimizes the network domains through artificial intelligence techniques is integrated in the QoS Broker. The validations by simulations and over a Cisco router testbed demonstrate that hybrid genetic algorithms are an effective option in this area.Overall, the advances and key insights provided in this thesis help advance our understanding of QoS functioning in the NGNs and prepare these systems to face increasingly complex problems, which abound in current industrial and scientific applications

InterCloud: Utility-Oriented Federation of Cloud Computing Environments for Scaling of Application Services

Cloud computing providers have setup several data centers at different geographical locations over the Internet in order to optimally serve needs of their customers around the world. However, existing systems do not support mechanisms and policies for dynamically coordinating load distribution among different Cloud-based data centers in order to determine optimal location for hosting application services to achieve reasonable QoS levels. Further, the Cloud computing providers are unable to predict geographic distribution of users consuming their services, hence the load coordination must happen automatically, and distribution of services must change in response to changes in the load. To counter this problem, we advocate creation of federated Cloud computing environment (InterCloud) that facilitates just-in-time, opportunistic, and scalable provisioning of application services, consistently achieving QoS targets under variable workload, resource and network conditions. The overall goal is to create a computing environment that supports dynamic expansion or contraction of capabilities (VMs, services, storage, and database) for handling sudden variations in service demands. This paper presents vision, challenges, and architectural elements of InterCloud for utility-oriented federation of Cloud computing environments. The proposed InterCloud environment supports scaling of applications across multiple vendor clouds. We have validated our approach by conducting a set of rigorous performance evaluation study using the CloudSim toolkit. The results demonstrate that federated Cloud computing model has immense potential as it offers significant performance gains as regards to response time and cost saving under dynamic workload scenarios.Comment: 20 pages, 4 figures, 3 tables, conference pape

Advances in Grid Computing

This book approaches the grid computing with a perspective on the latest achievements in the field, providing an insight into the current research trends and advances, and presenting a large range of innovative research papers. The topics covered in this book include resource and data management, grid architectures and development, and grid-enabled applications. New ideas employing heuristic methods from swarm intelligence or genetic algorithm and quantum encryption are considered in order to explain two main aspects of grid computing: resource management and data management. The book addresses also some aspects of grid computing that regard architecture and development, and includes a diverse range of applications for grid computing, including possible human grid computing system, simulation of the fusion reaction, ubiquitous healthcare service provisioning and complex water systems

Preliminary specification and design documentation for software components to achieve catallaxy in computational systems

This Report is about the preliminary specifications and design documentation for software components to achieve Catallaxy in computational systems. -- Die Arbeit beschreibt die Spezifikation und das Design von Softwarekomponenten, um das Konzept der Katallaxie in Grid Systemen umzusetzen. Eine Einführung ordnet das Konzept der Katallaxie in bestehende Grid Taxonomien ein und stellt grundlegende Komponenten vor. Anschließend werden diese Komponenten auf ihre Anwendbarkeit in bestehenden Application Layer Netzwerken untersucht.Grid Computing