A new MDA-SOA based framework for intercloud interoperability

Cloud computing has been one of the most important topics in Information Technology which aims to assure scalable and reliable on-demand services over the Internet. The expansion of the application scope of cloud services would require cooperation between clouds from different providers that have heterogeneous functionalities. This collaboration between different cloud vendors can provide better Quality of Services (QoS) at the lower price. However, current cloud systems have been developed without concerns of seamless cloud interconnection, and actually they do not support intercloud interoperability to enable collaboration between cloud service providers. Hence, the PhD work is motivated to address interoperability issue between cloud providers as a challenging research objective. This thesis proposes a new framework which supports inter-cloud interoperability in a heterogeneous computing resource cloud environment with the goal of dispatching the workload to the most effective clouds available at runtime. Analysing different methodologies that have been applied to resolve various problem scenarios related to interoperability lead us to exploit Model Driven Architecture (MDA) and Service Oriented Architecture (SOA) methods as appropriate approaches for our inter-cloud framework. Moreover, since distributing the operations in a cloud-based environment is a nondeterministic polynomial time (NP-complete) problem, a Genetic Algorithm (GA) based job scheduler proposed as a part of interoperability framework, offering workload migration with the best performance at the least cost. A new Agent Based Simulation (ABS) approach is proposed to model the inter-cloud environment with three types of agents: Cloud Subscriber agent, Cloud Provider agent, and Job agent. The ABS model is proposed to evaluate the proposed framework.Fundação para a Ciência e a Tecnologia (FCT) - (Referencia da bolsa: SFRH SFRH / BD / 33965 / 2009) and EC 7th Framework Programme under grant agreement n° FITMAN 604674 (http://www.fitman-fi.eu

An IoT Cloud and Big Data Architecture for the Maintenance of Home Appliances

Billions of interconnected Internet of Things (IoT) sensors and devices collect tremendous amounts of data from real-world scenarios. Big data is generating increasing interest in a wide range of industries. Once data is analyzed through compute-intensive Machine Learning (ML) methods, it can derive critical business value for organizations. Powerfulplatforms are essential to handle and process such massive collections of information cost-effectively and conveniently. This work introduces a distributed and scalable platform architecture that can be deployed for efficient real-world big data collection and analytics. The proposed system was tested with a case study for Predictive Maintenance of Home Appliances, where current and vibration sensors with high acquisition frequency were connected to washing machines and refrigerators. The introduced platform was used to collect, store, and analyze the data. The experimental results demonstrated that the presented system could be advantageous for tackling real-world IoT scenarios in a cost-effective and local approach.Comment: 6 pages, 6 figures, IECON 202

DEPAS: A Decentralized Probabilistic Algorithm for Auto-Scaling

The dynamic provisioning of virtualized resources offered by cloud computing infrastructures allows applications deployed in a cloud environment to automatically increase and decrease the amount of used resources. This capability is called auto-scaling and its main purpose is to automatically adjust the scale of the system that is running the application to satisfy the varying workload with minimum resource utilization. The need for auto-scaling is particularly important during workload peaks, in which applications may need to scale up to extremely large-scale systems. Both the research community and the main cloud providers have already developed auto-scaling solutions. However, most research solutions are centralized and not suitable for managing large-scale systems, moreover cloud providers' solutions are bound to the limitations of a specific provider in terms of resource prices, availability, reliability, and connectivity. In this paper we propose DEPAS, a decentralized probabilistic auto-scaling algorithm integrated into a P2P architecture that is cloud provider independent, thus allowing the auto-scaling of services over multiple cloud infrastructures at the same time. Our simulations, which are based on real service traces, show that our approach is capable of: (i) keeping the overall utilization of all the instantiated cloud resources in a target range, (ii) maintaining service response times close to the ones obtained using optimal centralized auto-scaling approaches.Comment: Submitted to Springer Computin

Cloudbus Toolkit for Market-Oriented Cloud Computing

This keynote paper: (1) presents the 21st century vision of computing and identifies various IT paradigms promising to deliver computing as a utility; (2) defines the architecture for creating market-oriented Clouds and computing atmosphere by leveraging technologies such as virtual machines; (3) provides thoughts on market-based resource management strategies that encompass both customer-driven service management and computational risk management to sustain SLA-oriented resource allocation; (4) presents the work carried out as part of our new Cloud Computing initiative, called Cloudbus: (i) Aneka, a Platform as a Service software system containing SDK (Software Development Kit) for construction of Cloud applications and deployment on private or public Clouds, in addition to supporting market-oriented resource management; (ii) internetworking of Clouds for dynamic creation of federated computing environments for scaling of elastic applications; (iii) creation of 3rd party Cloud brokering services for building content delivery networks and e-Science applications and their deployment on capabilities of IaaS providers such as Amazon along with Grid mashups; (iv) CloudSim supporting modelling and simulation of Clouds for performance studies; (v) Energy Efficient Resource Allocation Mechanisms and Techniques for creation and management of Green Clouds; and (vi) pathways for future research.Comment: 21 pages, 6 figures, 2 tables, Conference pape

Improving data center efficiency through smart grid integration and intelligent analytics

The ever-increasing growth of the demand in IT computing, storage and large-scale cloud services leads to the proliferation of data centers that consist of (tens of) thousands of servers. As a result, data centers are now among the largest electricity consumers worldwide. Data center energy and resource efficiency has started to receive significant attention due to its economical, environmental, and performance impacts. In tandem, facing increasing challenges in stabilizing the power grids due to growing needs of intermittent renewable energy integration, power market operators have started to offer a number of demand response (DR) opportunities for energy consumers (such as data centers) to receive credits by modulating their power consumption dynamically following specific requirements. This dissertation claims that data centers have strong capabilities to emerge as major enablers of substantial electricity integration from renewables. The participation of data centers into emerging DR, such as regulation service reserves (RSRs), enables the growth of the data center in a sustainable, environmentally neutral, or even beneficial way, while also significantly reducing data center electricity costs. In this dissertation, we first model data center participation in DR, and then propose runtime policies to dynamically modulate data center power in response to independent system operator (ISO) requests, leveraging advanced server power and workload management techniques. We also propose energy and reserve bidding strategies to minimize the data center energy cost. Our results demonstrate that a typical data center can achieve up to 44% monetary savings in its electricity cost with RSR provision, dramatically surpassing savings achieved by traditional energy management strategies. In addition, we investigate the capabilities and benefits of various types of energy storage devices (ESDs) in DR. Finally, we demonstrate RSR provision in practice on a real server. In addition to its contributions on improving data center energy efficiency, this dissertation also proposes a novel method to address data center management efficiency. We propose an intelligent system analytics approach, "discovery by example", which leverages fingerprinting and machine learning methods to automatically discover software and system changes. Our approach eases runtime data center introspection and reduces the cost of system management.2018-11-04T00:00:00

Personal mobile grids with a honeybee inspired resource scheduler

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.The overall aim of the thesis has been to introduce Personal Mobile Grids (PMGrids) as a novel paradigm in grid computing that scales grid infrastructures to mobile devices and extends grid entities to individual personal users. In this thesis, architectural designs as well as simulation models for PM-Grids are developed. The core of any grid system is its resource scheduler. However, virtually all current conventional grid schedulers do not address the non-clairvoyant scheduling problem, where job information is not available before the end of execution. Therefore, this thesis proposes a honeybee inspired resource scheduling heuristic for PM-Grids (HoPe) incorporating a radical approach to grid resource scheduling to tackle this problem. A detailed design and implementation of HoPe with a decentralised self-management and adaptive policy are initiated. Among the other main contributions are a comprehensive taxonomy of grid systems as well as a detailed analysis of the honeybee colony and its nectar acquisition process (NAP), from the resource scheduling perspective, which have not been presented in any previous work, to the best of our knowledge. PM-Grid designs and HoPe implementation were evaluated thoroughly through a strictly controlled empirical evaluation framework with a well-established heuristic in high throughput computing, the opportunistic scheduling heuristic (OSH), as a benchmark algorithm. Comparisons with optimal values and worst bounds are conducted to gain a clear insight into HoPe behaviour, in terms of stability, throughput, turnaround time and speedup, under different running conditions of number of jobs and grid scales. Experimental results demonstrate the superiority of HoPe performance where it has successfully maintained optimum stability and throughput in more than 95% of the experiments, with HoPe achieving three times better than the OSH under extremely heavy loads. Regarding the turnaround time and speedup, HoPe has effectively achieved less than 50% of the turnaround time incurred by the OSH, while doubling its speedup in more than 60% of the experiments. These results indicate the potential of both PM-Grids and HoPe in realising futuristic grid visions. Therefore considering the deployment of PM-Grids in real life scenarios and the utilisation of HoPe in other parallel processing and high throughput computing systems are recommended

Managing Mobility for Distributed Smart Cities Services

The IoT refers to the idea of internetworking physical devices, vehicles, buildings, and any other item embedded with the appropriate electronics, software, sensors, actuators, and network connectivity to allows them to interchange data and to provide highly effective new services. In this thesis we focus on the communications issues of the IoT in relation to mobility and we provide different solutions to alleviate the impact of these potential problems and to guarantee the information delivery in mobile scenarios. Our reference context is a Smart City where various mobile devices collaboratively participate, periodically sending information from their sensors. We assume that these services are located in platforms based in cloud infrastructures where the information is protected through the use of virtualisation ensuring their security and privacy. This thesis is structured into seven chapters. We first detail our objectives and identify the current problems we intend to address. Next, we provide a thorough review of the state of the art of all the areas involved in our work, highlighting how we improved the existing solutions with our research. The overall approach of the solutions we propose in this thesis use prototypes that encompasses and integrates different technologies and standards in a small infrastructure, using real devices in real scenarios with two of the most commonly used networks around the world: WiFi and 802.15.4 to efficiently solve the problems we originally identified. We focussed on protocols based on a producer/consumer paradigm, namely AMQP and particularly MQTT. We observed the behaviour of these protocols using in lab experiments and in external environments, using a mesh wireless network as the backbone network. Various issues raised by mobility were taken into consideration, and thus, we repeated the tests with different messages sizes and different inter-message periodicity, in order to model different possible applications. We also present a model for dimensioning the number of sources for mobile nodes and calculating the number of buffers required in the mobile node as a function of the number of sources and the size of the messages. We included a mechanism for avoiding data loss based on intermediate buffering adapted to the MQTT protocol that, in conjunction with the use of an alternative to the Network Manager in certain contexts, improves the connection establishment for wireless mobile clients. We also performed a detailed study of the jitter behaviour of a mobile node when transmitting messages with this proposal while moving through a real outdoor scenario. To emulate simple IoT networks we used the Cooja simulator to study and determine the effects on the probability of delivering messages when both publishers and subscribers were added to different scenarios. Finally we present an approach that combines the MQTT protocol with DTN which we specifically designed for constrained environments and guarantees that important information will never be lost. The advantage of our proposed solutions is that they make an IoT system more resilient to changes in the point of attachment of the mobile devices in an IoT network without requiring IoT application & service developers to explicitly consider this issue. Moreover, our solutions do not require additional support from the network through protocols such as MobileIP or LISP. We close the thesis by providing some conclusions, and identifying future lines of work which we unable to address here.Internet de las cosas (IoT) se refiere a la idea de interconectar sensores, actuadores, dispositivos físicos, vehículos, edificios y cualquier elemento dotado de la electrónica, así como del software y de la conectividad de red que los hace capaces de intercambiar datos para proporcionar servicios altamente efectivos. En esta tesis nos centramos en temas relacionados con la comunicación de sistemas IoT, específicamente en situaciones de movilidad y en los problemas que esto conlleva. Con este fin ofrecemos diferentes soluciones que alivian su impacto y garantizan la entrega de información en estas situaciones. El contexto de referencia es una ciudad inteligente donde varios dispositivos móviles participan de forma colaborativa enviando periódicamente información desde sus sensores hacia servicios ubicados en plataformas en la nube (cloud computing) donde mediante el uso de virtualización, la información está protegida garantizando su seguridad y privacidad. Las soluciones propuestas en esta tesis se enfocan en probar sobre una pequeña infraestructura un prototipo que abarca e integra diferentes tecnologías y estándares para resolver eficientemente los problemas previamente identificados. Hemos enfocado nuestro esfuerzo en el uso de dispositivos sobre escenarios reales con dos de las redes más extendidas en todo el mundo: WiFi y enlaces 802.15.4. Nos enfocamos en protocolos que ofrecen el paradigma productor/consumidor como el protocolo avanzado de colas de mensajes (AMQP) y particularmente el protocolo de transporte de mensajes telemétricos (MQTT), observamos su comportamiento a través de experimentos en laboratorio y en pruebas al aire libre, repitiendo las pruebas con diferentes tamaños de mensajes y diferente periodicidad entre mensajes. Para modelar las diferentes posibles aplicaciones de la propuesta, se tomaron en consideración varias cuestiones planteadas por la movilidad, resultando en un modelo para dimensionar eficientemente el número de fuentes para un nodo móvil y para calcular el tamaño requerido del buffer, en función del número de fuentes y del tamaño de los mensajes. Proponemos un mecanismo adaptado al protocolo MQTT que evita la pérdida de datos en clientes móviles, basado en un buffer intermedio entre la producción y publicación de mensajes que, en conjunto con el uso de una alternativa al gestor de conexiones inalámbricas "Network Manager", en ciertos contextos mejora el establecimiento de las conexiones. Para la evaluación de esta propuesta se presenta un estudio detallado de un nodo móvil que se mueve en un escenario real al aire libre, donde estudiamos el comportamiento del jitter y la transmisión de mensajes. Además, hemos utilizado emuladores de redes IoT para estudiar y determinar los efectos sobre la probabilidad de entrega de mensajes, cuando se agregan tanto publicadores como suscriptores a diferentes escenarios. Finalmente, se presenta una solución totalmente orientada a entornos con dispositivos de recursos limitados que combina los protocolos MQTT con redes tolerantes a retardos (DTN) para garantizar la entrega de información. La ventaja de las soluciones que proponemos reside en el hecho de que los sistemas IoT se vuelven resilientes a la movilidad y a los cambios de punto de acceso, permitiendo así que los desarrolladores creen fácilmente aplicaciones y servicios IoT evitando considerar estos problema. Otra ventaja de nuestras soluciones es que no necesitan soporte adicional de la red como sucede con protocolos como MobileIP o el protocolo que separa el identificador del localizador (LISP). Se destaca cómo hemos mejorado las soluciones existentes hasta el momento de la escritura de esta disertación, y se identifican futuras líneas de actuación que no han sido contempladas.Internet de les coses (IoT) es refereix a la idea d'interconnectar sensors, actuadors, dispositius físics, vehicles, edificis i qualsevol element dotat de l'electrònica, així com del programari i de la connectivitat de xarxa que els fa capaces d'intercanviar dades per proporcionar serveis altament efectius. En aquesta tesi ens centrem en temes relacionats amb la comunicació de sistemes IoT, específicament en situacions de mobilitat i en els problemes que això comporta. A aquest efecte oferim diferents solucions que alleugeren el seu impacte i garanteixen el lliurament d'informació en aquestes situacions. El context de referència és una ciutat intel·ligent on diversos dispositius mòbils participen de forma col·laborativa enviant periòdicament informació des dels seus sensors cap a serveis situats en plataformes en el núvol (cloud computing) on mitjançant l'ús de virtualització, la informació està protegida garantint la seva seguretat i privadesa. Les solucions proposades en aquesta tesi s'enfoquen a provar sobre una xicoteta infraestructura un prototip que abasta i integra diferents tecnologies i estàndards per a resoldre eficientment els problemes prèviament identificats. Hem enfocat el nostre esforç en l'ús de dispositius sobre escenaris reals amb dos de les xarxes més esteses a tot el món: WiFi i enllaços 802.15.4. Ens enfoquem en protocols que ofereixen el paradigma productor/consumidor com el protocol avançat de cues de missatges (AMQP) i particularment el protocol de transport de missatges telemètrics (MQTT), observem el seu comportament a través d'experiments en laboratori i en proves a l'aire lliure, repetint les proves amb diferents grandàries de missatges i diferent periodicitat entre missatges. Per a modelar les diferents possibles aplicacions de la proposta, es van prendre en consideració diverses qüestions plantejades per la mobilitat, resultant en un model per a dimensionar eficientment el nombre de fonts per a un node mòbil i per a calcular la grandària requerida del buffer, en funció del nombre de fonts i de la grandària dels missatges. Proposem un mecanisme adaptat al protocol MQTT que evita la pèrdua de dades per a clients mòbils, basat en un buffer intermedi entre la producció i publicació de missatges que en conjunt amb l'ús d'una alternativa al gestor de connexions sense fils "Network Manager'', en certs contextos millora l'establiment de les connexions. Per a l'avaluació d'aquesta proposta es presenta un estudi detallat d'un node mòbil que es mou en un escenari real a l'aire lliure, on estudiem el comportament del jitter i la transmissió de missatges. A més, hem utilitzat emuladors de xarxes IoT per a estudiar i determinar els efectes sobre la probabilitat de lliurament de missatges, quan s'agreguen tant publicadors com subscriptors a diferents escenaris. Finalment, es presenta una solució totalment orientada a entorns amb dispositius de recursos limitats que combina els protocols MQTT amb xarxes tolerants a retards (DTN) per a garantir el lliurament d'informació. L'avantatge de les solucions que proposem resideix en el fet que els sistemes IoT es tornen resilients a la mobilitat i als canvis de punt d'accés, permetent així que els desenvolupadors creuen fàcilment aplicacions i serveis IoT evitant considerar aquests problema. Un altre avantatge de les nostres solucions és que no necessiten suport addicional de la xarxa com succeeix amb protocols com MobileIP o el protocol que separa l'identificador del localitzador (LISP). Es destaca com hem millorat les solucions existents fins al moment de l'escriptura d'aquesta dissertació, i s'identifican futures línies d'actuació que no han sigut contemplades.Luzuriaga Quichimbo, JE. (2017). Managing Mobility for Distributed Smart Cities Services [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/84744TESI

Climbing Up Cloud Nine: Performance Enhancement Techniques for Cloud Computing Environments

With the transformation of cloud computing technologies from an attractive trend to a business reality, the need is more pressing than ever for efficient cloud service management tools and techniques. As cloud technologies continue to mature, the service model, resource allocation methodologies, energy efficiency models and general service management schemes are not yet saturated. The burden of making this all tick perfectly falls on cloud providers. Surely, economy of scale revenues and leveraging existing infrastructure and giant workforce are there as positives, but it is far from straightforward operation from that point. Performance and service delivery will still depend on the providers’ algorithms and policies which affect all operational areas. With that in mind, this thesis tackles a set of the more critical challenges faced by cloud providers with the purpose of enhancing cloud service performance and saving on providers’ cost. This is done by exploring innovative resource allocation techniques and developing novel tools and methodologies in the context of cloud resource management, power efficiency, high availability and solution evaluation. Optimal and suboptimal solutions to the resource allocation problem in cloud data centers from both the computational and the network sides are proposed. Next, a deep dive into the energy efficiency challenge in cloud data centers is presented. Consolidation-based and non-consolidation-based solutions containing a novel dynamic virtual machine idleness prediction technique are proposed and evaluated. An investigation of the problem of simulating cloud environments follows. Available simulation solutions are comprehensively evaluated and a novel design framework for cloud simulators covering multiple variations of the problem is presented. Moreover, the challenge of evaluating cloud resource management solutions performance in terms of high availability is addressed. An extensive framework is introduced to design high availability-aware cloud simulators and a prominent cloud simulator (GreenCloud) is extended to implement it. Finally, real cloud application scenarios evaluation is demonstrated using the new tool. The primary argument made in this thesis is that the proposed resource allocation and simulation techniques can serve as basis for effective solutions that mitigate performance and cost challenges faced by cloud providers pertaining to resource utilization, energy efficiency, and client satisfaction