Service Level Agreements in Cloud Computing and Big Data

Now-a-days Most of the industries are having large volumes of data. Data has range of Tera bytes to Peta byte. Organizations are looking to handle the growth of data. Enterprises are using cloud deployments to address the big data and analytics with respect to the interaction between cloud and big data. This paper presents big data issues and research directions towards the ongoing work of processing of big data in the distributed environments

Advances in Grid Computing

This book approaches the grid computing with a perspective on the latest achievements in the field, providing an insight into the current research trends and advances, and presenting a large range of innovative research papers. The topics covered in this book include resource and data management, grid architectures and development, and grid-enabled applications. New ideas employing heuristic methods from swarm intelligence or genetic algorithm and quantum encryption are considered in order to explain two main aspects of grid computing: resource management and data management. The book addresses also some aspects of grid computing that regard architecture and development, and includes a diverse range of applications for grid computing, including possible human grid computing system, simulation of the fusion reaction, ubiquitous healthcare service provisioning and complex water systems

The Case for an Adaptive Integration Framework for Data Aggregation/Dissemination in Service-Oriented Architectures

Abstract The migration to Service Oriented Architectures (SOA

Trusted resource allocation in volunteer edge-cloud computing for scientific applications

Data-intensive science applications in fields such as e.g., bioinformatics, health sciences, and material discovery are becoming increasingly dynamic and demanding with resource requirements. Researchers using these applications which are based on advanced scientific workflows frequently require a diverse set of resources that are often not available within private servers or a single Cloud Service Provider (CSP). For example, a user working with Precision Medicine applications would prefer only those CSPs who follow guidelines from HIPAA (Health Insurance Portability and Accountability Act) for implementing their data services and might want services from other CSPs for economic viability. With the generation of more and more data these workflows often require deployment and dynamic scaling of multi-cloud resources in an efficient and high-performance manner (e.g., quick setup, reduced computation time, and increased application throughput). At the same time, users seek to minimize the costs of configuring the related multi-cloud resources. While performance and cost are among the key factors to decide upon CSP resource selection, the scientific workflows often process proprietary/confidential data that introduces additional constraints of security postures. Thus, users have to make an informed decision on the selection of resources that are most suited for their applications while trading off between the key factors of resource selection which are performance, agility, cost, and security (PACS). Furthermore, even with the most efficient resource allocation across multi-cloud, the cost to solution might not be economical for all users which have led to the development of new paradigms of computing such as volunteer computing where users utilize volunteered cyber resources to meet their computing requirements. For economical and readily available resources, it is essential that such volunteered resources can integrate well with cloud resources for providing the most efficient computing infrastructure for users. In this dissertation, individual stages such as user requirement collection, user's resource preferences, resource brokering and task scheduling, in lifecycle of resource brokering for users are tackled. For collection of user requirements, a novel approach through an iterative design interface is proposed. In addition, fuzzy interference-based approach is proposed to capture users' biases and expertise for guiding their resource selection for their applications. The results showed improvement in performance i.e. time to execute in 98 percent of the studied applications. The data collected on user's requirements and preferences is later used by optimizer engine and machine learning algorithms for resource brokering. For resource brokering, a new integer linear programming based solution (OnTimeURB) is proposed which creates multi-cloud template solutions for resource allocation while also optimizing performance, agility, cost, and security. The solution was further improved by the addition of a machine learning model based on naive bayes classifier which captures the true QoS of cloud resources for guiding template solution creation. The proposed solution was able to improve the time to execute for as much as 96 percent of the largest applications. As discussed above, to fulfill necessity of economical computing resources, a new paradigm of computing viz-a-viz Volunteer Edge Computing (VEC) is proposed which reduces cost and improves performance and security by creating edge clusters comprising of volunteered computing resources close to users. The initial results have shown improved time of execution for application workflows against state-of-the-art solutions while utilizing only the most secure VEC resources. Consequently, we have utilized reinforcement learning based solutions to characterize volunteered resources for their availability and flexibility towards implementation of security policies. The characterization of volunteered resources facilitates efficient allocation of resources and scheduling of workflows tasks which improves performance and throughput of workflow executions. VEC architecture is further validated with state-of-the-art bioinformatics workflows and manufacturing workflows.Includes bibliographical references

Towards a cloud enabler : from an optical network resource provisioning system to a generalized architecture for dynamic infrastructure services provisioning

This work was developed during a period where most of the optical management and provisioning system where manual and proprietary. This work contributed to the evolution of the state of the art of optical networks with new architectures and advanced virtual infrastructure services. The evolution of optical networks, and internet globally, have been very promising during the last decade. The impact of mobile technology, grid, cloud computing, HDTV, augmented reality and big data, among many others, have driven the evolution of optical networks towards current service technologies, mostly based on SDN (Software Defined Networking) architectures and NFV(Network Functions Virtualisation). Moreover, the convergence of IP/Optical networks and IT services, and the evolution of the internet and optical infrastructures, have generated novel service orchestrators and open source frameworks. In fact, technology has evolved that fast that none could foresee how important Internet is for our current lives. Said in other words, technology was forced to evolve in a way that network architectures became much more transparent, dynamic and flexible to the end users (applications, user interfaces or simple APIs). This Thesis exposes the work done on defining new architectures for Service Oriented Networks and the contribution to the state of the art. The research work is divided into three topics. It describes the evolution from a Network Resource Provisioning System to an advanced Service Plane, and ends with a new architecture that virtualized the optical infrastructure in order to provide coordinated, on-demand and dynamic services between the application and the network infrastructure layer, becoming an enabler for the new generation of cloud network infrastructures. The work done on defining a Network Resource Provisioning System established the first bases for future work on network infrastructure virtualization. The UCLP (User Light Path Provisioning) technology was the first attempt for Customer Empowered Networks and Articulated Private Networks. It empowered the users and brought virtualization and partitioning functionalities into the optical data plane, with new interfaces for dynamic service provisioning. The work done within the development of a new Service Plane allowed the provisioning of on-demand connectivity services from the application, and in a multi-domain and multi-technology scenario based on a virtual network infrastructure composed of resources from different infrastructure providers. This Service Plane facilitated the deployment of applications consuming large amounts of data under deterministic conditions, so allowing the networks behave as a Grid-class resource. It became the first on-demand provisioning system that at lower levels allowed the creation of one virtual domain composed from resources of different providers. The last research topic presents an architecture that consolidated the work done in virtualisation while enhancing the capabilities to upper layers, so fully integrating the optical network infrastructure into the cloud environment, and so providing an architecture that enabled cloud services by integrating the request of optical network and IT infrastructure services together at the same level. It set up a new trend into the research community and evolved towards the technology we use today based on SDN and NFV. Summing up, the work presented is focused on the provisioning of virtual infrastructures from the architectural point of view of optical networks and IT infrastructures, together with the design and definition of novel service layers. It means, architectures that enabled the creation of virtual infrastructures composed of optical networks and IT resources, isolated and provisioned on-demand and in advance with infrastructure re-planning functionalities, and a new set of interfaces to open up those services to applications or third parties.Aquesta tesi es va desenvolupar durant un període on la majoria de sistemes de gestió de xarxa òptica eren manuals i basats en sistemes propietaris. En aquest sentit, la feina presentada va contribuir a l'evolució de l'estat de l'art de les xarxes òptiques tant a nivell d’arquitectures com de provisió d’infraestructures virtuals. L'evolució de les xarxes òptiques, i d'Internet a nivell mundial, han estat molt prometedores durant l'última dècada. L'impacte de la tecnologia mòbil, la computació al núvol, la televisió d'alta definició, la realitat augmentada i el big data, entre molts altres, han impulsat l'evolució cap a xarxes d’altes prestacions amb nous serveis basats en SDN (Software Defined Networking) i NFV (Funcions de xarxa La virtualització). D'altra banda, la convergència de xarxes òptiques i els serveis IT, junt amb l'evolució d'Internet i de les infraestructures òptiques, han generat nous orquestradors de serveis i frameworks basats en codi obert. La tecnologia ha evolucionat a una velocitat on ningú podria haver predit la importància que Internet està tenint en el nostre dia a dia. Dit en altres paraules, la tecnologia es va veure obligada a evolucionar d'una manera on les arquitectures de xarxa es fessin més transparent, dinàmiques i flexibles vers als usuaris finals (aplicacions, interfícies d'usuari o APIs simples). Aquesta Tesi presenta noves arquitectures de xarxa òptica orientades a serveis. El treball de recerca es divideix en tres temes. Es presenta un sistema de virtualització i aprovisionament de recursos de xarxa i la seva evolució a un pla de servei avançat, per acabar presentant el disseny d’una nova arquitectura capaç de virtualitzar la infraestructura òptica i IT i proporcionar serveis de forma coordinada, i sota demanda, entre l'aplicació i la capa d'infraestructura de xarxa òptica. Tot esdevenint un facilitador per a la nova generació d'infraestructures de xarxa en el núvol. El treball realitzat en la definició del sistema de virtualització de recursos va establir les primeres bases sobre la virtualització de la infraestructura de xarxa òptica en el marc de les “Customer Empowered Networks” i “Articulated Private Networks”. Amb l’objectiu de virtualitzar el pla de dades òptic, i oferir noves interfícies per a la provisió de serveis dinàmics de xarxa. En quant al pla de serveis presentat, aquest va facilitat la provisió de serveis de connectivitat sota demanda per part de l'aplicació, tant en entorns multi-domini, com en entorns amb múltiples tecnologies. Aquest pla de servei, anomenat Harmony, va facilitar el desplegament de noves aplicacions que consumien grans quantitats de dades en condicions deterministes. En aquest sentit, va permetre que les xarxes es comportessin com un recurs Grid, i per tant, va esdevenir el primer sistema d'aprovisionament sota demanda que permetia la creació de dominis virtuals de xarxa composts a partir de recursos de diferents proveïdors. Finalment, es presenta l’evolució d’un pla de servei cap una arquitectura global que consolida el treball realitzat a nivell de convergència d’infraestructures (òptica + IT) i millora les capacitats de les capes superiors. Aquesta arquitectura va facilitar la plena integració de la infraestructura de xarxa òptica a l'entorn del núvol. En aquest sentit, aquest resultats van evolucionar cap a les tendències actuals de SDN i NFV. En resum, el treball presentat es centra en la provisió d'infraestructures virtuals des del punt de vista d’arquitectures de xarxa òptiques i les infraestructures IT, juntament amb el disseny i definició de nous serveis de xarxa avançats, tal i com ho va ser el servei de re-planificació dinàmicaPostprint (published version

A Reinforcement Learning Quality of Service Negotiation Framework For IoT Middleware

The Internet of Things (IoT) ecosystem is characterised by heterogeneous devices dynamically interacting with each other to perform a specific task, often without human intervention. This interaction typically occurs in a service-oriented manner and is facilitated by an IoT middleware. The service provision paradigm enables the functionalities of IoT devices to be provided as IoT services to perform actuation tasks in critical-safety systems such as autonomous, connected vehicle system and industrial control systems. As IoT systems are increasingly deployed into an environment characterised by continuous changes and uncertainties, there have been growing concerns on how to resolve the Quality of Service (QoS) contentions between heterogeneous devices with conflicting preferences to guarantee the execution of mission-critical actuation tasks. With IoT devices with different QoS constraints as IoT service providers spontaneously interacts with IoT service consumers with varied QoS requirements, it becomes essential to find the best way to establish and manage the QoS agreement in the middleware as a compromise in the QoS could lead to negative consequences. This thesis presents a QoS negotiation framework, IoTQoSystem, for IoT service-oriented middleware. The QoS framework is underpinned by a negotiation process that is modelled as a Markov Decision Process (MDP). A model-based Reinforcement Learning negotiation strategy is proposed for generating an acceptable QoS solution in a dynamic, multilateral and multi-parameter scenarios. A microservice-oriented negotiation architecture is developed that combines negotiation, monitoring and forecasting to provide a self-managing mechanism for ensuring the successful execution of actuation tasks in an IoT environment. Using a case study, the developed QoS negotiation framework was evaluated using real-world data sets with different negotiation scenarios to illustrate its scalability, reliability and performance

Power Management for Cloud-Scale Data Centers

Recent years have seen the rapid growth of large and geographically distributed data centers deployed by Internet service operators to support various services such as cloud computing. Consequently, high electricity bills, as well as negative environmental implications (e.g., CO2 emission and global warming) come along. In this thesis, we first propose a novel electricity bill capping algorithm that not only minimizes the electricity cost, but also enforces a cost budget on the monthly bill for cloud-scale data centers that impact the power markets. Our solution first explicitly models the impacts of the power demands induced by cloud-scale data centers on electricity prices and the power consumption of cooling and networking in the minimization of electricity bill. In the second step, if the electricity cost exceeds a desired monthly budget due to unexpectedly high workloads, our solution guarantees the quality of service for premium customers and trades off the request throughput of ordinary customers. We formulate electricity bill capping as two related constrained optimization problems and propose efficient algorithms based on mixed integer programming. We then propose GreenWare, a novel middleware system that conducts dynamic request dispatching to maximize the percentage of renewable energy used to power a network of distributed data centers, subject to the desired cost budget of the Internet service operator. Our solution first explicitly models the intermittent generation of renewable energy, e.g., wind power and solar power, with respect to varying weather conditions in the geographical location of each data center. We then formulate the core objective of GreenWare as a constrained I optimization problem and propose an efficient request dispatching algorithm based on linear-fractional programming (LFP)