The quality-aware service selection problem: an adaptive evolutionary approach

Die Qualität der Serviceerbringung (kurz QoS) ist ein wichtiger Aspekt in verteilten, Service-orientierten Systemen. Wenn mehrere Implementierungen einer Funktionalität koexistieren, kann die Wahl eines konkreten Services aufgrund von QoS-Aspekten getroffen werden. Leistung, Verfügbarkeit und Kosten sind Beispiele für QoS-Attribute eines Services. In der vorliegenden Dissertation werden Aspekte dieses Selektionsproblems anhand eines konkreten, Service-orientieren Systems vertieft. Es handelt sich dabei um das TAG-System in ATLAS, einem Hochenergiephysikexperiment am CERN, der Europäischen Organisation für Kernforschung. Die Daten und Services des TAG-Systems sind weltweit verteilt und müssen auf Anfrage selektiert und zu einem Workflow zusammengesetzt werden. Die Optimierung wird aus zwei unterschiedlichen Blickwinkeln. Die Selektion wird als ein dynamisches Pfadoptimierungsproblem unter Nebenbedingungen modelliert, wodurch QoS-Attribute sowohl der Knoten (Services) als auch der Kanten (Netzwerk) berücksichtigt werden können. Dynamische Aspekte des verteilten sind in der Problemformulierung integriert, da sie eine spezifische Herausforderung und Anforderung an Lösungsalgorithmen stellen. Für die dynamische Pareto-Optimierung von Serviceselektionsproblemen wird im Rahmen dieser Arbeit ein Optimierungsansatz mit einem genetischen Algorithmus präsentiert, der über einen persistenten Speicher von früheren Lösungen sowie eine automatische Adaptierung der Mutationsrate eine effiziente Anpassung an das sich ständig verändernde System gewährleistet. Eine Ontologie der Systemkomponenten sowie deren QoS-Attribute bildet die Basis für die Optimierung. Der Ansatz wird im Rahmen der Dissertation hinsichtlich der Qualität der erzielten Lösungen, der Adaptierung an änderungen sowie der Laufzeit evaluiert. Teile des Ansatzes wurden schließ lich in das TAG-System integriert und darin evaluiert.Quality of Service (QoS) is an important aspect in distributed, service-oriented systems. When several concrete services exist that implement the same functionality, the choice of a service instance among many can be made based on QoS considerations, objectives and constraints. Typically considered properties are performance, availability, and costs. In this thesis, aspects of the QoS-aware service selection problem are studied in the context of a distributed, service-oriented system from ATLAS, a high-energy physics experiment at CERN, the European Organization for Nuclear Research. In this so-called TAG system, data and modular services are distributed world-wide and need to be selected and composed on the fly, as a user starts a request. There are two conflicting optimization viewpoints. The service selection is modeled as a dynamic multi-constrained optimal path problem, which allows considering QoS attributes of service instances and of the network. The dynamic aspects of the system are included in the problem definition, as they represent a specific challenge. To address these issues regarding dynamics and conflicting viewpoints, this work proposes a service selection optimization framework based on a multi-objective genetic algorithm capable of efficiently dealing with changing conditions by using a persistent memory of good solutions, and a stepwise adaptation of the mutation rate. A system and QoS attribute ontology as well as a description of dynamics of distributed systems build the basis of the framework. The presented approach is evaluated in terms of optimization quality, adaptability to changes, runtime performance and scalability

Pay One, Get Hundreds for Free: Reducing Cloud Costs through Shared Query Execution

Cloud-based data analysis is nowadays common practice because of the lower system management overhead as well as the pay-as-you-go pricing model. The pricing model, however, is not always suitable for query processing as heavy use results in high costs. For example, in query-as-a-service systems, where users are charged per processed byte, collections of queries accessing the same data frequently can become expensive. The problem is compounded by the limited options for the user to optimize query execution when using declarative interfaces such as SQL. In this paper, we show how, without modifying existing systems and without the involvement of the cloud provider, it is possible to significantly reduce the overhead, and hence the cost, of query-as-a-service systems. Our approach is based on query rewriting so that multiple concurrent queries are combined into a single query. Our experiments show the aggregated amount of work done by the shared execution is smaller than in a query-at-a-time approach. Since queries are charged per byte processed, the cost of executing a group of queries is often the same as executing a single one of them. As an example, we demonstrate how the shared execution of the TPC-H benchmark is up to 100x and 16x cheaper in Amazon Athena and Google BigQuery than using a query-at-a-time approach while achieving a higher throughput

A Survey on Array Storage, Query Languages, and Systems

Since scientific investigation is one of the most important providers of massive amounts of ordered data, there is a renewed interest in array data processing in the context of Big Data. To the best of our knowledge, a unified resource that summarizes and analyzes array processing research over its long existence is currently missing. In this survey, we provide a guide for past, present, and future research in array processing. The survey is organized along three main topics. Array storage discusses all the aspects related to array partitioning into chunks. The identification of a reduced set of array operators to form the foundation for an array query language is analyzed across multiple such proposals. Lastly, we survey real systems for array processing. The result is a thorough survey on array data storage and processing that should be consulted by anyone interested in this research topic, independent of experience level. The survey is not complete though. We greatly appreciate pointers towards any work we might have forgotten to mention.Comment: 44 page

Little Boxes: A Dynamic Optimization Approach for Enhanced Cloud Infrastructures

The increasing demand for diverse, mobile applications with various degrees of Quality of Service requirements meets the increasing elasticity of on-demand resource provisioning in virtualized cloud computing infrastructures. This paper provides a dynamic optimization approach for enhanced cloud infrastructures, based on the concept of cloudlets, which are located at hotspot areas throughout a metropolitan area. In conjunction, we consider classical remote data centers that are rigid with respect to QoS but provide nearly abundant computation resources. Given fluctuating user demands, we optimize the cloudlet placement over a finite time horizon from a cloud infrastructure provider's perspective. By the means of a custom tailed heuristic approach, we are able to reduce the computational effort compared to the exact approach by at least three orders of magnitude, while maintaining a high solution quality with a moderate cost increase of 5.8% or less