Observation of current approaches to utilize the elastic cloud for big data stream processing

This paper conducts a systematic literature map to collect information about current approaches to utilize the elastic cloud for data stream processing in the big data context. First is a description and setup of the used scientific methodology which adheres to generally accepted methods for systematic literature maps. After building a reference set and constructing search queries for the data collection came the data set cleaning where the publications were first automatically filtered and consecutively manually reviewed to determine the relevant papers. The collected data was evaluated and visualized to help answer the defined research questions and present information. Finally the results of the thesis are discussed and the limitations and implications addressed.Diese Arbeit befasst sich mit der Durchführung einer Systematic Literature Map um einen Überblick über ein Feld zu gewähren. Das untersuchte Feld dieser Arbeit befasst sich mit der Verwendung der elastischen Eigenschaften der Cloud für Datenstrom Prozessierung im Big Data Umfeld. Bestandteil der Systematic Literature Map ist sowohl das Sammeln aller Publikationen, welche für das untersuchte Feld relevant sind, als auch die Auswertung und Präsentation der gesammelten Daten. Um die Informationen zielgerichtet zu evaluieren, wurden Forschungsfragen definiert, welche als Leitfaden dienen. Zu Beginn wurden die verwendeten wissenschaftlichen Methoden vorgestellt, welche sich an anerkannten Prozeduren orientieren. Nach dem zusammenstellen von einigen relevanten Publikationen, wurden auf deren Basis Suchanfragen für die Datensammlung erstellt. Danach wurden die Daten aus den Online Datenbanken bekannter Verleger exportiert und Duplikate entfernt. Um die endgültigen relevanten Publikationen festzustellen, wurden anhand von Schlagworten irrelevante Publikationen aussortiert und schließlich manuell einzeln bewertet. Die gesammelten Daten wurden teilweise automatisch ausgewertet und manuell klassifiziert um mit den Ergebnissen die vorher definierten Forschungsfragen zu beantworten. Abschließend werden die Ergebnisse diskutiert und die Einschränkungen und Implikationen dieser Arbeit behandelt

Energy efficient mobile video streaming using mobility

Undeniably the support of data services over the wireless Internet is becoming increasingly challenging with the plethora of different characteristic requirements of each service type. Evidently, about half of the data traffic shifted across the Internet to date consists of multimedia content such as video clips or music files that necessitate stringent real-time constraints in playback and for which increasing volumes of data should be shifted with the introduction of higher quality content. This work recasts the problem of multimedia content delivery in the mobile Internet. We propose an optimization framework with the major tenet being that real-time playback constraints can be satisfied while at the same time enabling controlled delay tolerance in packet transmission by capitalizing on pre-fetching and data buffering. More specifically two strategies are proposed amenable for real time implementation that utilize the inherent delay tolerance of popular applications based on different flavors of HTTP streaming. The proposed mechanisms have the potential of achieving many-fold energy efficiency gains at no cost on the perceived user experience

Task Scheduling in Data Stream Processing Systems

In the era of big data, with streaming applications such as social media, surveillance monitoring and real-time search generating large volumes of data, efficient Data Stream Processing Systems (DSPSs) have become essential. When designing an efficient DSPS, a number of challenges need to be considered including task allocation, scalability, fault tolerance, QoS, parallelism degree, and state management, among others. In our research, we focus on task allocation as it has a significant impact on performance metrics such as data processing latency and system throughput. An application processed by DSPSs is represented as a Directed Acyclic Graph (DAG), where each vertex represents a task and the edges show the dataflow between the tasks. Task allocation can be defined as the assignment of the vertices in the DAG to the physical compute nodes such that the data movement between the nodes is minimised. Finding an optimal task placement for stream processing systems is NP-hard. Thus, approximate scheduling approaches are required to improve the performance of DSPSs. In this thesis, we present our three proposed schedulers, each having a different heuristic partitioning approach to minimise inter-node communication for either homogeneous or heterogeneous clusters. We demonstrate how each scheduler can efficiently assign groups of highly communicating tasks to compute nodes. Our schedulers are able to outperform two state-of-the-art schedulers for three micro-benchmarks and two real-world applications, increasing throughput and reducing data processing latency as a result of a better task placement

Queue-priority optimized algorithm: a novel task scheduling for runtime systems of application integration platforms

The need for integration of applications and services in business processes from enterprises has increased with the advancement of cloud and mobile applications. Enterprises started dealing with high volumes of data from the cloud and from mobile applications, besides their own. This is the reason why integration tools must adapt themselves to handle with high volumes of data, and to exploit the scalability of cloud computational resources without increasing enterprise operations costs. Integration platforms are tools that integrate enterprises’ applications through integration processes, which are nothing but workflows composed of a set of atomic tasks connected through communication channels. Many integration platforms schedule tasks to be executed by computational resources through the First-in-first-out heuristic. This article proposes a Queue-priority algorithm that uses a novel heuristic and tackles high volumes of data in the task scheduling of integration processes. This heuristic is optimized by the Particle Swarm Optimization computational method. The results of our experiments were confirmed by statistical tests, and validated the proposal as a feasible alternative to improve integration platforms in the execution of integration processes under a high volume of data.info:eu-repo/semantics/acceptedVersio

A manifesto for future generation cloud computing: research directions for the next decade

The Cloud computing paradigm has revolutionised the computer science horizon during the past decade and has enabled the emergence of computing as the fifth utility. It has captured significant attention of academia, industries, and government bodies. Now, it has emerged as the backbone of modern economy by offering subscription-based services anytime, anywhere following a pay-as-you-go model. This has instigated (1) shorter establishment times for start-ups, (2) creation of scalable global enterprise applications, (3) better cost-to-value associativity for scientific and high performance computing applications, and (4) different invocation/execution models for pervasive and ubiquitous applications. The recent technological developments and paradigms such as serverless computing, software-defined networking, Internet of Things, and processing at network edge are creating new opportunities for Cloud computing. However, they are also posing several new challenges and creating the need for new approaches and research strategies, as well as the re-evaluation of the models that were developed to address issues such as scalability, elasticity, reliability, security, sustainability, and application models. The proposed manifesto addresses them by identifying the major open challenges in Cloud computing, emerging trends, and impact areas. It then offers research directions for the next decade, thus helping in the realisation of Future Generation Cloud Computing