Quality of Service Aware Data Stream Processing for Highly Dynamic and Scalable Applications

Huge amounts of georeferenced data streams are arriving daily to data stream management systems that are deployed for serving highly scalable and dynamic applications. There are innumerable ways at which those loads can be exploited to gain deep insights in various domains. Decision makers require an interactive visualization of such data in the form of maps and dashboards for decision making and strategic planning. Data streams normally exhibit fluctuation and oscillation in arrival rates and skewness. Those are the two predominant factors that greatly impact the overall quality of service. This requires data stream management systems to be attuned to those factors in addition to the spatial shape of the data that may exaggerate the negative impact of those factors. Current systems do not natively support services with quality guarantees for dynamic scenarios, leaving the handling of those logistics to the user which is challenging and cumbersome. Three workloads are predominant for any data stream, batch processing, scalable storage and stream processing. In this thesis, we have designed a quality of service aware system, SpatialDSMS, that constitutes several subsystems that are covering those loads and any mixed load that results from intermixing them. Most importantly, we natively have incorporated quality of service optimizations for processing avalanches of geo-referenced data streams in highly dynamic application scenarios. This has been achieved transparently on top of the codebases of emerging de facto standard best-in-class representatives, thus relieving the overburdened shoulders of the users in the presentation layer from having to reason about those services. Instead, users express their queries with quality goals and our system optimizers compiles that down into query plans with an embedded quality guarantee and leaves logistic handling to the underlying layers. We have developed standard compliant prototypes for all the subsystems that constitutes SpatialDSMS

3rd EGEE User Forum

We have organized this book in a sequence of chapters, each chapter associated with an application or technical theme introduced by an overview of the contents, and a summary of the main conclusions coming from the Forum for the chapter topic. The first chapter gathers all the plenary session keynote addresses, and following this there is a sequence of chapters covering the application flavoured sessions. These are followed by chapters with the flavour of Computer Science and Grid Technology. The final chapter covers the important number of practical demonstrations and posters exhibited at the Forum. Much of the work presented has a direct link to specific areas of Science, and so we have created a Science Index, presented below. In addition, at the end of this book, we provide a complete list of the institutes and countries involved in the User Forum

Service level agreements in spatial data infrastructures

Die vorliegende Arbeit entwickelt ein Konzept für die Integration von Service Level Agreements (SLAs) in Geodateninfrastrukturen (GDIs). Der ausgewählten mehrstufigen Ansatz beinhaltet die Entwicklung eines abstrakten SLA-Modells und einer web-basierten SLA-Management-Architektur. Das Ziel des abstrakten SLA-Modells ist die konzeptionelle Beschreibung der Struktur und des Inhaltes von SLAs speziell für die ausgewählten Anwendungsbereiche. Der Zweck der web-basierten SLA-Management-Architektur ist es, die (Online-) Aushandlung von SLAs in bereits existierenden GDIs zu ermöglichen, ohne dass eine vorherige (Offline-) Kommunikation zwischen Dienstanbieter und Dienstnutzer vonnöten ist. Der gewählte Policy-basierte Ansatz deckt nicht nur die Aushandlung von SLAs und die eigentliche Dienstnutzung ab, es wird der vollständige Lebenszyklus von SLAs unterstützt. Dazu gehört sowohl die permanente Überwachung der angebotenen Dienste als auch die permanente Evaluierung aller aktiven SLAs.This thesis develops a concept for the integration of Service Level Agreements (SLAs) in Spatial Data Infrastructure (SDIs). The selected multi-step approach involves the development of an abstract SLA model and a web-based SLA management architecture. The aim of the abstract SLA model is to describe the domain-specific structure and content of SLAs that can be applied in SDIs from a conceptual point of view. The purpose of the web-based SLA management architecture is to enable the on-demand and online negotiation of SLAs in established SDIs without the need of prior offline communication between service providers and service consumers. The selected policy-based approach covers not only agreement negotiation and service consumption, but also the complete agreement life cycle including service monitoring and agreement evaluation.<br/

Technologies and Applications for Big Data Value

This open access book explores cutting-edge solutions and best practices for big data and data-driven AI applications for the data-driven economy. It provides the reader with a basis for understanding how technical issues can be overcome to offer real-world solutions to major industrial areas. The book starts with an introductory chapter that provides an overview of the book by positioning the following chapters in terms of their contributions to technology frameworks which are key elements of the Big Data Value Public-Private Partnership and the upcoming Partnership on AI, Data and Robotics. The remainder of the book is then arranged in two parts. The first part “Technologies and Methods” contains horizontal contributions of technologies and methods that enable data value chains to be applied in any sector. The second part “Processes and Applications” details experience reports and lessons from using big data and data-driven approaches in processes and applications. Its chapters are co-authored with industry experts and cover domains including health, law, finance, retail, manufacturing, mobility, and smart cities. Contributions emanate from the Big Data Value Public-Private Partnership and the Big Data Value Association, which have acted as the European data community's nucleus to bring together businesses with leading researchers to harness the value of data to benefit society, business, science, and industry. The book is of interest to two primary audiences, first, undergraduate and postgraduate students and researchers in various fields, including big data, data science, data engineering, and machine learning and AI. Second, practitioners and industry experts engaged in data-driven systems, software design and deployment projects who are interested in employing these advanced methods to address real-world problems

D19 final plan for using and disseminating knowledge

This document presents the Final Plan for Using and Disseminating Knowledge acquired throughout the development of the CYCLOPS project as deliverable D19. It includes a description of the main achievements in disseminating knowledge, and the consortium and each participant’s plans for the exploitation of the results for the consortium as a whole, or for individual participants or groups of participants. It updates the Plan for Using and Disseminating Knowledge that was presented as Deliverable D4 and describes the final dissemination plan of the CYCLOPS project. This deliverable provides a strategy aimed at addressing various target communities in order to achieve the project dissemination and exploitation goals. After an update of the dissemination instruments employed, the deliverable focuses on the description of the dissemination activities carried out. In addition to the normal dissemination and exploitation of the work through scientific journals and professional bodies, Civil Protection Community will be specifically targeted for dissemination of the CYCLOPS deliverables, and their future exploitation of the results. Other written deliverables focus on presenting dissemination activities in specific subject areas. In particular deliverable D17 reports “the results of the dissemination of EGEE towards the Civil Protection community, and about the coordination between the EGEE and CYCLOPS activities”, deliverable D18 focuses on “collecting the CYCLOPS project results for dissemination towards different interested audiences such as Grid communities, other Civil protection agencies, but also national and international initiative and projects, SMEs, etc.” and deliverable D20 that reports “the extent to which actors beyond the research community have been involved to help spread awareness and to explore the wider societal implications of the proposed work

Cloud technology options towards Free Flow of Data

This whitepaper collects the technology solutions that the projects in the Data Protection, Security and Privacy Cluster propose to address the challenges raised by the working areas of the Free Flow of Data initiative. The document describes the technologies, methodologies, models, and tools researched and developed by the clustered projects mapped to the ten areas of work of the Free Flow of Data initiative. The aim is to facilitate the identification of the state-of-the-art of technology options towards solving the data security and privacy challenges posed by the Free Flow of Data initiative in Europe. The document gives reference to the Cluster, the individual projects and the technologies produced by them