Big Data Pipelines on the Computing Continuum: Tapping the Dark Data

The computing continuum enables new opportunities for managing big data pipelines concerning efficient management of heterogeneous and untrustworthy resources. We discuss the big data pipelines lifecycle on the computing continuum and its associated challenges, and we outline a future research agenda in this area.acceptedVersio

DataCloud: Enabling the Big Data Pipelines on the Computing Continuum

acceptedVersio

Big Data Pipelines on the Computing Continuum: Ecosystem and Use Cases Overview

Organisations possess and continuously generate huge amounts of static and stream data, especially with the proliferation of Internet of Things technologies. Collected but unused data, i.e., Dark Data, mean loss in value creation potential. In this respect, the concept of Computing Continuum extends the traditional more centralised Cloud Computing paradigm with Fog and Edge Computing in order to ensure low latency pre-processing and filtering close to the data sources. However, there are still major challenges to be addressed, in particular related to management of various phases of Big Data processing on the Computing Continuum. In this paper, we set forth an ecosystem for Big Data pipelines in the Computing Continuum and introduce five relevant real-life example use cases in the context of the proposed ecosystem.acceptedVersio

A Cloud Orchestrator for Deploying Public Services on the Cloud – The Case of STRATEGIC Project

In recent times, public bodies are adopting IaaS solutions for deploying online governmental services. A sufficient number are adopting private cloud solutions while others hybrid or public offerings, making the necessity of a Cloud Orchestrator highly imperative. In this paper, the STRATEGIC Cloud Orchestrator is presented which targets deployment services in multi-cloud providers. The Cloud Orchestrator architecture and design have been developed using a purely top-down approach, driven by user requirements coming from the three different European Municipalities (London Borough of Camden-UK, Genoa-IT and Stari Grad-SR) that will adopt the STRATEGIC solution. Also, the summary of the user requirements, the technical approach and value proposition are being describe

D3.8 CLOUD INFRASTRUCTURE INCENTIVES MANAGEMENT AND DATA GOVERNANCE SOFTWARE PROTOTYPE 3

The third and final version of the Cloud Infrastructure, Incentives Management and Data Governance software prototype includes the cloud gateways and APIs, the cloud provisioning mechanisms, the implemented version of the algorithms as well as the data governance tools according to the D3.1 [1], D3.4 [2] and D3.7 [3] specifications and is built upon the first and second versions of the prototype described in D3.2 [4] and D3.5 [5] respectively. The prototype’s cloud infrastructure is supported by RECAS-BARI and is utilized by EGI through cloud gateways. These gateways allow the prototype to gather data from heterogenous data sources, such as Twitter and the global terrorism database and have integrated microservices to serve the needs of the different PolicyCLOUD pilots. The final version of the Incentives Management tool is also provided in this deliverable. This final version has been integrated with the Policy Development Toolkit (PDT) and has been deployed in the EGI Cloud. This third version of the prototype also includes the latest updates of the ABAC based access control mechanism and the Keycloak integration. This is broken down to 8 key components that have been combined to provide fine-tuned and secure access control and authentication. Specifically, Keycloak has been integrated with the Marketplace, the Gateways and the PDT and custom access policies have been developed for the gateways microservices. Finally, both the introduction of a XACML editor to ease access policies creation and the integration of EGI Check-in, an alternative way to authenticate to the prototype with academic and social credential, enhance the user experience in the PolicyCLOUD platformThis deliverable is submitted to the EC, not yet approved

Federated machine learning through edge ready architectures with privacy preservation as a service

International audienceThis paper presents the details of a novel approach, based on edge and advanced privacy preserving solutions, that tries to accelerate the adoption of personal data federation for the benefit of the evolution of valuable advanced AI models. The approach focuses on the establishment of high degree of trust between data owner and data management infrastructure so that consent in data processing is given by means of functional and enforceable options applicable at all levels of workloads and processes. The overall set of solutions will be delivered as an open-source set of implementations in the context of the PAROMA-MED project

Big Data Pipelines on the Computing Continuum: Ecosystem and Use Cases Overview

Organisations possess and continuously generate huge amounts of static and stream data, especially with the proliferation of Internet of Things technologies. Collected but unused data, i.e., Dark Data, mean loss in value creation potential. In this respect, the concept of Computing Continuum extends the traditional more centralised Cloud Computing paradigm with Fog and Edge Computing in order to ensure low latency pre-processing and filtering close to the data sources. However, there are still major challenges to be addressed, in particular related to management of various phases of Big Data processing on the Computing Continuum. In this paper, we set forth an ecosystem for Big Data pipelines in the Computing Continuum and introduce five relevant real-life example use cases in the context of the proposed ecosystem

DataCloud: Enabling the Big Data Pipelines on the Computing Continuum

With the recent developments of Internet of Things (IoT) and cloud-based technologies, massive amounts of data are generated by heterogeneous sources and stored through dedicated cloud solutions. Often organizations generate much more data than they are able to interpret, and current Cloud Computing technologies cannot fully meet the requirements of the Big Data processing applications and their data transfer overheads. Many data are stored for compliance purposes only but not used and turned into value, thus becoming Dark Data, which are not only an untapped value, but also pose a risk for organizations

Big Data Pipelines on the Computing Continuum: Ecosystem and Use Cases Overview

Organisations possess and continuously generate huge amounts of static and stream data, especially with the proliferation of Internet of Things technologies. Collected but unused data, i.e., Dark Data, mean loss in value creation potential. In this respect, the concept of Computing Continuum extends the traditional more centralised Cloud Computing paradigm with Fog and Edge Computing in order to ensure low latency pre-processing and filtering close to the data sources. However, there are still major challenges to be addressed, in particular related to management of various phases of Big Data processing on the Computing Continuum. In this paper, we set forth an ecosystem for Big Data pipelines in the Computing Continuum and introduce five relevant real-life example use cases in the context of the proposed ecosystem