A Cloud-Based Framework for Machine Learning Workloads and Applications

[EN] In this paper we propose a distributed architecture to provide machine learning practitioners with a set of tools and cloud services that cover the whole machine learning development cycle: ranging from the models creation, training, validation and testing to the models serving as a service, sharing and publication. In such respect, the DEEP-Hybrid-DataCloud framework allows transparent access to existing e-Infrastructures, effectively exploiting distributed resources for the most compute-intensive tasks coming from the machine learning development cycle. Moreover, it provides scientists with a set of Cloud-oriented services to make their models publicly available, by adopting a serverless architecture and a DevOps approach, allowing an easy share, publish and deploy of the developed models.This work was supported by the project DEEP-Hybrid-DataCloud ``Designing and Enabling E-infrastructures for intensive Processing in a Hybrid DataCloud'' that has received funding from the European Union's Horizon 2020 Research and Innovation Programme under Grant 777435Lopez Garcia, A.; Marco De Lucas, J.; Antonacci, M.; Zu Castell, W.; David, M.; Hardt, M.; Lloret Iglesias, L.... (2020). A Cloud-Based Framework for Machine Learning Workloads and Applications. IEEE Access. 8:18681-18692. https://doi.org/10.1109/ACCESS.2020.2964386S1868118692

Distributed web-scale infrastructure for crawling, indexing and search with semantic support

In this paper, we describe our work in progress in the scope of web-scale information extraction and information retrieval utilizing distributed computing. We present a distributed architecture built on top of the MapReduce paradigm for information retrieval, information processing and intelligent search supported by spatial capabilities. Proposed architecture is focused on crawling documents in several different formats, information extraction, lightweight semantic annotation of the extracted information, indexing of extracted information and finally on indexing of documents based on the geo-spatial information found in a document. We demonstrate the architecture on two use cases, where the first is search in job offers retrieved from the LinkedIn portal and the second is search in BBC news feeds and discuss several problems we had to face during the implemen-tation. We also discuss spatial search applications for both cases because both LinkedIn job offer pages and BBC news feeds contain a lot of spatial information to extract and process

DEEP: Hybrid Approach for Deep Learning

Trabajo presentado al ISC High Performance, celebrado en Frankfurt (Alemania) del 16 al 20 de junio de 2019.The DEEP-HybridDataCloud project researches on intensive computing techniques such as deep learning, that require specialized GPU hardware to explore very large datasets, through a hybrid-cloud approach that enables the access to such resources. DEEP is built on User-centric policy, i.e. we understand the needs of our user communities and help them to combine their services in a way that encapsulates technical details the end user does not have to deal with. DEEP takes care to support users of different levels of experience by providing different integration paths. We show our current solutions to the problem, which among others include the Open Catalog for deep learning applications, DEEP-as-a-Service API for providing web access to machine learning models, CI/CD pipeline for user applications, Testbed resources. We also present our use-cases tackling various problems by means of deep learning and serving to demonstrate usefulness and scalability of our approach.DEEP HybridDataCloud receives funding from the European Union's Horizon 2020 research and innovation programme under agreement RIA 777435