Desarrollo de interfaces de alto nivel para la interaccion con gestores de maquinas virtuales en infraestructuras tipo cloud

Calatrava Arroyo, A. (2010). Desarrollo de interfaces de alto nivel para la interaccion con gestores de maquinas virtuales en infraestructuras tipo cloud. http://hdl.handle.net/10251/8607.Archivo delegad

High Performance Scientific Computing over Hybrid Cloud Platforms

Tesis por compendioScientific applications generally require large computational requirements, memory and data management for their execution. Such applications have traditionally used high-performance resources, such as shared memory supercomputers, clusters of PCs with distributed memory, or resources from Grid infrastructures on which the application needs to be adapted to run successfully. In recent years, the advent of virtualization techniques, together with the emergence of Cloud Computing, has caused a major shift in the way these applications are executed. However, the execution management of scientific applications on high performance elastic platforms is not a trivial task. In this doctoral thesis, Elastic Cloud Computing Cluster (EC3) has been developed. EC3 is an open-source tool able to execute high performance scientific applications by creating self-managed cost-efficient virtual hybrid elastic clusters on top of IaaS Clouds. These self-managed clusters have the capability to adapt the size of the cluster, i.e. the number of nodes, to the workload, thus creating the illusion of a real cluster without requiring an investment beyond the actual usage. They can be fully customized and migrated from one provider to another, in an automatically and transparent process for the users and jobs running in the cluster. EC3 can also deploy hybrid clusters across on-premises and public Cloud resources, where on-premises resources are supplemented with public Cloud resources to accelerate the execution process. Different instance types and the use of spot instances combined with on-demand resources are also cluster configurations supported by EC3. Moreover, using spot instances, together with checkpointing techniques, the tool can significantly reduce the total cost of executions while introducing automatic fault tolerance. EC3 is conceived to facilitate the use of virtual clusters to users, that might not have an extensive knowledge about these technologies, but they can benefit from them. Thus, the tool offers two different interfaces for its users, a web interface where EC3 is exposed as a service for non-experienced users and a powerful command line interface. Moreover, this thesis explores the field of light-weight virtualization using containers as an alternative to the traditional virtualization solution based on virtual machines. This study analyzes the suitable scenario for the use of containers and proposes an architecture for the deployment of elastic virtual clusters based on this technology. Finally, to demonstrate the functionality and advantages of the tools developed during this thesis, this document includes several use cases covering different scenarios and fields of knowledge, such as structural analysis of buildings, astrophysics or biodiversity.Las aplicaciones científicas generalmente precisan grandes requisitos de cómputo, memoria y gestión de datos para su ejecución. Este tipo de aplicaciones tradicionalmente ha empleado recursos de altas prestaciones, como supercomputadores de memoria compartida, clústers de PCs de memoria distribuida, o recursos provenientes de infraestructuras Grid, sobre los que se adaptaba la aplicación para que se ejecutara satisfactoriamente. El auge que han tenido las técnicas de virtualización en los últimos años, propiciando la aparición de la computación en la nube (Cloud Computing), ha provocado un importante cambio en la forma de ejecutar este tipo de aplicaciones. Sin embargo, la gestión de la ejecución de aplicaciones científicas sobre plataformas de computación elásticas de altas prestaciones no es una tarea trivial. En esta tesis doctoral se ha desarrollado Elastic Cloud Computing Cluster (EC3), una herramienta de código abierto capaz de llevar a cabo la ejecución de aplicaciones científicas de altas prestaciones creando para ello clústers virtuales, híbridos y elásticos, autogestionados y eficientes en cuanto a costes, sobre plataformas Cloud de tipo Infraestructura como Servicio (IaaS). Estos clústers autogestionados tienen la capacidad de adaptar su tamaño, es decir, el número de nodos, a la carga de trabajo, creando así la ilusión de un clúster real sin requerir una inversión por encima del uso actual. Además, son completamente configurables y pueden ser migrados de un proveedor a otro de manera automática y transparente a los usuarios y trabajos en ejecución en el cluster. EC3 también permite desplegar clústers híbridos sobre recursos Cloud públicos y privados, donde los recursos privados son complementados con recursos Cloud públicos para acelerar el proceso de ejecución. Otras configuraciones híbridas, como el empleo de diferentes tipos de instancias y el uso de instancias puntuales combinado con instancias bajo demanda son también soportadas por EC3. Además, el uso de instancias puntuales junto con técnicas de checkpointing permite a EC3 reducir significantemente el coste total de las ejecuciones a la vez que proporciona tolerancia a fallos. EC3 está concebido para facilitar el uso de clústers virtuales a los usuarios, que, aunque no tengan un conocimiento extenso sobre este tipo de tecnologías, pueden beneficiarse fácilmente de ellas. Por ello, la herramienta ofrece dos interfaces diferentes a sus usuarios, una interfaz web donde se expone EC3 como servicio para usuarios no experimentados y una potente interfaz de línea de comandos. Además, esta tesis doctoral se adentra en el campo de la virtualización ligera, mediante el uso de contenedores como alternativa a la solución tradicional de virtualización basada en máquinas virtuales. Este estudio analiza el escenario propicio para el uso de contenedores y propone una arquitectura para el despliegue de clusters virtuales elásticos basados en esta tecnología. Finalmente, para demostrar la funcionalidad y ventajas de las herramientas desarrolladas durante esta tesis, esta memoria recoge varios casos de uso que abarcan diferentes escenarios y campos de conocimiento, como estudios estructurales de edificios, astrofísica o biodiversidad.Les aplicacions científiques generalment precisen grans requisits de còmput, de memòria i de gestió de dades per a la seua execució. Este tipus d'aplicacions tradicionalment hi ha empleat recursos d'altes prestacions, com supercomputadors de memòria compartida, clústers de PCs de memòria distribuïda, o recursos provinents d'infraestructures Grid, sobre els quals s'adaptava l'aplicació perquè s'executara satisfactòriament. L'auge que han tingut les tècniques de virtualitzaciò en els últims anys, propiciant l'aparició de la computació en el núvol (Cloud Computing), ha provocat un important canvi en la forma d'executar este tipus d'aplicacions. No obstant això, la gestió de l'execució d'aplicacions científiques sobre plataformes de computació elàstiques d'altes prestacions no és una tasca trivial. En esta tesi doctoral s'ha desenvolupat Elastic Cloud Computing Cluster (EC3), una ferramenta de codi lliure capaç de dur a terme l'execució d'aplicacions científiques d'altes prestacions creant per a això clústers virtuals, híbrids i elàstics, autogestionats i eficients quant a costos, sobre plataformes Cloud de tipus Infraestructura com a Servici (IaaS). Estos clústers autogestionats tenen la capacitat d'adaptar la seua grandària, es dir, el nombre de nodes, a la càrrega de treball, creant així la il·lusió d'un cluster real sense requerir una inversió per damunt de l'ús actual. A més, són completament configurables i poden ser migrats d'un proveïdor a un altre de forma automàtica i transparent als usuaris i treballs en execució en el cluster. EC3 també permet desplegar clústers híbrids sobre recursos Cloud públics i privats, on els recursos privats són complementats amb recursos Cloud públics per a accelerar el procés d'execució. Altres configuracions híbrides, com l'us de diferents tipus d'instàncies i l'ús d'instàncies puntuals combinat amb instàncies baix demanda són també suportades per EC3. A més, l'ús d'instàncies puntuals junt amb tècniques de checkpointing permet a EC3 reduir significantment el cost total de les execucions al mateix temps que proporciona tolerància a fallades. EC3e stà concebut per a facilitar l'ús de clústers virtuals als usuaris, que, encara que no tinguen un coneixement extensiu sobre este tipus de tecnologies, poden beneficiar-se fàcilment d'elles. Per això, la ferramenta oferix dos interfícies diferents dels seus usuaris, una interfície web on s'exposa EC3 com a servici per a usuaris no experimentats i una potent interfície de línia d'ordres. A més, esta tesi doctoral s'endinsa en el camp de la virtualitzaciò lleugera, per mitjà de l'ús de contenidors com a alternativa a la solució tradicional de virtualitzaciò basada en màquines virtuals. Este estudi analitza l'escenari propici per a l'ús de contenidors i proposa una arquitectura per al desplegament de clusters virtuals elàstics basats en esta tecnologia. Finalment, per a demostrar la funcionalitat i avantatges de les ferramentes desenrotllades durant esta tesi, esta memòria arreplega diversos casos d'ús que comprenen diferents escenaris i camps de coneixement, com a estudis estructurals d'edificis, astrofísica o biodiversitat.Calatrava Arroyo, A. (2016). High Performance Scientific Computing over Hybrid Cloud Platforms [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/75265TESISCompendi

Uso de infraestructuras híbridas Grid y Cloud para la computación científica

[ES] El auge de las técnicas de virtualización en los últimos años ha propiciado la aparición del Cloud Computing. Esta nueva tecnología ha abierto un camino hacia el empleo de infraestructuras computacionales híbridas en el ámbito científico, basadas en potentes recursos Grid combinados con las infraestructuras virtuales dinámicas y elásticas que proporciona el Cloud. Pero esta combinación de recursos para dar soporte a ejecuciones de aplicaciones científicas intensivas no es trivial, propiciando la aparición de nuevos retos y oportunidades en áreas como la provisión de recursos o la metaplanificación. En esta tesis de máster, en primer lugar, se han desarrollado modelos teóricos de metaplanificación híbrida Grid/Cloud que permiten la integración y aprovechamiento de ambas infraestructuras por aplicaciones científicas HTC (High Throughput Computing) de acuerdo al estado del arte actual. Estos modelos teóricos se han puesto en práctica a través del desarrollo de herramientas que permiten el despliegue y ejecución concurrente de aplicaciones científicas sobre plataformas Grid y Cloud (incluyendo Clouds privados y públicos). En segundo lugar, se ha realizado un estudio de la sobrecarga que supone el proceso de virtualización con respecto a una máquina física. Finalmente, para poder valorar y poner en práctica la efectividad de los modelos, se ha incluido un caso de estudio para una aplicación científica computacionalmente compleja capaz de realizar el proceso de diseño de proteínas de propósito específico.[EN] The advent of virtualization techniques in recent years has led to the emergence of Cloud Computing. This new technology has paved the way towards the use of hybrid computing infrastructures in science, based on powerful Grid resources combined with dynamic and elastic virtual infrastructures that provides the Cloud. But this combination of resources to support the execution of computationally intensive scientific applications is not trivial, giving rise to new challenges and opportunities in areas such as the provision of resources or meta-scheduling. This master's thesis, has first developed theoretical models of hybrid Grid/Cloud metascheduling that enable the integration and use of both infrastructures by scientific HTC (High Throughput Computing) applications according to the current state of art. These theoretical models have been implemented through the development of prototype implementations that allow the deployment and concurrent execution of scientific applications on Grid and Cloud platforms (including private and public Clouds). Secondly, we have made a study of the overheads of the virtualization process with respect to a physical machine. Finally, it assesses the effectiveness of the models. For that, we have included a case study that involves a computationally intensive scientific application that is able to perform the optimization of proteins with target properties.Calatrava Arroyo, A. (2012). Uso de infraestructuras híbridas Grid y Cloud para la computación científica. http://hdl.handle.net/10251/27150Archivo delegad

A service-oriented architecture for scientific computing on cloud infrastructures

This paper describes a service-oriented architecture that eases the process of scientific application deployment and execution in IaaS Clouds, with a focus on High Throughput Computing applications. The system integrates i) a catalogue and repository of Virtual Machine Images, ii) an application deployment and configuration tool, iii) a meta-scheduler for job execution management and monitoring. The developed system significantly reduces the time required to port a scientific application to these computational environments. This is exemplified by a case study with a computationally intensive protein design application on both a private Cloud and a hybrid three-level infrastructure (Grid, private and public Cloud).The authors wish to thank the financial support received from the Generalitat Valenciana for the project GV/2012/076 and to the Ministerio de Econom´ıa y Competitividad for the project CodeCloud (TIN2010-17804)Moltó, G.; Calatrava Arroyo, A.; Hernández García, V. (2013). A service-oriented architecture for scientific computing on cloud infrastructures. En High Performance Computing for Computational Science - VECPAR 2012. Springer Verlag (Germany). 163-176. doi:10.1007/978-3-642-38718-0_18S163176Vaquero, L.M., Rodero-Merino, L., Caceres, J., Lindner, M.: A break in the clouds. ACM SIGCOMM Computer Communication Review 39(1), 50 (2008)Armbrust, M., Fox, A., Griffith, R., Joseph, A.: Above the clouds: A berkeley view of cloud computing. Technical report, UC Berkeley Reliable Adaptive Distributed Systems Laboratory (2009)Rehr, J., Vila, F., Gardner, J., Svec, L., Prange, M.: Scientific computing in the cloud. Computing in Science 99 (2010)Keahey, K., Figueiredo, R., Fortes, J., Freeman, T., Tsugawa, M.: Science Clouds: Early Experiences in Cloud Computing for Scientific Applications. In: Cloud Computing and its Applications (2008)Carrión, J.V., Moltó, G., De Alfonso, C., Caballer, M., Hernández, V.: A Generic Catalog and Repository Service for Virtual Machine Images. In: 2nd International ICST Conference on Cloud Computing (CloudComp 2010) (2010)Moltó, G., Hernández, V., Alonso, J.: A service-oriented WSRF-based architecture for metascheduling on computational Grids. Future Generation Computer Systems 24(4), 317–328 (2008)Krishnan, S., Clementi, L., Ren, J., Papadopoulos, P., Li, W.: Design and Evaluation of Opal2: A Toolkit for Scientific Software as a Service. In: 2009 IEEE Congress on Services (2009)Distributed Management Task Force (DMTF): The Open Virtualization Format Specification (Technical report)Raman, R., Livny, M., Solomon, M.: Matchmaking: Distributed Resource Management for High Throughput Computing. In: Proceedings of the Seventh IEEE International Symposium on High Performance Distributed Computing, pp. 28–31 (1998)Wei, J., Zhang, X., Ammons, G., Bala, V., Ning, P.: Managing security of virtual machine images in a cloud environment. ACM Press, New York (2009)Keahey, K., Freeman, T.: Contextualization: Providing One-Click Virtual Clusters. In: Fourth IEEE International Conference on eScience, pp. 301–308 (2008)Foster, I.: Globus toolkit version 4: Software for service-oriented systems. Journal of Computer Science and Technology 21(4), 513–520 (2006)Moltó, G., Suárez, M., Tortosa, P., Alonso, J.M., Hernández, V., Jaramillo, A.: Protein design based on parallel dimensional reduction. Journal of Chemical Information and Modeling 49(5), 1261–1271 (2009)Calatrava, A.: In: Use of Grid and Cloud Hybrid Infrastructures for Scientific Computing (M.Sc. Thesis in Spanish), Universitat Politècnica de València (2012)Keahey, K., Freeman, T., Lauret, J., Olson, D.: Virtual workspaces for scientific applications. Journal of Physics: Conference Series 78(1), 012038 (2007)Pallickara, S., Pierce, M., Dong, Q., Kong, C.: Enabling Large Scale Scientific Computations for Expressed Sequence Tag Sequencing over Grid and Cloud Computing Clusters. In: Eigth International Conference on Parallel Processing and Applied Mathematics (PPAM 2009), Citeseer (2009)Merzky, A., Stamou, K., Jha, S.: Application Level Interoperability between Clouds and Grids. In: 2009 Workshops at the Grid and Pervasive Computing Conference, pp. 143–150 (2009)Thain, D., Tannenbaum, T., Livny, M.: Distributed computing in practice: the Condor experience. Concurrency and Computation: Practice and Experience 17(2-4), 323–356 (2005)Simmhan, Y., van Ingen, C., Subramanian, G., Li, J.: Bridging the Gap between Desktop and the Cloud for eScience Applications. In: 2010 IEEE 3rd International Conference on Cloud Computing, pp. 474–481. IEEE (2010)Chappell, D.: Introducing windows azure. Technical report (2009

Container-based Virtual Elastic Clusters

[EN] eScience demands large-scale computing clusters to support the efficient execution of resource-intensive scientific applications. Virtual Machines (VMs) have introduced the ability to provide customizable execution environments, at the expense of performance loss for applications. However, in recent years, containers have emerged as a light-weight virtualization technology compared to VMs. Indeed, the usage of containers for virtual clusters allows better performance for the applications and fast deployment of additional working nodes, for enhanced elasticity. This paper focuses on the deployment, configuration and management of Virtual Elastic computer Clusters (VEC) dedicated to process scientific workloads. The nodes of the scientific cluster are hosted in containers running on bare-metal machines. The opensource tool Elastic Cluster for Docker (EC4Docker) is introduced, integrated with Docker Swarm to create auto-scaled virtual computer clusters of containers across distributed deployments. We also discuss the benefits and limitations of this solution and analyse the performance of the developed tools under a real scenario by means of a scientific use case that demonstrates the feasibility of the proposed approach.This work has been developed under the support of the program "Ayudas para la contratacion de personal investigador en formacion de catheter predoctoral, programa VALi+d", grant number ACIF/2013/003, from the Conselleria d'Educacio of the Generalitat Valenciana. The authors wish to thank the financial support received form The Spanish Ministry of Economy and Competitiveness to develop the project "CLUVIEM", with reference TIN2013-44390-R.Alfonso Laguna, CD.; Calatrava Arroyo, A.; Moltó, G. (2017). Container-based Virtual Elastic Clusters. Journal of Systems and Software. 127:1-11. https://doi.org/10.1016/j.jss.2017.01.007S11112

A Pilot Experience with Software Programming Environments as a Service for Teaching Activities

[EN] Software programming is one of the key abilities for the development of Computational Thinking (CT) skills in Science, Technology, Engineering and Mathematics (STEM). However, specific software tools to emulate realistic scenarios are required for effective teaching. Unfortunately, these tools have some limitations in educational environments due to the need of an adequate configuration and orchestration, which usually assumes an unaffordable work overload for teachers and is inaccessible for students outside the laboratories. To mitigate the aforementioned limitations, we rely on cloud solutions that automate the process of orchestration and configuration of software tools on top of cloud computing infrastructures. This way, the paper presents ACTaaS as a cloud-based educational resource that deploys and orchestrates a whole realistic software programming environment. ACTaaS provides a simple, fast and automatic way to set up a professional integrated environment without involving an overload to the teacher, and it provides an ubiquitous access to the environment. The solution has been tested in a pilot group of 28 students. Currently, there is no tool like ACTaaS that allows such a high grade of automation for the deployment of software production environments focused on educational activities supporting a wide range of cloud providers. Preliminary results through a pilot group predict its effectiveness due to the efficiency to set up a class environment in minutes without overloading the teachers, and providing ubiquitous access to students. In addition, the first student opinions about the experience were greatly positive.This research was funded by Conselleria d'Innovacio, Universitat, Ciencia i Societat Digital for the project "CloudSTEM" grant number AICO/2019/313, and the Vicerrectorado de Estudios, Calidad y Acreditacion of the Universitat Politecnica de Valencia grant number PIME/19-20/166.Calatrava Arroyo, A.; Ramos Montes, M.; Segrelles Quilis, JD. (2021). A Pilot Experience with Software Programming Environments as a Service for Teaching Activities. Applied Sciences. 11(1). https://doi.org/10.3390/app11010341S111Campbell, J. O., Bourne, J. R., Mosterman, P. J., & Brodersen, A. J. (2002). The Effectiveness of Learning Simulations for Electronic Laboratories. Journal of Engineering Education, 91(1), 81-87. doi:10.1002/j.2168-9830.2002.tb00675.xFraser, D. M., Pillay, R., Tjatindi, L., & Case, J. M. (2007). Enhancing the Learning of Fluid Mechanics Using Computer Simulations. Journal of Engineering Education, 96(4), 381-388. doi:10.1002/j.2168-9830.2007.tb00946.xTroussas, C., Krouska, A., & Sgouropoulou, C. (2020). Collaboration and fuzzy-modeled personalization for mobile game-based learning in higher education. Computers & Education, 144, 103698. doi:10.1016/j.compedu.2019.103698González-Martínez, J. A., Bote-Lorenzo, M. L., Gómez-Sánchez, E., & Cano-Parra, R. (2015). Cloud computing and education: A state-of-the-art survey. Computers & Education, 80, 132-151. doi:10.1016/j.compedu.2014.08.017Moreno, A. M., Sanchez-Segura, M.-I., Medina-Dominguez, F., & Carvajal, L. (2012). Balancing software engineering education and industrial needs. Journal of Systems and Software, 85(7), 1607-1620. doi:10.1016/j.jss.2012.01.060Desai, C., Janzen, D., & Savage, K. (2008). A survey of evidence for test-driven development in academia. ACM SIGCSE Bulletin, 40(2), 97-101. doi:10.1145/1383602.1383644Barriocanal, E. G., Urbán, M.-Á. S., Cuevas, I. A., & Pérez, P. D. (2002). An experience in integrating automated unit testing practices in an introductory programming course. ACM SIGCSE Bulletin, 34(4), 125-128. doi:10.1145/820127.820183OASIS Topology and Orchestration Specification for Cloud Applications (TOSCA) https://www.oasis-open.org/committees/tc_home.php?wg_abbrev=toscaTomarchio, O., Calcaterra, D., & Modica, G. D. (2020). Cloud resource orchestration in the multi-cloud landscape: a systematic review of existing frameworks. Journal of Cloud Computing, 9(1). doi:10.1186/s13677-020-00194-7Cloudify https://cloudify.coStarCluster http://web.mit.edu/stardev/cluster/ElastiCluster https://elasticluster.github.io/elasticluster/Apache ARIA TOSCA Orchestration Engine http://ariatosca.incubator.apache.orgOpenTOSCA http://www.opentosca.orgGiannakopoulos, I., Papailiou, N., Mantas, C., Konstantinou, I., Tsoumakos, D., & Koziris, N. (2014). CELAR: Automated application elasticity platform. 2014 IEEE International Conference on Big Data (Big Data). doi:10.1109/bigdata.2014.7004481Yangui, S., Marshall, I.-J., Laisne, J.-P., & Tata, S. (2013). CompatibleOne: The Open Source Cloud Broker. Journal of Grid Computing, 12(1), 93-109. doi:10.1007/s10723-013-9285-0Caballer, M., Blanquer, I., Moltó, G., & de Alfonso, C. (2014). Dynamic Management of Virtual Infrastructures. Journal of Grid Computing, 13(1), 53-70. doi:10.1007/s10723-014-9296-5Ansible https://www.ansible.com/JUnit Framework for Java https://junit.org/Check Unit Testing Framework for C https://libcheck.github.io/check

Metodología para el desarrollo y evaluación de competencias de programación software en la nube

Hoy en día, las competencias relativas a la programación de software son fundamentales en las titulaciones STEM (Scientific, Technology, Engineering and Mathematics). Por ello, habitualmente éstas incorporan asignaturas específicas enfocadas a su adquisición y evaluación, en las que se llevan a cabo actividades educativas de carácter práctico que sumergen al alumno en entornos realistas de desarrollo software. Para la ejecución de dichas actividades, se requiere de instrumental específico comúnmente utilizado en entornos profesionales, tales como repositorios de versiones o entornos de desarrollo y testeo software, entre otros, que necesitan de una configuración y orquestación adecuada para que sean efectivos y que en ocasiones supone al docente una sobrecarga inasumible. En este artículo, se presenta una metodología de trabajo para sesiones prácticas que permitirá a los docentes desarrollar y evaluar las competencias relativas a la programación software. Además, se presenta un recurso docente que da soporte a dicha metodología mediante el despliegue y uso del instrumental software específico necesario en la nube, con el objeto de disponer de forma sencilla y rápida de un entorno integrado para la aplicación de la metodología sin que suponga una sobrecarga al docente.Nowadays, the competences related to software programming are fundamental in the STEM (Scientific, Technology, Engineering and Mathematics) degrees. For that, they usually incorporate specific subjects focused on the acquisition and evaluation of software programming skills. These subjects commonly have practical educational activities that try to immerse the student in realistic environments. For the appropriate execution of those type of activities, we require specific software tools commonly used in professional environments, such as version-control repositories for tracking changes in the source code or environments to develop and test code, among others. These tools require an adequate configuration and orchestration to be effective, sometimes involving the teacher in a unapproachable overload. In this article, we present a work methodology for practical sessions that will allow teachers to develop and evaluate the competences related to software programming. In addition, a teaching resource is presented that supports this methodology through the deployment and use of specific software tools necessary in the cloud, in order to have a simple and fast way of an integrated environment for the application of the methodology without involving an overload to the teacher.Los autores agradecen la financiación recibida por el Vicerrectorado de Estudios, Calidad y Acreditación de la Universitat Politècnica de València para desarrollar el Proyecto de Innovación y Mejora Educativa “Comunidades de Aprendizaje como servicios en la nube para el desarrollo y evaluación automática de Competencias Transversales y Objetivos Formativos específicos”, con referencia B29

Serverless computing for container-based architectures

[EN] New architectural patterns (e.g. microservices), the massive adoption of Linux contain- ers (e.g. Docker containers), and improvements in key features of Cloud computing such as auto-scaling, have helped developers to decouple complex and monolithic sys- tems into smaller stateless services. In turn, Cloud providers have introduced serverless computing, where applications can be defined as a workflow of event-triggered functions. However, serverless services, such as AWS Lambda, impose serious restrictions for these applications (e.g. using a predefined set of programming languages or difficulting the installation and deployment of external libraries). This paper addresses such issues by introducing a framework and a methodology to create Serverless Container-aware AR- chitectures (SCAR). The SCAR framework can be used to create highly-parallel event- driven serverless applications that run on customized runtime environments defined as Docker images on top of AWS Lambda. This paper describes the architecture of SCAR together with the cache-based optimizations applied to minimize cost, exemplified on a massive image processing use case. The results show that, by means of SCAR, AWS Lambda becomes a convenient platform for High Throughput Computing, specially for highly-parallel bursty workloads of short stateless jobs.The authors would like to thank the Spanish "Ministerio de Economia, Industria y Competitividad" for the project "BigCLOE" under grant reference TIN2016-79951-R. The authors would also like to thank Jorge Gomes from LIP for the development of the udocker tool.Pérez-González, AM.; Moltó, G.; Caballer Fernández, M.; Calatrava Arroyo, A. (2018). Serverless computing for container-based architectures. Future Generation Computer Systems. 83:50-59. https://doi.org/10.1016/j.future.2018.01.022S50598

Análisis de alternativas para la eliminación de la sobreexplotación de acuíferos en el Valle de Guadalentín

[EN] In this paper we analyse the economic impact of alternative water management instruments that allow addressing the problem of non-renewable groundwater pumping in the aquifers of the Guadalentín Valley (southeast Spain). Their impact is assessed using a partial equilibrium mathematical programming model that maximises the farm net margin resulting from the use of the available water resources for irrigation in the area. Our results show that the buyback of groundwater pumping rights is the option with the greatest public budgetary cost and economic impact. On the contrary, the combination of an environmental tax on groundwater pumping and the substitution of groundwater by subsidised desalinised water allow eliminating aquifer overdraft in the area while minimising the public budgetary cost and the economic impact on the agricultural sector.[ES] Este trabajo analiza el impacto económico del uso de diferentes instrumentos de gestión del agua para eliminar la sobreexplotación en los acuíferos del Valle del Guadalentín (cuenca del Segura). Dicho impacto se evalúa, con un enfoque de equilibrio parcial, mediante un modelo de programación matemática que maximiza el margen neto derivado de utilizar las diferentes fuentes de suministro de agua disponibles en la zona para el regadío. Nuestros resultados muestran cómo la compra de derechos de aguas subterráneas es la opción con mayor coste para la Administración e impacto económico. Por el contrario, la combinación de una tasa sobre las extracciones de aguas subterráneas y la sustitución de éstas por agua desalada subvencionada permiten eliminar la sobreexplotación de los acuíferos compatibilizando la contención del coste presupuestario con la minimización del impacto sobre el sector agrario.Los autores agradecen sus valiosos comentarios a dos revisores anónimos. Este trabajo ha sido financiado por el Instituto Nacional de Investigación y Tecnología Agraria y Alimentaria (INIA) a través del proyecto con referencia RTA2010-00109-C04-03Calatrava, J.; Guillem, A.; Martínez-Granados, D. (2011). Analysis of alternatives to eliminate aquifer overdraft in the Guadalentín Valley, SE Spain. Economía Agraria y Recursos Naturales - Agricultural and Resource Economics. 11(2):33-62. https://doi.org/10.7201/earn.2011.02.02SWORD336211

A survey of the European Open Science Cloud services for expanding the capacity and capabilities of multidisciplinary scientific applications

Open Science is a paradigm in which scientific data, procedures, tools and results are shared transparently and reused by society. The European Open Science Cloud (EOSC) initiative is an effort in Europe to provide an open, trusted, virtual and federated computing environment to execute scientific applications and store, share and reuse research data across borders and scientific disciplines. Additionally, scientific services are becoming increasingly data-intensive, not only in terms of computationally intensive tasks but also in terms of storage resources. To meet those resource demands, computing paradigms such as High-Performance Computing (HPC) and Cloud Computing are applied to e-science applications. However, adapting applications and services to these paradigms is a challenging task, commonly requiring a deep knowledge of the underlying technologies, which often constitutes a general barrier to its uptake by scientists. In this context, EOSC-Synergy, a collaborative project involving more than 20 institutions from eight European countries pooling their knowledge and experience to enhance EOSC’s capabilities and capacities, aims to bring EOSC closer to the scientific communities. This article provides a summary analysis of the adaptations made in the ten thematic services of EOSC-Synergy to embrace this paradigm. These services are grouped into four categories: Earth Observation, Environment, Biomedicine, and Astrophysics. The analysis will lead to the identification of commonalities, best practices and common requirements, regardless of the thematic area of the service. Experience gained from the thematic services can be transferred to new services for the adoption of the EOSC ecosystem framework. The article made several recommendations for the integration of thematic services in the EOSC ecosystem regarding Authentication and Authorization (federated regional or thematic solutions based on EduGAIN mainly), FAIR data and metadata preservation solutions (both at cataloguing and data preservation—such as EUDAT’s B2SHARE), cloud platform-agnostic resource management services (such as Infrastructure Manager) and workload management solutions.This work was supported by the European Union’s Horizon 2020 research and innovation programme under grant agreement No 857647, EOSC-Synergy, European Open Science Cloud - Expanding Capacities by building Capabilities. Moreover, this work is partially funded by grant No 2015/24461-2, São Paulo Research Foundation (FAPESP). Francisco Brasileiro is a CNPq/Brazil researcher (grant 308027/2020-5).Peer Reviewed"Article signat per 20 autors/es: Amanda Calatrava, Hernán Asorey, Jan Astalos, Alberto Azevedo, Francesco Benincasa, Ignacio Blanquer, Martin Bobak, Francisco Brasileiro, Laia Codó, Laura del Cano, Borja Esteban, Meritxell Ferret, Josef Handl, Tobias Kerzenmacher, Valentin Kozlov, Aleš Křenek, Ricardo Martins, Manuel Pavesio, Antonio Juan Rubio-Montero, Juan Sánchez-Ferrero "Postprint (published version