The CloudVeneto initiative: 10 years of operations to support interdisciplinary open science

CloudVeneto is a private cloud targeted to scientific communities based on OpenStack software. It was designed in 2013 and put in operation one year later, to support INFN projects, mainly HEP ones. Its resources are physically distributed among two sites: the Physics Department of University of Padova-INFN Padova Unit and the INFN Legnaro National Laboratories. During these 10 years CloudVeneto evolved to integrate also resources funded by ten Departments of the University of Padova, and to support several scientific disciplines of different domains. The use cases the communities have to face up often show a common pattern. This was an opportunity for us to develop and improve the services on our infrastructure to provide common solutions to different use cases. It happened for example with the Container as a Service (CaaS) that makes the management of Kubernetes clusters easier from a user point of view. Moreover, CloudVeneto joined the INFN national cloud infrastructure (INFN Cloud), making available some resources to this federated infrastructure. CloudVeneto is also involved in an R&D project to realize a distributed analysis facility for the CMS experiment based on the HTCondor batch system. In this paper we describe some use-cases of different projects pointing out the common patterns and the new implementations and configurations done in the infrastructure

Implementation and use of a highly available and innovative IaaS solution: the Cloud Area Padovana

While in the business world the cloud paradigm is typically implemented purchasing resources and services from third party providers (e.g. Amazon), in the scientific environment there's usually the need of on-premises IaaS infrastructures which allow efficient usage of the hardware distributed among (and owned by) different scientific administrative domains. In addition, the requirement of open source adoption has led to the choice of products like OpenStack by many organizations. We describe a use case of the Italian National Institute for Nuclear Physics (INFN) which resulted in the implementation of a unique cloud service, called ’Cloud Area Padovana’, which encompasses resources spread over two different sites: the INFN Legnaro National Laboratories and the INFN Padova division. We describe how this IaaS has been implemented, which technologies have been adopted and how services have been configured in high-availability (HA) mode. We also discuss how identity and authorization management were implemented, adopting a widely accepted standard architecture based on SAML2 and OpenID: by leveraging the versatility of those standards the integration with authentication federations like IDEM was implemented. We also discuss some other innovative developments, such as a pluggable scheduler, implemented as an extension of the native OpenStack scheduler, which allows the allocation of resources according to a fair-share based model and which provides a persistent queuing mechanism for handling user requests that can not be immediately served. Tools, technologies, procedures used to install, configure, monitor, operate this cloud service are also discussed. Finally we present some examples that show how this IaaS infrastructure is being used

Epistolario: cartas dirigidas a Domingo Amunátegui Solar

I. Antonio Varas. Los primeros años de Antonio VarasII. Jacinto Chacón. A propósito de Francisco Bilbao III. Pedro Aguirre Cerda. La política chilena en 1925IV. Crescente Errázuriz. La separación de la Iglesia del EstadoV. Alberto Edwards. La convención presidencial de 1876VI. Julio Vicuña Cifuentes. Recuerdo de ChileVII. Armando Quezada Acharán. Las facultades extraordinarias de 1919VIII. Manuel Rivas Vicuña. La formación del Tribunal de honor en la elección presidencial de 1920, las reformas constitucionales de 1924 y la competencia entre Rivas Vicuña e Ibañez en 192

Evolution of the CloudVeneto.it private cloud to support research and innovation

CloudVeneto.it was initially funded and deployed by INFN in 2014 for serving the computational and storage demands of INFN research projects mainly related to HEP and Nuclear Physics. It is an OpenStack-based scientific cloud with resources spread across two different sites connected with a high speed optical link: INFN Padova Unit and the INFN Legnaro National Laboratories. The infrastructure has grown throughout the years with additional funds from ten University of Padova departments, and nowadays supports a broader range of scientific and engineering disciplines. Its hardware resources provide around 2500 computational cores and 360 TB of storage to about 250 users working for more than 70 projects. In the last months we enhanced the cloud platform in two ways: 1) by integrating a number of heterogeneous GPU cards to address the special needs of user communities whose computations involve machine learning training; 2) by enabling the users to simply deploy on-demand Kubernetes clusters for Big Data Analytics applications taking advantage of the operator framework. In particular, the Kubernetes operators for Apache Kafka and Spark platforms were integrated to address real-time data ingestion and streaming processing on the cloud. This article describes the technical details of these two solutions and their integration with the cloud infrastructure

Accounting in the CloudVeneto private cloud

CloudVeneto is a private cloud implemented as the result of merging two existing cloud infrastructures: the INFN Cloud Area Padovana, and a private cloud owned by 10 departments of University of Padova. This infrastructure is a full production facility, in continuous growth, both in terms of users, and in terms of computing and storage resources. Even if the usage of CloudVeneto is not regulated by a strict pay-per-use model, the availability of accounting information for such infrastructure is a requirement, to detect if the resources allocated to the user communities are effciently used, and to perform an effective capacity planning. We present in this paper how the accounting system used in CloudVeneto evolved over time, focusing on the accounting framework being used now, implemented by integrating existing components

Evolution of the CloudVeneto.it private cloud to support research and innovation

CloudVeneto.it was initially funded and deployed by INFN in 2014 for serving the computational and storage demands of INFN research projects mainly related to HEP and Nuclear Physics. It is an OpenStack-based scientific cloud with resources spread across two different sites connected with a high speed optical link: INFN Padova Unit and the INFN Legnaro National Laboratories. The infrastructure has grown throughout the years with additional funds from ten University of Padova departments, and nowadays supports a broader range of scientific and engineering disciplines. Its hardware resources provide around 2500 computational cores and 360 TB of storage to about 250 users working for more than 70 projects. In the last months we enhanced the cloud platform in two ways: 1) by integrating a number of heterogeneous GPU cards to address the special needs of user communities whose computations involve machine learning training; 2) by enabling the users to simply deploy on-demand Kubernetes clusters for Big Data Analytics applications taking advantage of the operator framework. In particular, the Kubernetes operators for Apache Kafka and Spark platforms were integrated to address real-time data ingestion and streaming processing on the cloud. This article describes the technical details of these two solutions and their integration with the cloud infrastructure

Merging OpenStack-based private clouds: the case of CloudVeneto.it

The Cloud Area Padovana, deployed in 2014, is a scientific IaaS cloud, spread between two different sites: the INFN Padova Unit and the INFN Legnaro National Labs. It provides about 1100 logical cores and 50 TB of storage. The entire computing facility, owned by INFN, satisfies the computational and storage demands of more than 100 users belonging to about 30 research projects, mainly related to HEP and nuclear physics. The Padova data centre also has hosted and operated since 2015 an independent IaaS cloud managing network, storage and computing resources owned by 10 departments of the University of Padova, supporting a broader range of scientific and engineering disciplines. This infrastructure provides about 480 logical cores and 90 TB of storage and supports more than 40 research projects. These two clouds share only a limited set of ICT services and tools (mainly for configuration, monitoring and accounting), whereas their daily operations and maintenance are carried out separately by INFN and University personnel. At the end of 2017 we planned to merge the two infrastructures in order to optimise the use of resources (both human and ICT) and to avoid useless duplication of services. We discuss here how we plan to implement this integration, resulting in a single cloud infrastructure named CloudVeneto.it

Merging OpenStack-based private clouds: the case of CloudVeneto.it

The Cloud Area Padovana, deployed in 2014, is a scientific IaaS cloud, spread between two different sites: the INFN Padova Unit and the INFN Legnaro National Labs. It provides about 1100 logical cores and 50 TB of storage. The entire computing facility, owned by INFN, satisfies the computational and storage demands of more than 100 users belonging to about 30 research projects, mainly related to HEP and nuclear physics. The Padova data centre also has hosted and operated since 2015 an independent IaaS cloud managing network, storage and computing resources owned by 10 departments of the University of Padova, supporting a broader range of scientific and engineering disciplines. This infrastructure provides about 480 logical cores and 90 TB of storage and supports more than 40 research projects. These two clouds share only a limited set of ICT services and tools (mainly for configuration, monitoring and accounting), whereas their daily operations and maintenance are carried out separately by INFN and University personnel. At the end of 2017 we planned to merge the two infrastructures in order to optimise the use of resources (both human and ICT) and to avoid useless duplication of services. We discuss here how we plan to implement this integration, resulting in a single cloud infrastructure named CloudVeneto.it