An ARM cluster for running CMSSW jobs

The ARM platform extends from the mobile phone area to development board computers and servers. It could be that in the future the importance of the ARM platform will increase for High Performance Computing/High Throughput Computing (HPC/HTC) if new more powerful (server) boards are released. For this reason Compact Muon Solenoid Software (CMSSW) has previously been ported to ARM in earlier work. The CMSSW is deployed using the CERN Virtual Machine File System (CVMFS) and the jobs are run inside Singularity containers. Some ARM AArch64 CMSSW releases are available in CVMFS for testing and development. In this work CVMFS and Singularity have been compiled and installed on an ARM cluster and the AArch64 CMSSW releases in CVMFS have been used. We report on our experiences with this ARM cluster for CMSSW jobs. Commodity hardware designed around the 64-bit architecture has been the basis of current virtualization trends with the advantage to emulate diverse environments for a wide range of computational scenarios. However, in parallel the mobile revolution have given a rise to ARM SoCs with primary focus on power efficiency. While still in the experimental phase, the power efficiency and 64-bit heterogeneous computing already point to an alternative option for traditional x86_64 CPUs servers for datacenters.In this paper we present the latest CMS open data release published on the CERN Oopen Data portal. Samples of collision and simulated datasets were released together with detailed information about the data provenance. The associated data production chains cover the necessary computing environments, the configuration files and the computational procedures used in each data production step. We describe data curation techniques used to obtain and publish the data provenance information and we study the possibility of reproducing parts of the released data using the publicly available information. The present work demonstrates the usefulness of releasing selected samples of raw and primary data in order to fully ensure the completeness of information about the data production chain for the attention of general data scientists and other non-specialists interested in using particle physics data for education or research purposes.Peer reviewe

Secure Cloud Connectivity for Scientific Applications

Cloud computing improves utilization and flexibility in allocating computing resources while reducing the infrastructural costs. However, in many cases cloud technology is still proprietary and tainted by security issues rooted in the multi-user and hybrid cloud environment. A lack of secure connectivity in a hybrid cloud environment hinders the adaptation of clouds by scientific communities that require scaling-out of the local infrastructure using publicly available resources for large-scale experiments. In this article, we present a case study of the DII-HEP secure cloud infrastructure and propose an approach to securely scale-out a private cloud deployment to public clouds in order to support hybrid cloud scenarios. A challenge in such scenarios is that cloud vendors may offer varying and possibly incompatible ways to isolate and interconnect virtual machines located in different cloud networks. Our approach is tenant driven in the sense that the tenant provides its connectivity mechanism. We provide a qualitative and quantitative analysis of a number of alternatives to solve this problem. We have chosen one of the standardized alternatives, Host Identity Protocol, for further experimentation in a production system because it supports legacy applications in a topologically-independent and secure way.Peer reviewe

Building Scalable Cloud Infrastructures with OpenStack and GlusterFS

With the recent advances in efficient virtualization techniques in using commodity servers cloud computing has emerged as a powerful technology to meet new requirements for supporting a new generation of computing services based on utility model. However barriers to widespread adoption still exists and the dominant platform is yet to be seen in years to come. Hence the challenge of providing scalable cloud infrastructures requires a continuous exploration of new technologies and techniques. This thesis describes an experimental investigation of integrating two such open source technologies, OpenStack and GlusterFS, to build our cloud environment. We designed a number of test case scenarios that help us answer the questions around performance, stability and scalability of the cloud infrastructure deployed. Additionally, the work based on this thesis was accepted to the Conference on Computing in High Energy and Nuclear Physics (CHEP2013), and the paper is due for publishing

Multicloud Management Plane : Experiments with Scientific and Telco Clouds

Virtualization is the foundation of what experience today as Cloud Computing. Delivering software and services initially through virtual machines and later through containers, has the industry and research efforts centered around OpenStack and Kubernetes. Both projects are community driven and industry supported, thus providing flexible frameworks for virtualizing large scale production infrastructures, yet primarily within a single administrative domain. In this thesis, we provide architecture solutions that extend further OpenStack and Kubernetes for managing heterogeneous resources across multiple cloud environments. More specifically, with OpenStack we use GlusterFS for a storage solution that brings stability and scalability for a private deployed cloud, and Host Identity Protocol as a secure and scalable network technique for scaling workloads across many environments. With Kubernetes we use KubeFed and Network Service Mesh in providing federation and cross-cluster connectivity for container workloads. Nonetheless, adding flexibility also brings new challenging problems in terms of performance and stability. We perform experimental evaluations using workloads and scenarios particular to our representative use-cases from science and industry, namely CERN and Telecom providers. The results indicate the techniques employed have multicloud capabilities for managing and deploying workloads at large scale respective to our chosen use-cases.Virtualisointi on tämän päivän pilvilaskennan kokemuksen perusta. Ohjelmistojen ja palveluiden toimittaminen alun perin virtuaalikoneiden ja myöhemmin ohjelmistosäiliöiden kautta on keskittänyt teollisuuden ja tutkimuksen ponnistelut OpenStackiin ja Kubernetesiin. Molemmat hankkeet ovat yhteisövetoisia ja teollisuuden tukemia, mikä tarjoaa joustavia puitteita laajamittaisten tuotantoinfrastruktuurien virtualisoimiseen, kuitenkin ensisijaisesti yhden hallintoalueen sisällä. Tässä väitöskirjassa tarjoamme arkkitehtuuriratkaisuja, jotka laajentavat OpenStackia ja Kubernetesia heterogeenisten resurssien hallintaan useissa pilviympäristöissä. Tarkemmin sanottuna OpenStackissa käytämme GlusterFS:ää tallennusratkaisuna, joka tuo vakautta ja skaalautuvuutta yksityiseen pilvipalveluun ja Host Identity Protocolia turvallisena ja skaalautuvana verkkotekniikkana työkuormien skaalaamiseen eri pilvijärjestelmien yli. Kubernetesin avulla käytämme KubeFediä ja Network Service Meshiä tarjoamaan federointia ja klusterien välisiä yhteyksiä ohjelmistosäiliöiden työkuormille. Joustavuuden lisääminen tuo kuitenkin myös uusia haastavia suorituskyvyn ja vakauden ongelmia. Suoritamme kokeellisia arviointeja käyttämällä työkuormia ja skenaarioita, jotka ovat erityisesti edustavia tieteen ja teollisuuden, eli CERNin ja Telecom-palveluntarjoajien, käyttötapauksia. Tulokset osoittavat, että käytetyillä tekniikoilla on monipilvi-ominaisuudet työkuormien hallintaan ja käyttöönottamiseksi suuressa mittakaavassa valitsemiemme käyttötapausten mukaisesti

An ARM cluster for running CMSSW jobs

The ARM platform extends from the mobile phone area to development board computers and servers. It could be that in the future the importance of the ARM platform will increase for High Performance Computing/High Throughput Computing (HPC/HTC) if new more powerful (server) boards are released. For this reason Compact Muon Solenoid Software (CMSSW) has previously been ported to ARM in earlier work. The CMSSW is deployed using the CERN Virtual Machine File System (CVMFS) and the jobs are run inside Singularity containers. Some ARM AArch64 CMSSW releases are available in CVMFS for testing and development. In this work CVMFS and Singularity have been compiled and installed on an ARM cluster and the AArch64 CMSSW releases in CVMFS have been used. We report on our experiences with this ARM cluster for CMSSW jobs. Commodity hardware designed around the 64-bit architecture has been the basis of current virtualization trends with the advantage to emulate diverse environments for a wide range of computational scenarios. However, in parallel the mobile revolution have given a rise to ARM SoCs with primary focus on power efficiency. While still in the experimental phase, the power efficiency and 64-bit heterogeneous computing already point to an alternative option for traditional x86_64 CPUs servers for datacenters

Toxicity tests using living organisms

Toxic chemicals pose a serious and continuous risk for specific parts of the food chain, leading to the extinction of different communities. The effects of toxins might be direct or indirect. The sub lethal effects lead to the change of the behavior of an organism. The acute or chronic tests might be performed for water or terrestrial organisms, according to the OECD standards, in different levels of the food chain and thus it is possible to predict the pollution effect in the ecosystem. This study has taken in consideration the acute and chronic tests using Daphnia sp. The goal of the research is the behavior evaluation of these organisms, common inhabitants of water systems, towards the change of the concentration of different chemicals. These organisms are taken from their natural habitat and then raised in laboratory conditions, in accordance with their natural habitat needs. Acute tests are performed using different concentrations of specific chemicals. Physical-chemical parameters of their natural water habitat are taken in consideration as well. The experimental laboratory data are then elaborated in order to determine the highest concentration of the tested chemical that shows adverse effects on Daphnia population, as well as the lowest concentration with no effect. The laboratory results are evaluated and compared with other similar studies

Drinking water quality determination through the water pollution indicators, Elbasan district

Today's drinking water supply in the Elbasan district continues to be an issue of concern. The population growth nowadays is accompanied by the increase of request for water use. To complete their needs, people have chosen to use groundwater by opening wells. Having their personal well does not guarantee water safety. The aim of this paper is to determine the quality of drinking water through coliform bacteria indicators, in Elbasan district. There are three groups of coliform bacteria, total coliform, intestinal enterococci and Escherichia coli. Each of these is an indicator of drinking water quality and each has a different level of risk. In this study are analyzed many water samples taken from villages Labinot fushë, Gjergjan, Papër, Shushicë, and Bradashesh. In some of these villages, the water samples are taken from wells and pipes. The total coliform, E. coli, and intestinal enterococci in drinking water of these villages are reflected in this study. The data recorded shows which of the analyzed water sources are within the international standards to be used as drinking water. The coliform pollution levels are higher in the wells water than in the pipes one. The distant areas from the city of Elbasan are more polluted than those nearby