A DevOps approach to integration of software components in an EU research project

We present a description of the development and deployment infrastructure being created to support the integration effort of HARNESS, an EU FP7 project. HARNESS is a multi-partner research project intended to bring the power of heterogeneous resources to the cloud. It consists of a number of different services and technologies that interact with the OpenStack cloud computing platform at various levels. Many of these components are being developed independently by different teams at different locations across Europe, and keeping the work fully integrated is a challenge. We use a combination of Vagrant based virtual machines, Docker containers, and Ansible playbooks to provide a consistent and up-to-date environment to each developer. The same playbooks used to configure local virtual machines are also used to manage a static testbed with heterogeneous compute and storage devices, and to automate ephemeral larger-scale deployments to Grid5000. Access to internal projects is managed by GitLab, and automated testing of services within Docker-based environments and integrated deployments within virtual-machines is provided by Buildbot

Ecosystem (dis)benefits arising from formal and informal land-use in Manchester (UK); a case study of urban soil characteristics associated with local green space management

Urban soils are subject to anthropogenic influences and, reciprocally, provide benefits and dis-benefits to human wellbeing; for example carbon storage, nutrient cycling and the regulation trace element and contaminant mobility. Collective stewardship of urban green commons provides contemporary examples of the diversity of uses and management of green space in cities and represents a growing movement in user participation in, and awareness of, the importance of urban ecological health. Exploring the range of social-ecological benefits exemplified in the urban environment has generally focused on above-ground processes, with few studies examining the potential for (dis)benefits arising from edaphic characteristics of collectively-managed spaces. An investigation into the influence of formal and informal green space management on carbon fluxes and heavy metal concentrations in urban soils was carried out in Manchester (UK) finding that carbon storage in soils of collectively managed urban green commons (7.15 ±1.42 kg C m⁻²) was significantly greater than at formally managed sites (for example city parks: 5.08 ±0.69 kg C m⁻²), though the latter exhibited reduced losses through CO2 emission. Variation in heavy metal concentrations and mobility were likewise observed, exemplified by the acidification of surface soils by leaf litter at orchard sites, and the resultant increase in the mobility of lead (Pb) and zinc (Zn). The results of this study indicate the importance of small-scale contemporary urban green space management on selected ecosystem services provided by the limited soil resource of cities. Thus, a greater consideration of the effects of horticultural and amenity activities with regards to soil quality/functionality is required to ensure available urban green commons retain or increase their ecological quality over time

From Bare Metal to Virtual: Lessons Learned when a Supercomputing Institute Deploys its First Cloud

As primary provider for research computing services at the University of Minnesota, the Minnesota Supercomputing Institute (MSI) has long been responsible for serving the needs of a user-base numbering in the thousands. In recent years, MSI---like many other HPC centers---has observed a growing need for self-service, on-demand, data-intensive research, as well as the emergence of many new controlled-access datasets for research purposes. In light of this, MSI constructed a new on-premise cloud service, named Stratus, which is architected from the ground up to easily satisfy data-use agreements and fill four gaps left by traditional HPC. The resulting OpenStack cloud, constructed from HPC-specific compute nodes and backed by Ceph storage, is designed to fully comply with controls set forth by the NIH Genomic Data Sharing Policy. Herein, we present twelve lessons learned during the ambitious sprint to take Stratus from inception and into production in less than 18 months. Important, and often overlooked, components of this timeline included the development of new leadership roles, staff and user training, and user support documentation. Along the way, the lessons learned extended well beyond the technical challenges often associated with acquiring, configuring, and maintaining large-scale systems.Comment: 8 pages, 5 figures, PEARC '18: Practice and Experience in Advanced Research Computing, July 22--26, 2018, Pittsburgh, PA, US

Handling Confidential Data on the Untrusted Cloud: An Agent-based Approach

Cloud computing allows shared computer and storage facilities to be used by a multitude of clients. While cloud management is centralized, the information resides in the cloud and information sharing can be implemented via off-the-shelf techniques for multiuser databases. Users, however, are very diffident for not having full control over their sensitive data. Untrusted database-as-a-server techniques are neither readily extendable to the cloud environment nor easily understandable by non-technical users. To solve this problem, we present an approach where agents share reserved data in a secure manner by the use of simple grant-and-revoke permissions on shared data.Comment: 7 pages, 9 figures, Cloud Computing 201

Sharing a conceptual model of grid resources and services

Grid technologies aim at enabling a coordinated resource-sharing and problem-solving capabilities over local and wide area networks and span locations, organizations, machine architectures and software boundaries. The heterogeneity of involved resources and the need for interoperability among different grid middlewares require the sharing of a common information model. Abstractions of different flavors of resources and services and conceptual schemas of domain specific entities require a collaboration effort in order to enable a coherent information services cooperation. With this paper, we present the result of our experience in grid resources and services modelling carried out within the Grid Laboratory Uniform Environment (GLUE) effort, a joint US and EU High Energy Physics projects collaboration towards grid interoperability. The first implementation-neutral agreement on services such as batch computing and storage manager, resources such as the hierarchy cluster, sub-cluster, host and the storage library are presented. Design guidelines and operational results are depicted together with open issues and future evolutions.Comment: 4 pages, 0 figures, CHEP 200