A distributed Quadtree Dictionary approach to multi-resolution visualization of scattered neutron data

Grid computing is described as dependable, seamless, pervasive access to resources and services, whereas mobile computing allows the movement of people from place to place while staying connected to resources at each location. Mobile grid computing is a new computing paradigm, which joins these two technologies by enabling access to the collection of resources within a user\u27s virtual organization while still maintaining the freedom of mobile computing through a service paradigm. A major problem in virtual organization is needs mismatch, in which one resources requests a service from another resources it is unable to fulfill, since virtual organizations are necessarily heterogeneous collections of resources. In this dissertation we propose a solution to the needs mismatch problem in the case of high energy physics data. Specifically, we propose a Quadtree Dictionary (QTD) algorithm to provide lossless, multi-resolution compression of datasets and enable their visualization on devices of all capabilities. As a prototype application, we extend the Integrated Spectral Analysis Workbench (ISAW) developed at the Intense Pulsed Neutron Source Division of the Argonne National Laboratory into a mobile Grid application, Mobile ISAW. In this dissertation we compare our QTD algorithm with several existing compression techniques on ISAW\u27s Single-Crystal Diffractometer (SCD) datasets. We then extend our QTD algorithm to a distributed setting and examine its effectiveness on the next generation of SCD datasets. In both a serial and distributed setting, our QTD algorithm performs no worse than existing techniques such as the square wavelet transform in terms of energy conservation, while providing the worst-case savings of 8:1

Virtualizing the Stampede2 Supercomputer with Applications to HPC in the Cloud

Methods developed at the Texas Advanced Computing Center (TACC) are described and demonstrated for automating the construction of an elastic, virtual cluster emulating the Stampede2 high performance computing (HPC) system. The cluster can be built and/or scaled in a matter of minutes on the Jetstream self-service cloud system and shares many properties of the original Stampede2, including: i) common identity management, ii) access to the same file systems, iii) equivalent software application stack and module system, iv) similar job scheduling interface via Slurm. We measure time-to-solution for a number of common scientific applications on our virtual cluster against equivalent runs on Stampede2 and develop an application profile where performance is similar or otherwise acceptable. For such applications, the virtual cluster provides an effective form of "cloud bursting" with the potential to significantly improve overall turnaround time, particularly when Stampede2 is experiencing long queue wait times. In addition, the virtual cluster can be used for test and debug without directly impacting Stampede2. We conclude with a discussion of how science gateways can leverage the TACC Jobs API web service to incorporate this cloud bursting technique transparently to the end user.Comment: 6 pages, 0 figures, PEARC '18: Practice and Experience in Advanced Research Computing, July 22--26, 2018, Pittsburgh, PA, US

Taking the Next Step: Building an Arabidopsis Information Portal

The Arabidopsis Information Portal (AIP), a resource expected to provide access to all community data and combine outputs into a single user-friendly interface, has emerged from community discussions over the last 23 months. These discussions began during two closely linked workshops in early 2010 that established the International Arabidopsis Informatics Consortium (IAIC). The design of the AIP will provide core functionality while remaining flexible to encourage multiple contributors and constant innovation. An IAIC-hosted Design Workshop in December 2011 proposed a structure for the AIP to provide a framework for the minimal components of a functional community portal while retaining flexibility to rapidly extend the resource to other species. We now invite broader participation in the AIP development process so that the resource can be implemented in a timely manner.This is the publisher’s final pdf. The published article is copyrighted by the American Society of Plant Biologists and can be found at: http://www.plantcell.org/

2011 Report of NSF Workshop Series on Scientific Software Security Innovation Institute

Over the period of 2010-2011, a series of two workshops were held in response to NSF Dear Colleague Letter NSF 10-050 calling for exploratory workshops to consider requirements for Scientific Software Innovation Institutes (S2I2s). The specific topic of the workshop series was the potential benefits of a security-focused software institute that would serve the entire NSF research and development community. The first workshop was held on August 6th, 2010 in Arlington, VA and represented an initial exploration of the topic. The second workshop was held on October 26th, 2011 in Chicago, IL and its goals were to 1) Extend our understanding of relevant needs of MREFC and large NSF Projects, 2) refine outcome from first workshop with broader community input, and 3) vet concepts for a trusted cyberinfrastructure institute. Towards those goals, the participants other 2011workshop included greater representation from MREFC and large NSF projects, and, for the most part, did not overlap with the participants from the 2010 workshop. A highlight of the second workshop was, at the invitation of the organizers, a presentation by Scott Koranda of the LIGO project on the history of LIGO’s identity management activities and how those could have benefited from a security institute. A key analysis he presented is that, by his estimation, LIGO could have saved 2 senior FTE-years of effort by following suitable expert guidance had it existed. The overarching finding from the workshops is that security is a critical crosscutting issue for the NSF software infrastructure and recommended a security focused activity to address this issue broadly, for example a security software institute (S2I2) under the SI2 program. Additionally, the 2010 workshop participants agreed to 15 key additional findings, which the 2011 workshop confirmed, with some refinement as discussed in this report.NSF Grant # 1043843Ope

The iPlant Collaborative: Cyberinfrastructure for Plant Biology

The iPlant Collaborative (iPlant) is a United States National Science Foundation (NSF) funded project that aims to create an innovative, comprehensive, and foundational cyberinfrastructure in support of plant biology research (PSCIC, 2006). iPlant is developing cyberinfrastructure that uniquely enables scientists throughout the diverse fields that comprise plant biology to address Grand Challenges in new ways, to stimulate and facilitate cross-disciplinary research, to promote biology and computer science research interactions, and to train the next generation of scientists on the use of cyberinfrastructure in research and education. Meeting humanity's projected demands for agricultural and forest products and the expectation that natural ecosystems be managed sustainably will require synergies from the application of information technologies. The iPlant cyberinfrastructure design is based on an unprecedented period of research community input, and leverages developments in high-performance computing, data storage, and cyberinfrastructure for the physical sciences. iPlant is an open-source project with application programming interfaces that allow the community to extend the infrastructure to meet its needs. iPlant is sponsoring community-driven workshops addressing specific scientific questions via analysis tool integration and hypothesis testing. These workshops teach researchers how to add bioinformatics tools and/or datasets into the iPlant cyberinfrastructure enabling plant scientists to perform complex analyses on large datasets without the need to master the command-line or high-performance computational services

Trusted CI Webinar: Best Practices for Academic Cloud Service Providers with Rion Dooley

A “cloud resource” provides a hosted, self-service means for users to run virtual machines or containers such that they can have a custom software stack and isolation from other users. Virtual machines or container images can be curated and provided by the cloud resource operator, provided by the user, or provided by third parties. Operating a cloud resource involves addressing security requirements of multiple stakeholders: those of the resource operator and those of the resource user. These parties may have different incentives related to security as well as different levels of acumen. Operators may at times run images whose trustworthiness is not established and grant users privileged access within their running image that would be uncommon on non-virtualized computing resources. Moreover, users, with their elevated privileges, can misconfigure services, expose sensitive data or choose protocols/solutions that offer less security for the sake of installation or operating costs. These factors can lead to an environment that, by its nature, is difficult to secure. A community consisting of The Agave Platform, Cornell University Center for Advanced Computing, CyVerse, Jetstream and Trusted CI collaborated in authoring a set of Security Best Practices for developing in, and operating an academic cloud resource. In this webinar, we will discuss the nine use cases they deemed most important to academic cloud services. This webinar will be relevant to cloud users, evangelists, and providers. All are encouraged to join and contribute to the conversation. The full white paper is available online at http://hdl.handle.net/2022/22123.NSF Grant #1547272NSF Grant #1450459Ope

Reactive, Interactive, and High Throughput Computation in the Agave Platform

The Agave Platform is an open Science-as-a-Service platform that empowers users to run code, manage data, collaborate meaningfully, and integrate easily with the world around them. In this extended abstract, we examine how new use cases have helped evolve the ways in which Agave supports code execution. We start out by identifying three new code execution paradigms identified in the past 2 years: Interactive, Reactive, and High Throughput computing. We then briefly describe the primary use cases driving each paradigm and how the platform evolved in response to them. Finally, we highlight areas of future work in the platform related to code execution

Containers-as-a-service via the Actor Model

Modern science gateways face distributed systems engineering challenges from multiple fronts. In this extended abstract, we introduce Abaco (Actor Based Containers), a framework and hosted service based on Linux containers and the actor model of concurrent computing for tackling problems related to portability, scalability, reproducibility, and the heterogeneity of physical infrastructure powering science gateways. We describe the initial implementation of Abaco, its use in the Agave science-as-a-service platform, as well as some related projects and areas of future work