30 research outputs found
Deploying Jupyter Notebooks at scale on XSEDE resources for Science Gateways and workshops
Jupyter Notebooks have become a mainstream tool for interactive computing in
every field of science. Jupyter Notebooks are suitable as companion
applications for Science Gateways, providing more flexibility and
post-processing capability to the users. Moreover they are often used in
training events and workshops to provide immediate access to a pre-configured
interactive computing environment. The Jupyter team released the JupyterHub web
application to provide a platform where multiple users can login and access a
Jupyter Notebook environment. When the number of users and memory requirements
are low, it is easy to setup JupyterHub on a single server. However, setup
becomes more complicated when we need to serve Jupyter Notebooks at scale to
tens or hundreds of users. In this paper we will present three strategies for
deploying JupyterHub at scale on XSEDE resources. All options share the
deployment of JupyterHub on a Virtual Machine on XSEDE Jetstream. In the first
scenario, JupyterHub connects to a supercomputer and launches a single node job
on behalf of each user and proxies back the Notebook from the computing node
back to the user's browser. In the second scenario, implemented in the context
of a XSEDE consultation for the IRIS consortium for Seismology, we deploy
Docker in Swarm mode to coordinate many XSEDE Jetstream virtual machines to
provide Notebooks with persistent storage and quota. In the last scenario we
install the Kubernetes containers orchestration framework on Jetstream to
provide a fault-tolerant JupyterHub deployment with a distributed filesystem
and capability to scale to thousands of users. In the conclusion section we
provide a link to step-by-step tutorials complete with all the necessary
commands and configuration files to replicate these deployments.Comment: 7 pages, 3 figures, PEARC '18: Practice and Experience in Advanced
Research Computing, July 22--26, 2018, Pittsburgh, PA, US
Cyberinfrastructure, Science Gateways, Campus Bridging, and Cloud Computing
Computers accelerate our ability to achieve scientific
breakthroughs. As technology evolves and new research
needs come to light, the role for cyberinfrastructure as
âknowledgeâ infrastructure continues to expand. This
article defines and discusses cyberinfrastructure and the
related topics of science gateways and campus bridging;
identifies future challenges in cyberinfrastructure;
and discusses challenges and opportunities related to
the evolution of cyberinfrastructure, âbig dataâ (datacentric,
data-enabled, and data-intensive research and
data analytics), and cloud computing.This material is based upon work supported by the
National Science Foundation under grants 0504075,
0451237, 0723054, 1062432, 0116050, 0521433,
0503697, and 1053575, and several IBM Shared University
Research grants and support provided by Lilly
Endowment, Inc. for the Indiana University Pervasive
Technology Institute. Any opinions, findings and
conclusions or recommendations expressed herein are
those of the authors and do not necessarily reflect the
views of the supporting agencies
Science Gateways: The Long Road to the Birth of an Institute
Nowadays, research in various disciplines is enhanced via computational methods, cutting-edge technologies and diverse resources including computational infrastructures and instruments. Such infrastructures are often complex and researchers need means to conduct their research in an efficient way without getting distracted with information technology nuances. Science gateways address such demands and offer user interfaces tailored to a specific community. Creators of science gateways face a breadth of topics and manifold challenges, which necessitate close collaboration with the domain specialists but also calling in experts for diverse aspects of a science gateway such as project management, licensing, team composition, sustainability, HPC, visualization, and usability specialists. The Science Gateway Community Institute tackles the challenges around science gateways to support domain specialists and developers via connecting them to diverse experts, offering consultancy as well as providing a software collaborative, which contains ready-to-use science gateway frameworks and science gateway components
Summary of the First Workshop on Sustainable Software for Science: Practice and Experiences (WSSSPE1)
Challenges related to development, deployment, and maintenance of reusable software for science are becoming a growing concern. Many scientistsâ research increasingly depends on the quality and availability of software upon which their works are built. To highlight some of these issues and share experiences, the First Workshop on Sustainable Software for Science: Practice and Experiences (WSSSPE1) was held in November 2013 in conjunction with the SC13 Conference. The workshop featured keynote presentations and a large number (54) of solicited extended abstracts that were grouped into three themes and presented via panels. A set of collaborative notes of the presentations and discussion was taken during the workshop.
Unique perspectives were captured about issues such as comprehensive documentation, development and deployment practices, software licenses and career paths for developers. Attribution systems that account for evidence of software contribution and impact were also discussed. These include mechanisms such as Digital Object Identifiers, publication of âsoftware papersâ, and the use of online systems, for example source code repositories like GitHub. This paper summarizes the issues and shared experiences that were discussed, including cross-cutting issues and use cases. It joins a nascent literature seeking to understand what drives software work in science, and how it is impacted by the reward systems of science. These incentives can determine the extent to which developers are motivated to build software for the long-term, for the use of others, and whether to work collaboratively or separately. It also explores community building, leadership, and dynamics in relation to successful scientific software
CyberGIS - Toward synergistic advancement of cyberinfrastructure and GIScience: A workshop summary
This community activity report describes the outcomes of a CyberGIS workshop, held in conjunction with the UCGIS 2010 annual winter meeting and sponsored by the National Science Foundation (NSF) Office of Cyberinfrastructure. Over the one and one-half day workshop, a multidisciplinary group of experts from the international communities of cyberinfrastructure, GIScience, spatial analysis and modeling, and several other related scientific domains were brought together for a participatory meeting composed of both small- and large-group settings and to discuss the CyberGIS road map