Deploying Jupyter Notebooks at scale on XSEDE resources for Science Gateways and workshops

Jupyter Notebooks have become a mainstream tool for interactive computing in every field of science. Jupyter Notebooks are suitable as companion applications for Science Gateways, providing more flexibility and post-processing capability to the users. Moreover they are often used in training events and workshops to provide immediate access to a pre-configured interactive computing environment. The Jupyter team released the JupyterHub web application to provide a platform where multiple users can login and access a Jupyter Notebook environment. When the number of users and memory requirements are low, it is easy to setup JupyterHub on a single server. However, setup becomes more complicated when we need to serve Jupyter Notebooks at scale to tens or hundreds of users. In this paper we will present three strategies for deploying JupyterHub at scale on XSEDE resources. All options share the deployment of JupyterHub on a Virtual Machine on XSEDE Jetstream. In the first scenario, JupyterHub connects to a supercomputer and launches a single node job on behalf of each user and proxies back the Notebook from the computing node back to the user's browser. In the second scenario, implemented in the context of a XSEDE consultation for the IRIS consortium for Seismology, we deploy Docker in Swarm mode to coordinate many XSEDE Jetstream virtual machines to provide Notebooks with persistent storage and quota. In the last scenario we install the Kubernetes containers orchestration framework on Jetstream to provide a fault-tolerant JupyterHub deployment with a distributed filesystem and capability to scale to thousands of users. In the conclusion section we provide a link to step-by-step tutorials complete with all the necessary commands and configuration files to replicate these deployments.Comment: 7 pages, 3 figures, PEARC '18: Practice and Experience in Advanced Research Computing, July 22--26, 2018, Pittsburgh, PA, US

Introduction to Science Gateways and Portals Minitrack

N/

Cyberinfrastructure, Science Gateways, Campus Bridging, and Cloud Computing

Computers accelerate our ability to achieve scientific breakthroughs. As technology evolves and new research needs come to light, the role for cyberinfrastructure as “knowledge” infrastructure continues to expand. This article defines and discusses cyberinfrastructure and the related topics of science gateways and campus bridging; identifies future challenges in cyberinfrastructure; and discusses challenges and opportunities related to the evolution of cyberinfrastructure, “big data” (datacentric, data-enabled, and data-intensive research and data analytics), and cloud computing.This material is based upon work supported by the National Science Foundation under grants 0504075, 0451237, 0723054, 1062432, 0116050, 0521433, 0503697, and 1053575, and several IBM Shared University Research grants and support provided by Lilly Endowment, Inc. for the Indiana University Pervasive Technology Institute. Any opinions, findings and conclusions or recommendations expressed herein are those of the authors and do not necessarily reflect the views of the supporting agencies

Science Gateways: The Long Road to the Birth of an Institute

Nowadays, research in various disciplines is enhanced via computational methods, cutting-edge technologies and diverse resources including computational infrastructures and instruments. Such infrastructures are often complex and researchers need means to conduct their research in an efficient way without getting distracted with information technology nuances. Science gateways address such demands and offer user interfaces tailored to a specific community. Creators of science gateways face a breadth of topics and manifold challenges, which necessitate close collaboration with the domain specialists but also calling in experts for diverse aspects of a science gateway such as project management, licensing, team composition, sustainability, HPC, visualization, and usability specialists. The Science Gateway Community Institute tackles the challenges around science gateways to support domain specialists and developers via connecting them to diverse experts, offering consultancy as well as providing a software collaborative, which contains ready-to-use science gateway frameworks and science gateway components

Summary of the First Workshop on Sustainable Software for Science: Practice and Experiences (WSSSPE1)

Challenges related to development, deployment, and maintenance of reusable software for science are becoming a growing concern. Many scientists’ research increasingly depends on the quality and availability of software upon which their works are built. To highlight some of these issues and share experiences, the First Workshop on Sustainable Software for Science: Practice and Experiences (WSSSPE1) was held in November 2013 in conjunction with the SC13 Conference. The workshop featured keynote presentations and a large number (54) of solicited extended abstracts that were grouped into three themes and presented via panels. A set of collaborative notes of the presentations and discussion was taken during the workshop. Unique perspectives were captured about issues such as comprehensive documentation, development and deployment practices, software licenses and career paths for developers. Attribution systems that account for evidence of software contribution and impact were also discussed. These include mechanisms such as Digital Object Identifiers, publication of “software papers”, and the use of online systems, for example source code repositories like GitHub. This paper summarizes the issues and shared experiences that were discussed, including cross-cutting issues and use cases. It joins a nascent literature seeking to understand what drives software work in science, and how it is impacted by the reward systems of science. These incentives can determine the extent to which developers are motivated to build software for the long-term, for the use of others, and whether to work collaboratively or separately. It also explores community building, leadership, and dynamics in relation to successful scientific software

The Global Impact of Science Gateways, Virtual Research Environments and Virtual Laboratories

Science gateways, virtual laboratories and virtual research environments are all terms used to refer to community-developed digital environments that are designed to meet a set of needs for a research community. Specifically, they refer to integrated access to research community resources including software, data, collaboration tools, workflows, instrumentation and high-performance computing, usually via Web and mobile applications. Science gateways, virtual laboratories and virtual research environments are enabling significant contributions to many research domains, facilitating more efficient, open, reproducible research in bold new ways. This paper explores the global impact achieved by the sum effects of these programs in increasing research impact, demonstrates their value in the broader digital landscape and discusses future opportunities. This is evidenced through examination of national and international programs in this field

MOTIVASI, TINGKAT PENDIDIKAN, DAN STATUS SOSIAL EKONOMI DIHUBUNGKAN DENGAN HASIL-HASIL PELATIHAN DAN PRAKTEK KETERAMPILAN KERJA PADA PELATIHAN CALON BABY SITTER: Studi Analisa Data terhadap Calon Baby Sitter di Panti Penitipan Bayi "Ade Irma Suryani Nasution" Medan

Fokus permasalahan yang diangkat dalam Tesis ini adalah Motivasi, tingkat pendidikan dan status sosial ekonomi dihubungkan dengan hasil-hasil pelatihan dan praktek keterampilan kerja calon baby sitter. Karena keempat faktor ini merupakan suatu pernyataan yang abstrak maka perlu dijabarkan ke dalam indikator variabel sesuai dengan tujuan penelitian yaitu untuk mengetahui tingkat motivasi, tingkat pendidikan, status sosial ekonomi yang dihubungkan dengan hasil-hasil pelatihan dan praktek keterampilan kerja calon baby sitter, dan untuk menguji adakah hubungan antara keempat variabel tersebut. Penelitian ini berlokasi di Jalan T. Cik Ditiro, Kecamatan Medan Baru - Medan. Teori yang digunakan di dalam penelitian ini adalah teori-teori tentang Motivasi yang dikutip dari pendapat W.J.S. Poerwodarminta (1982: 655), Mc. Clelland (1987: 51) tentang "Social motives theory", dimana padadasarnya semua pekerjaan membutuhkan motivasi yang cukup bagi setiap orang untuk melaksanakan suatu pekerjaan dengan berhasil. Selanjutnya teori-teori tentang Tingkat Pendidikan yang dikutip dari pendapat Moh. Surya (1990: 20) tentang taman kanak-kanak, Kartini Kartono (1986: 139), tentang tingkat Sekolah Dasar, Zulkifli L (1987: 87), tentang tingkat SLTP dan tingkat SLTAdan universitas, dimana tinggi rendahnya pendidikan seseorang akan mencerminkan tingkat pengetahuan dan sikap serta prilakunya. Teori-teori tentang Status Sosial Ekonomi yang dikutip dari pendapat Mayor Polak (1979), Miller (1964) dan SC. Utami Munandar (1982) dimana tinggi-rendahnya status sosial ekonomi orang tua cenderung menentukan perbedaan tanggung jawab orang tua pada anaknya. Teoriteori tentang training oleh Sikula (1976), Mulyanan Sugandi (1993), dimana dijelaskan bahwa program latihan kerja dapat mempertinggi kemampuan kerja dengan mengembangkan cara-cara berpikir dan bertindak yang tepat serta memiliki pengetahuan tentang tugas. Penelitian ini menggunakan metode deskriptif dengan tehnik studi Analisis data dan juga digolongkan ke dalam penelitian eksplanasi (explanatory research), karena penelitian ini juga berupaya meneliti hubungan variabel yang dihipotesiskan. Penelitian ini menggunakan pendekatan kuantitatif dengan perhitungan statistik non-parametrik. Sampel dalam penelitian ini adalah sampel total, yaitu 50 orang responden yang mengikuti latihan calon baby sitter di Panti Perawatan dan Penitipan Bayi "ADE IRMA SURYANI NASUTION" Medan. Teknik analisis data yang digunakan adalah statistik non-parametrik. Alat pengumpul data yang dipergunakan untuk variabel independen adalah Kuesioner yang terlebih dahulu diuji validitas dan reliabilitasnya, sedangkan alat pengumpul data untuk variabel dependen adalah dokumentasi. Setelah dilakukan pengolahan dan analisa data diperoleh hasil penelitian sebagai berikut: Pertama; Hasil perhitungan statistik Yule's Qmenunjukkan QxiYa = 0,6 dan chi kuadratnya = 3,853 jadi dapat ditarik kesimpulan terdapat hubungan positif yang mantap dan signifikan antara motivasi dengan hasil-hasil pelatihan dimana *2 hitung 3,853 > %2 urn 3,84 pada dk = 1 dengan taraf kepercayaan 0,95% dan hasil perhitungan statistik Yule's Qmenunjukkan QxiYb = 0,54 dan uji chi kuadratnva = 3,4. "rial ini mengandung makna bahwa terdapat hubungan positif yang mantap serta berkonstribusi antara motivasi dengan praktek keterampilan kerja, dimana y? ^ung =3,4 <X2 ,abe. 3,84 pada dk = 1dengan taraf kepercayaan 0,95%. Kedua: Hasil perhitungan statistik Yule's Qmenunjukkan QX2Ya - - 0,45 dan chi kuadratnva = 1,86 makna yang dapat ditarik dari analisis ini yaitu terdapat hubungan negatif vang sedang tetapi tidak signifikan antara tingkat pendidikan dengan hasil-hasil pelatihan dimana %2 wtung =1,86 <X2 tabci 3,84. Sedangkan untuk perhitungan statistik Yule's QQX2Yb =- 0,25 dan chi kuadratnya =0,52. Ini dapat diartikan bahwa terdapat hubungan negatif yang rendah dan tidak signifikan antara tingkat pendidikan dengan praktek keterampilan kerja, dimana %2 wtung =0,52 < X2tabei 384 pada dk =1dengan taraf kepercayaan 0,95%. Ketiga Hasil perhitungan statistik Yule's Qmenunjukkan Qxsva =0,153 dan uji chi kuadratnya =0,23 dapat diartikan bahwa terdapat hubungan positif yang rendah tetapi tidak signifikan antara status sosial ekonomi dengan hasil-hasil pelatihan dimana x hhung = 0,23 < X2 .abei 3,84 pada dk = 1 dengan taraf kepercayaan 0,95%. Sedangkan untuk perhitungan statistik Yule's Qmenunjukkan Qxavb =0,4 dan uji chi kuadratnya - 175 Ini dapat diartikan bahwa terdapat hubungan positif yang sedang antara status sosial ekonomi dengan praktek keterampilan kerja tetapi tidak signifikan, dimana X2 hitung =1J <X2 tabei 3,84 pada dk =1dengan taraf kepercayaan 0,95%. Dari temuan penelitian ini dapat disimpulkan bahwa semakin tinggi motivasi calon baby sitter untuk mengikuti pelatihan maka semakin tinggi hasilhasil pelatihan yang akan diperolehnya. Begitu juga di dalam memperoleh nilai praktek, motivasi seseorang sangat berpengaruh atau konstnbusi yang dibenkan oleh motivasi terhadap nilai praktek sangat besar. Sedangkan untuk tingkat pendidikan dapat disimpulkan bahwa untuk menjadi seorang baby sitter tidak dibutuhkan orang-orang yang berpendidikan tinggi cukup hanya pada tingkat pendidikan SLTP dan SLTA saja, hal ini dapat dibuktikan dan hasil penelitian yang dilakukan tidak terdapat hubungan yang signifikan antara tingkat pendidikan dengan hasil-hasil pelatihan (nilai teori), begitu juga untuk hubungan antara tingkat pendidikan dengan praktek keterampilan kerja (nilai praktek) tidak berkorelasi, artinya semakin tinggi tingkat pendidikan seseorang ada kecenderungan semakin baik pelaksanaan praktek keterampilan kerjanya. Selanjutnya yang berkaitan dengan status sosial ekonomi orang tua calon baby sitter sama sekali tidak mempengaruhi kesempatan bagi calon baby sitter untuk berprestasi, atau dengan kata lain walaupun sebagian besar orang tua calon baby sitter termasuk dalam status sosial ekonomi kategori rendah tetapi tidak menutup kemungkinan bagi anaknya (calon baby sitter) yang mengikuti pelatihan untuk memihki nilai teon dan praktek keterampilan kerja yang tinggi

CodeCloud: A platform to enable execution of programming models on the Clouds

This paper presents a platform that supports the execution of scientific applications covering different programming models (such as Master/Slave, Parallel/MPI, MapReduce and Workflows) on Cloud infrastructures. The platform includes (i) a high-level declarative language to express the requirements of the applications featuring software customization at runtime, (ii) an approach based on virtual containers to encapsulate the logic of the different programming models, (iii) an infrastructure manager to interact with different IaaS backends, (iv) a configuration software to dynamically configure the provisioned resources and (v) a catalog and repository of virtual machine images. By using this platform, an application developer can adapt, deploy and execute parallel applications agnostic to the Cloud backend.The authors wish to thank the financial support received from both the Spanish Ministry of Economy and Competitiveness to develop the project "Servicios avanzados para el despliegue y contextualizacion de aplicaciones virtualizadas para dar soporte a modelos de programacion en entornos cloud", with reference TIN2010-17804.Caballer Fernández, M.; Alfonso Laguna, CD.; Moltó, G.; Romero Alcalde, E.; Blanquer Espert, I.; García García, A. (2014). CodeCloud: A platform to enable execution of programming models on the Clouds. Journal of Systems and Software. 93:187-198. https://doi.org/10.1016/j.jss.2014.02.005S1871989

Measuring Success for a Future Vision: Defining Impact in Science Gateways/Virtual Research Environments

Scholars worldwide leverage science gateways/VREs for a wide variety of research and education endeavors spanning diverse scientific fields. Evaluating the value of a given science gateway/VRE to its constituent community is critical in obtaining the financial and human resources necessary to sustain operations and increase adoption in the user community. In this paper, we feature a variety of exemplar science gateways/VREs and detail how they define impact in terms of e.g., their purpose, operation principles, and size of user base. Further, the exemplars recognize that their science gateways/VREs will continuously evolve with technological advancements and standards in cloud computing platforms, web service architectures, data management tools and cybersecurity. Correspondingly, we present a number of technology advances that could be incorporated in next-generation science gateways/VREs to enhance their scope and scale of their operations for greater success/impact. The exemplars are selected from owners of science gateways in the Science Gateways Community Institute (SGCI) clientele in the United States, and from the owners of VREs in the International Virtual Research Environment Interest Group (VRE-IG) of the Research Data Alliance. Thus, community-driven best practices and technology advances are compiled from diverse expert groups with an international perspective to envisage futuristic science gateway/VRE innovations