CSE Collaboration through Software: Improving Productivity and Sustainability

Tutorial presented at SIAM CSE17: CSE Collaboration through Software: Improving Productivity and Sustainability. http://meetings.siam.org/sess/dsp_programsess.cfm?SESSIONCODE=61488<br><br>A recording of this tutorial presentation is available at https://www.pathlms.com/siam/courses/4150/sections/5826/video_presentations/42638<br

Toward Community Software Ecosystems for High-Performance Computational Science

<p><i>Invited presentation at the 2018 SIAM Conference on Parallel Processing for Scientific Computing (SIAM-PP18),</i></p><p><i>March 8, 2018<br></i></p><p><br></p><p><b>Abstract:</b></p><p>Software—crosscutting technology that connects advances in mathematics, computer science, and domain-specific science and engineering—is a cornerstone of long-term collaboration and scientific progress. As we leverage unprecedented high-performance computing resources to work toward predictive science, software complexity is increasing due to multiphysics and multiscale modeling, the coupling of simulations and data analytics, and the demand for greater reproducibility in the midst of disruptive architectural changes. Applications increasingly require the combined use of independent software packages, which have diverse sponsors, priorities, and processes for development and release.</p><p> </p><p>These challenges create the unique opportunity to fundamentally change how scientific software is designed, developed, and sustained---with explicit work toward scientific software ecosystems. This presentation will introduce the xSDK, or Extreme-scale Scientific Software Development Kit, where community-defined policies are increasing the quality and interoperability across numerical libraries as needed by the DOE Exascale Computing Project. We will also discuss complementary efforts to increase scientific software productivity and sustainability.</p

Improving and Accelerating Scientific Discovery through Better Software

Whitepaper submitted to the 2017 DOE ASCR Applied Math Meeting.<div><div><div><div> </div> </div> </div></div><div><br></div><div>From the list of questions at https://www.orau.gov/ascr-appliedmath-pi2017/whitepaper-questions.htm, this white paper addresses the question 3b:</div><div><br></div><div>What is the appropriate role of software development in ASCR applied math research, and what are funding models that would support that role? Should we have a software management and sustainability plan similar to the required data management plans? <br></div><div><br></div

SIAM PP14 productivity slides

<p>These slides were presented at SIAM PP14. They include the opening slides for the Minisymposium on Software Productivity, as well as a case study of software engineering applied to a climate-related multi-scale project.</p

Overview: Software Productivity Challenges for Extreme Scale Science

The Department of Energy has initiated research in software productivity for application development and software infrastructure for extreme-scale scientific computing. The eventual goal is to enable “grand challenge” science simulations that can survive and even leverage disruptive changes in extreme-scale computer architectures, and thus enable new frontiers in modeling, simulation, and analysis of complex multiscale and multiphysics phenomena. This session will give an overview of DOE activities, including workshops, participating communities, and pilot projects.<br><br>Presented at SIAM CSE15 minisymposium Scientific Software Productivity at Extreme Scale

Using DOE Math Libraries: Introducing the xSDK

<p><i>Tutorial presented at the 2018 ECP Annual Meeting</i></p><p><i>February 5, 2018</i></p><p><br></p><p><b>Abstract:</b></p><p>Extreme-scale computational science increasingly demands multiscale and multiphysics formulations, which in turn require many high-quality, robust, portable high-performance math libraries. However, without coordination, independent packages often cannot be easily composed. The Extreme-scale Scientific Software Development Kit (xSDK) provides infrastructure for a collection of related and complementary software elements—developed by diverse, independent teams throughout the HPC community—that provide the building blocks, tools, models, processes, and related artifacts for rapid and efficient development of high-quality applications.</p><p> This tutorial describes the development of the xSDK, while at the same time explain the collaboration among math library teams toward two strategic objectives: building community and building sustainability—useful to consider for efforts toward Software Development Kits generally, in other areas of reusable software technologies. Requirements and design focus on 6 key themes: xSDK growth, xSDK distribution, integration and usage of xSDK packages, high-quality products, user support, and regular upgrades. We will discuss xSDK community policies that help to improve code quality and compatibility across independently developed packages, while also determining criteria for member packages’ inclusion in the xSDK. We will describe the release process and demonstrate the use of the xSDK in application codes. Finally, we will provide the opportunity for breakout discussions and hands-on sessions with developers of the individual member packages (hypre, MAGMA, MFEM, PETSc, SUNDIALS, SuperLU, Trilinos), focusing on capabilities and usage of these libraries as needed by ECP applications. </p

Package Management Practices Essential for Interoperability: Lessons Learned and Strategies Developed for FASTMath

<p>While open-source software packages for high-performance computing (HPC) are an essential foundation of many scientific applications, challenges arise in coordinating multiple packages that employ diverse development and release processes. This document identifies a few key issues and essential practices for packages to more easily interoperate within the same application. We discuss approaches under way within the multi-institutional FASTMath project, whose members develop robust, efficient, and scalable numerical research software for functionalities such as mesh management, discretization, and solvers. Our perspective and priorities for overcoming common problems—including inconsistent installation processes, inconsistent or missing configuration information, copying or spoofing dependent sources as a means of simplifying package code, and inconsistent/missing versioning—should be of interest to HPC software developers who aim to make their packages easier to interoperate with others.</p

CSE Complete: R&D for Productivity Improvement

Poster presented at SIAM CSE17 PP108 Minisymposterium: Software Productivity and Sustainability for CSE and Data Science<div><br></div><div><b>Abstract: </b>Computational Science and Engineering (CSE) is about delivering valuable scientific results. CSE activities are meaningful only to the extent they deliver these results. At the same time, certain practices, tools and processes, while not directly about producing CSE results, actually improve our ability to deliver results over a sufficiently long span of time.</div><p> In this poster we discuss efforts in the IDEAS Scientific Software Project that are intended to improve the overall effectiveness of CSE efforts. IDEAS is focused on R&D in new productivity improvements for CSE software, including establishing a scientific software ecosystem (xSDK) and creating, providing and demonstrating the value of content that can be used by CSE software teams to improve their software efforts. </p><p><br></p

IntroToHPCBootcamp.2023.09.pdf

This paper will be published in a special issue of the Journal of Computational Science Education, associated with the Tenth SC Workshop on Best Practices for HPC Training and Education (BPHTE23), held in conjunction with SC23.The U.S. Department of Energy (DOE) is a long-standing leader in research and development of high-performance computing (HPC) in the pursuit of science. However, we face daunting challenges in fostering a robust and diverse HPC workforce. Basic HPC is not typically taught at early stages of students’ academic careers, and the capacity and knowledge of HPC at many institutions are limited. Even so, such topics are prerequisites for advanced training programs, internships, graduate school, and ultimately for careers in HPC. To help address this challenge, as part of the DOE Exascale Computing Project’s Broadening Participation Initiative, we recently launched the Introduction to HPC Training and Workforce Pipeline Program to provide accessible introductory material on HPC, scalable AI, and analytics.We describe the Intro to HPC Bootcamp, an immersive program designed to engage students from underrepresented groups as they learn foundational HPC skills. The program takes a novel approach to HPC training by turning the traditional curriculum upside down. Instead of focusing on technology and its applications, the bootcamp focuses on energy justice to motivate the training of HPC skills through project-based pedagogy and real-life science stories. Additionally, the bootcamp prepares students for internships and future careers at DOE labs. The first bootcamp, hosted by the advanced computing facilities at Argonne, Lawrence Berkeley, and Oak Ridge National Labs and organized by Sustainable Horizons Institute, took place in August 2023.</p

IDEAS: Software Productivity and Sustainability Improvement Plans

Poster presented at SIAM CSE17 PP108 Minisymposterium: Software Productivity and Sustainability for CSE and Data Science<div><br></div><div><div>Scientific software is playing an increasingly important role in both accelerating scientific discovery, and developing a predictive understanding of complex systems needed to inform policy decisions.  To support this role amidst the increasing complexity of the system models, and the disruptive changes in hardware and software, improvements in software development productivity and sustainability are needed. To address this critical need, the Office of Biological and Environmental Research (BER) has have begun exploring the development and use of Software Productivity and Sustainability Improvement Plans (PSIPs) within its Subsurface Biogeochemical Research Program and the Interoperable Development of Extreme-scale Software Applications (IDEAS) project.</div></div