Change-driven development for scientific software

Scientific software production dates back to the days before the computer science discipline obtained its own name. Over the past 76 years, scientists have been producing software, which means that most of the modern techniques and software engineering methods available these days did not exist while part of this process was taking place. Change-driven development was born as a new approach to maintain and develop scientific software. Founded on the principles of software essence (changeability, complexity, intangibility, and conformity), integrated development tools, and automated source code transformation. This new, agile approach takes change as a working unit devised to drive the entire development process, which is performed in a four-stage cycle. One of the most interesting approaches to apply change-driven development on scientific software is to update, modernize and even parallelize sequential programs that have been written 20 or 30 years ago and are still running in production environments. This process will be thoroughly described and implemented. Two successful case studies will be presented and analyzed in depth.Instituto de Investigación en Informátic

Towards Test Driven Development for Computational Science with pFUnit

Developers working in Computational Science & Engineering (CSE)/High Performance Computing (HPC) must contend with constant change due to advances in computing technology and science. Test Driven Development (TDD) is a methodology that mitigates software development risks due to change at the cost of adding comprehensive and continuous testing to the development process. Testing frameworks tailored for CSE/HPC, like pFUnit, can lower the barriers to such testing, yet CSE software faces unique constraints foreign to the broader software engineering community. Effective testing of numerical software requires a comprehensive suite of oracles, i.e., use cases with known answers, as well as robust estimates for the unavoidable numerical errors associated with implementation with finite-precision arithmetic. At first glance these concerns often seem exceedingly challenging or even insurmountable for real-world scientific applications. However, we argue that this common perception is incorrect and driven by (1) a conflation between model validation and software verification and (2) the general tendency in the scientific community to develop relatively coarse-grained, large procedures that compound numerous algorithmic steps.We believe TDD can be applied routinely to numerical software if developers pursue fine-grained implementations that permit testing, neatly side-stepping concerns about needing nontrivial oracles as well as the accumulation of errors. We present an example of a successful, complex legacy CSE/HPC code whose development process shares some aspects with TDD, which we contrast with current and potential capabilities. A mix of our proposed methodology and framework support should enable everyday use of TDD by CSE-expert developers

Free and Open Source Software underpinning the European Forest Data Centre

Worldwide, governments are growingly focusing on free and open source software (FOSS) as a move toward transparency and the freedom to run, copy, study, change and improve the software. The European Commission (EC) is also supporting the development of FOSS [...]. In addition to the financial savings, FOSS contributes to scientific knowledge freedom in computational science (CS) and is increasingly rewarded in the science-policy interface within the emerging paradigm of open science. Since complex computational science applications may be affected by software uncertainty, FOSS may help to mitigate part of the impact of software errors by CS community- driven open review, correction and evolution of scientific code. The continental scale of EC science-based policy support implies wide networks of scientific collaboration. Thematic information systems also may benefit from this approach within reproducible integrated modelling. This is supported by the EC strategy on FOSS: "for the development of new information systems, where deployment is foreseen by parties outside of the EC infrastructure, [F]OSS will be the preferred choice and in any case used whenever possible". The aim of this contribution is to highlight how a continental scale information system may exploit and integrate FOSS technologies within the transdisciplinary research underpinning such a complex system. A European example is discussed where FOSS innervates both the structure of the information system itself and the inherent transdisciplinary research for modelling the data and information which constitute the system content. [...

Theory-driven development of a medication adherence intervention delivered by eHealth and transplant team in allogeneic stem cell transplantation: the SMILe implementation science project

Medication adherence to immunosuppressants in allogeneic stem cell transplantation (alloSCT) is essential to achieve favorable clinical outcomes (e.g. control of Graft-versus-Host Disease). Over 600 apps supporting medication adherence exist, yet they lack successful implementation and sustainable use likely because of lack of end-user involvement and theoretical underpinnings in their development and insufficient attention to implementation methods to support their use in real-life settings. Medication adherence has three phases: initiation, implementation and persistence. We report the theory-driven development of an intervention module to support medication adherence (implementation and persistence phase) in alloSCT outpatients as a first step for future digitization and implementation in clinical setting within the SMILe project (Development, implementation and testing of an integrated care model in allogeneic SteM cell transplantatIon faciLitated by eHealth).; We applied Michie's Behavior Change Wheel (BCW) and the Capability-Opportunity-Motivation and Behavior (COM-B) model using three suggested stages followed by one stage added by our team regarding preparation for digitization of the intervention: (I) Defining the problem in behavioral terms; (II) Identifying intervention options; (III) Identifying content and implementation options; (IV) SMILe Care Model Prototype Development. Scientific evidence, data from a contextual analysis and patients'/caregivers' and clinical experts' inputs were compiled to work through these steps.; (I) Correct immunosuppressant taking and timing were defined as target behaviors. The intervention's focus was determined within the COM-B dimensions Capability (lack of knowledge, lack of routine), Opportunity (lack of cues, interruptions in daily routine) and Motivation (lack of problem solving, trivialization). (II) Five intervention functions were chosen, i.e. education, training, modelling, persuasion and enablement. (III) Twenty-four behavior change techniques were selected, e.g., goal setting, action planning and problem solving. (IV) Finally, seventeen user stories were developed to guide the SMILeApp's software development process.; Our example on the theory-driven development of an intervention module in alloSCT delivered by eHealth and transplant team using a rigorous 3 + 1-stage approach based on BCW, COM-B and agile software development techniques, can be used as methodological guidance for other eHealth intervention developers. Our approach has the potential to enhance successful implementation and sustained use of eHealth solutions in real-life settings

Free and open source software underpinning the european forest data centre

Excerpt: Worldwide, governments are growingly focusing on free and open source software (FOSS) as a move toward transparency and the freedom to run, copy, study, change and improve the software. The European Commission (EC) is also supporting the development of FOSS [...]. In addition to the financial savings, FOSS contributes to scientific knowledge freedom in computational science (CS) and is increasingly rewarded in the science-policy interface within the emerging paradigm of open science. Since complex computational science applications may be affected by software uncertainty, FOSS may help to mitigate part of the impact of software errors by CS community- driven open review, correction and evolution of scientific code. The continental scale of EC science-based policy support implies wide networks of scientific collaboration. Thematic information systems also may benefit from this approach within reproducible integrated modelling. This is supported by the EC strategy on FOSS: "for the development of new information systems, where deployment is foreseen by parties outside of the EC infrastructure, [F]OSS will be the preferred choice and in any case used whenever possible". The aim of this contribution is to highlight how a continental scale information system may exploit and integrate FOSS technologies within the transdisciplinary research underpinning such a complex system. A European example is discussed where FOSS innervates both the structure of the information system itself and the inherent transdisciplinary research for modelling the data and information which constitute the system content. [...

Free and open source software underpinning the european forest data centre

Excerpt: Worldwide, governments are growingly focusing on free and open source software (FOSS) as a move toward transparency and the freedom to run, copy, study, change and improve the software. The European Commission (EC) is also supporting the development of FOSS [...]. In addition to the financial savings, FOSS contributes to scientific knowledge freedom in computational science (CS) and is increasingly rewarded in the science-policy interface within the emerging paradigm of open science. Since complex computational science applications may be affected by software uncertainty, FOSS may help to mitigate part of the impact of software errors by CS community- driven open review, correction and evolution of scientific code. The continental scale of EC science-based policy support implies wide networks of scientific collaboration. Thematic information systems also may benefit from this approach within reproducible integrated modelling. This is supported by the EC strategy on FOSS: "for the development of new information systems, where deployment is foreseen by parties outside of the EC infrastructure, [F]OSS will be the preferred choice and in any case used whenever possible". The aim of this contribution is to highlight how a continental scale information system may exploit and integrate FOSS technologies within the transdisciplinary research underpinning such a complex system. A European example is discussed where FOSS innervates both the structure of the information system itself and the inherent transdisciplinary research for modelling the data and information which constitute the system content. [...

Free and open source software underpinning the european forest data centre

Excerpt: Worldwide, governments are growingly focusing on free and open source software (FOSS) as a move toward transparency and the freedom to run, copy, study, change and improve the software. The European Commission (EC) is also supporting the development of FOSS [...]. In addition to the financial savings, FOSS contributes to scientific knowledge freedom in computational science (CS) and is increasingly rewarded in the science-policy interface within the emerging paradigm of open science. Since complex computational science applications may be affected by software uncertainty, FOSS may help to mitigate part of the impact of software errors by CS community- driven open review, correction and evolution of scientific code. The continental scale of EC science-based policy support implies wide networks of scientific collaboration. Thematic information systems also may benefit from this approach within reproducible integrated modelling. This is supported by the EC strategy on FOSS: "for the development of new information systems, where deployment is foreseen by parties outside of the EC infrastructure, [F]OSS will be the preferred choice and in any case used whenever possible". The aim of this contribution is to highlight how a continental scale information system may exploit and integrate FOSS technologies within the transdisciplinary research underpinning such a complex system. A European example is discussed where FOSS innervates both the structure of the information system itself and the inherent transdisciplinary research for modelling the data and information which constitute the system content. [...

Research and Education in Computational Science and Engineering

Over the past two decades the field of computational science and engineering (CSE) has penetrated both basic and applied research in academia, industry, and laboratories to advance discovery, optimize systems, support decision-makers, and educate the scientific and engineering workforce. Informed by centuries of theory and experiment, CSE performs computational experiments to answer questions that neither theory nor experiment alone is equipped to answer. CSE provides scientists and engineers of all persuasions with algorithmic inventions and software systems that transcend disciplines and scales. Carried on a wave of digital technology, CSE brings the power of parallelism to bear on troves of data. Mathematics-based advanced computing has become a prevalent means of discovery and innovation in essentially all areas of science, engineering, technology, and society; and the CSE community is at the core of this transformation. However, a combination of disruptive developments---including the architectural complexity of extreme-scale computing, the data revolution that engulfs the planet, and the specialization required to follow the applications to new frontiers---is redefining the scope and reach of the CSE endeavor. This report describes the rapid expansion of CSE and the challenges to sustaining its bold advances. The report also presents strategies and directions for CSE research and education for the next decade.Comment: Major revision, to appear in SIAM Revie