The space physics environment data analysis system (SPEDAS)

With the advent of the Heliophysics/Geospace System Observatory (H/GSO), a complement of multi-spacecraft missions and ground-based observatories to study the space environment, data retrieval, analysis, and visualization of space physics data can be daunting. The Space Physics Environment Data Analysis System (SPEDAS), a grass-roots software development platform (www.spedas.org), is now officially supported by NASA Heliophysics as part of its data environment infrastructure. It serves more than a dozen space missions and ground observatories and can integrate the full complement of past and upcoming space physics missions with minimal resources, following clear, simple, and well-proven guidelines. Free, modular and configurable to the needs of individual missions, it works in both command-line (ideal for experienced users) and Graphical User Interface (GUI) mode (reducing the learning curve for first-time users). Both options have “crib-sheets,” user-command sequences in ASCII format that can facilitate record-and-repeat actions, especially for complex operations and plotting. Crib-sheets enhance scientific interactions, as users can move rapidly and accurately from exchanges of technical information on data processing to efficient discussions regarding data interpretation and science. SPEDAS can readily query and ingest all International Solar Terrestrial Physics (ISTP)-compatible products from the Space Physics Data Facility (SPDF), enabling access to a vast collection of historic and current mission data. The planned incorporation of Heliophysics Application Programmer’s Interface (HAPI) standards will facilitate data ingestion from distributed datasets that adhere to these standards. Although SPEDAS is currently Interactive Data Language (IDL)-based (and interfaces to Java-based tools such as Autoplot), efforts are under-way to expand it further to work with python (first as an interface tool and potentially even receiving an under-the-hood replacement). We review the SPEDAS development history, goals, and current implementation. We explain its “modes of use” with examples geared for users and outline its technical implementation and requirements with software developers in mind. We also describe SPEDAS personnel and software management, interfaces with other organizations, resources and support structure available to the community, and future development plans.Published versio

Complexity Heliophysics: A lived and living history of systems and complexity science in Heliophysics

In this piece we study complexity science in the context of Heliophysics, describing it not as a discipline, but as a paradigm. In the context of Heliophysics, complexity science is the study of a star, interplanetary environment, magnetosphere, upper and terrestrial atmospheres, and planetary surface as interacting subsystems. Complexity science studies entities in a system (e.g., electrons in an atom, planets in a solar system, individuals in a society) and their interactions, and is the nature of what emerges from these interactions. It is a paradigm that employs systems approaches and is inherently multi- and cross-scale. Heliophysics processes span at least 15 orders of magnitude in space and another 15 in time, and its reaches go well beyond our own solar system and Earth's space environment to touch planetary, exoplanetary, and astrophysical domains. It is an uncommon domain within which to explore complexity science. After first outlining the dimensions of complexity science, the review proceeds in three epochal parts: 1) A pivotal year in the Complexity Heliophysics paradigm: 1996; 2) The transitional years that established foundations of the paradigm (1996-2010); and 3) The emergent literature largely beyond 2010. This review article excavates the lived and living history of complexity science in Heliophysics. The intention is to provide inspiration, help researchers think more coherently about ideas of complexity science in Heliophysics, and guide future research. It will be instructive to Heliophysics researchers, but also to any reader interested in or hoping to advance the frontier of systems and complexity science

Toward a next generation particle precipitation model: Mesoscale prediction through machine learning (a case study and framework for progress)

We advance the modeling capability of electron particle precipitation from the magnetosphere to the ionosphere through a new database and use of machine learning (ML) tools to gain utility from those data. We have compiled, curated, analyzed, and made available a new and more capable database of particle precipitation data that includes 51 satellite years of Defense Meteorological Satellite Program (DMSP) observations temporally aligned with solar wind and geomagnetic activity data. The new total electron energy flux particle precipitation nowcast model, a neural network called PrecipNet, takes advantage of increased expressive power afforded by ML approaches to appropriately utilize diverse information from the solar wind and geomagnetic activity and, importantly, their time histories. With a more capable representation of the organizing parameters and the target electron energy flux observations, PrecipNet achieves a \u3e50% reduction in errors from a current state-of-the-art model oval variation, assessment, tracking, intensity, and online nowcasting (OVATION Prime), better captures the dynamic changes of the auroral flux, and provides evidence that it can capably reconstruct mesoscale phenomena. We create and apply a new framework for space weather model evaluation that culminates previous guidance from across the solar-terrestrial research community. The research approach and results are representative of the “new frontier” of space weather research at the intersection of traditional and data science-driven discovery and provides a foundation for future efforts

Toward a next generation particle precipitation model: Mesoscale prediction through machine learning (a case study and framework for progress)

We advance the modeling capability of electron particle precipitation from the magnetosphere to the ionosphere through a new database and use of machine learning tools to gain utility from those data. We have compiled, curated, analyzed, and made available a new and more capable database of particle precipitation data that includes 51 satellite years of Defense Meteorological Satellite Program (DMSP) observations temporally aligned with solar wind and geomagnetic activity data. The new total electron energy flux particle precipitation nowcast model, a neural network called PrecipNet, takes advantage of increased expressive power afforded by machine learning approaches to appropriately utilize diverse information from the solar wind and geomagnetic activity and, importantly, their time histories. With a more capable representation of the organizing parameters and the target electron energy flux observations, PrecipNet achieves a $>$50\% reduction in errors from a current state-of-the-art model (OVATION Prime), better captures the dynamic changes of the auroral flux, and provides evidence that it can capably reconstruct mesoscale phenomena. We create and apply a new framework for space weather model evaluation that culminates previous guidance from across the solar-terrestrial research community. The research approach and results are representative of the `new frontier' of space weather research at the intersection of traditional and data science-driven discovery and provides a foundation for future efforts

The Space Physics Environment Data Analysis System (SPEDAS)

With the advent of the Heliophysics/Geospace System Observatory (H/GSO), a complement of multi-spacecraft missions and ground-based observatories to study the space environment, data retrieval, analysis, and visualization of space physics data can be daunting. The Space Physics Environment Data Analysis System (SPEDAS), a grass-roots software development platform (www.spedas.org), is now officially supported by NASA Heliophysics as part of its data environment infrastructure. It serves more than a dozen space missions and ground observatories and can integrate the full complement of past and upcoming space physics missions with minimal resources, following clear, simple, and well-proven guidelines. Free, modular and configurable to the needs of individual missions, it works in both command-line (ideal for experienced users) and Graphical User Interface (GUI) mode (reducing the learning curve for first-time users). Both options have “crib-sheets,” user-command sequences in ASCII format that can facilitate record-and-repeat actions, especially for complex operations and plotting. Crib-sheets enhance scientific interactions, as users can move rapidly and accurately from exchanges of technical information on data processing to efficient discussions regarding data interpretation and science. SPEDAS can readily query and ingest all International Solar Terrestrial Physics (ISTP)-compatible products from the Space Physics Data Facility (SPDF), enabling access to a vast collection of historic and current mission data. The planned incorporation of Heliophysics Application Programmer’s Interface (HAPI) standards will facilitate data ingestion from distributed datasets that adhere to these standards. Although SPEDAS is currently Interactive Data Language (IDL)-based (and interfaces to Java-based tools such as Autoplot), efforts are under-way to expand it further to work with python (first as an interface tool and potentially even receiving an under-the-hood replacement). We review the SPEDAS development history, goals, and current implementation. We explain its “modes of use” with examples geared for users and outline its technical implementation and requirements with software developers in mind. We also describe SPEDAS personnel and software management, interfaces with other organizations, resources and support structure available to the community, and future development plans

GMAG: An open-source python package for ground-based magnetometers

Magnetometers are a key component of heliophysics research providing valuable insight into the dynamics of electromagnetic field regimes and their coupling throughout the solar system. On satellites, magnetometers provide detailed observations of the extension of the solar magnetic field into interplanetary space and of planetary environments. At Earth, magnetometers are deployed on the ground in extensive arrays spanning the polar cap, auroral and sub-auroral zone, mid- and low-latitudes and equatorial electrojet with nearly global coverage in azimuth (longitude or magnetic local time—MLT). These multipoint observations are used to diagnose both ionospheric and magnetospheric processes as well as the coupling between the solar wind and these two regimes at a fraction of the cost of in-situ instruments. Despite their utility in research, ground-based magnetometer data can be difficult to use due to a variety of file formats, multiple points of access for the data, and limited software. In this short article we review the Open-Source Python library GMAG which provides rapid access to ground-based magnetometer data from a number of arrays in a Pandas DataFrame, a common data format used throughout scientific research

Unifying the validation of ambient solar wind models

Progress in space weather research and awareness needs community-wide strategies and procedures to evaluate our modeling assets. Here we present the activities of the Ambient Solar Wind Validation Team embedded in the COSPAR ISWAT initiative. We aim to bridge the gap between model developers and end-users to provide the community with an assessment of the state-of-the-art in solar wind forecasting. To this end, we develop an open online platform for validating solar wind models by comparing their solutions with in situ spacecraft measurements. The online platform will allow the space weather community to test the quality of state-of-the-art solar wind models with unified metrics providing an unbiased assessment of progress over time. In this study, we propose a metadata architecture and recommend community-wide forecasting goals and validation metrics. We conclude with a status update of the online platform and outline future perspectives

Principles Of Heliophysics: a textbook on the universal processes behind planetary habitability

This textbook gives a perspective of heliophysics in a way that emphasizes universal processes from a perspective that draws attention to what provides Earth (and similar (exo-)planets) with a relatively stable setting in which life as we know it can thrive. The book is intended for students in physical sciences in later years of their university training and for beginning graduate students in fields of solar, stellar, (exo-)planetary, and planetary-system sciences.Comment: 419 pages, 119 figures, and 200 "activities" in the form of problems, exercises, explorations, literature readings, and "what if" challenge