Toward a next generation particle precipitation model: Mesoscale prediction through machine learning (a case study and framework for progress)

We advance the modeling capability of electron particle precipitation from the magnetosphere to the ionosphere through a new database and use of machine learning (ML) tools to gain utility from those data. We have compiled, curated, analyzed, and made available a new and more capable database of particle precipitation data that includes 51 satellite years of Defense Meteorological Satellite Program (DMSP) observations temporally aligned with solar wind and geomagnetic activity data. The new total electron energy flux particle precipitation nowcast model, a neural network called PrecipNet, takes advantage of increased expressive power afforded by ML approaches to appropriately utilize diverse information from the solar wind and geomagnetic activity and, importantly, their time histories. With a more capable representation of the organizing parameters and the target electron energy flux observations, PrecipNet achieves a \u3e50% reduction in errors from a current state-of-the-art model oval variation, assessment, tracking, intensity, and online nowcasting (OVATION Prime), better captures the dynamic changes of the auroral flux, and provides evidence that it can capably reconstruct mesoscale phenomena. We create and apply a new framework for space weather model evaluation that culminates previous guidance from across the solar-terrestrial research community. The research approach and results are representative of the “new frontier” of space weather research at the intersection of traditional and data science-driven discovery and provides a foundation for future efforts

Initiation Criteria for the Onset of Geomagnetic Substorms Based on Auroral Observations and Electrojet Current Signatures

In recent years, several substorm onset criteria have been developed, either from auroral observations (many authors) or from auroral electrojet properties such as those described by (Forsyth et al., 2015; Maimaiti et al., 2019; Newell & Gjerloev, 2011; Partamies et al., 2011) The different criteria are being investigated using a low order physics model of the magnetosphere called WINDMI (Spencer et al., 2009) and inferences are being made in line with the WINDMI model. The model variables will be compared with the criteria for substorm onset proposed by examining the SML index. The WINDMI model uses solar wind and IMF measurements from ACE spacecraft as input to a system of 8 non-linear ordinary differential equations. The state variables of the differential equations represent the energy stored in the geomagnetic tail, central plasma sheet, ring current, and field-aligned currents. In this work, the relationship between the output of the WINDMI model and SML (True data) will be established for different events and the timing of the onset for each event, the model parameters, and the model intermediate state space variables are examined and analyzed

Auroral Image Processing Techniques - Machine Learning Classification and Multi-Viewpoint Analysis

Every year, millions of scientific images are acquired in order to study the auroral phenomena. The accumulated data contain a vast amount of untapped information that can be used in auroral science. Yet, auroral research has traditionally been focused on case studies, where one or a few auroral events have been investigated and explained in detail. Consequently, theories have often been developed on the basis of limited data sets, which can possibly be biased in location, spatial resolution or temporal resolution. Advances in technology and data processing now allow for acquisition and analysis of large image data sets. These tools have made it feasible to perform statistical studies based on auroral data from numerous events, varying geophysical conditions and multiple locations in the Arctic and Antarctic. Such studies require reliable auroral image processing techniques to organize, extract and represent the auroral information in a scientifically rigorous manner, preferably with a minimal amount of user interaction. This dissertation focuses on two such branches of image processing techniques: machine learning classification and multi-viewpoint analysis. Machine learning classification: This thesis provides an in-depth description on the implementation of machine learning methods for auroral image classification; from raw images to labeled data. The main conclusion of this work is that convolutional neural networks stand out as a particularly suitable classifier for auroral image data, achieving up to 91 % average class-wise accuracy. A major challenge is that most auroral images have an ambiguous auroral form. These images can not be readily labeled without establishing an auroral morphology, where each class is clearly defined. Multi-viewpoint analysis: Three multi-viewpoint analysis techniques are evaluated and described in this work: triangulation, shell-projection and 3-D reconstruction. These techniques are used for estimating the volume distribution of artificially induced aurora and the height and horizontal distribution of a newly reported auroral feature: Lumikot aurora. The multi-viewpoint analysis techniques are compared and methods for obtaining uncertainty estimates are suggested. Overall, this dissertation evaluates and describes auroral image processing techniques that require little or no user input. The presented methods may therefore facilitate statistical studies such as: probability studies of auroral classes, investigations of the evolution and formation of auroral structures, and studies of the height and distribution of auroral displays. Furthermore, automatic classification and cataloging of large image data sets will support auroral scientists in finding the data of interest, reducing the needed time for manual inspection of auroral images

Empirical Studies Related to Open Questions Regarding Geomagnetic Storms

Earth’s magnetosphere is subject to disturbances, as evidenced by variations of the geomagnetic field in space and on the ground. It is generally understood that most such disturbances are controlled by variations in the solar wind, with interplanetary magnetic field orientations directed southward opposite to Earth’s dipole magnetic axis being most conducive to energy transfers into the magnetosphere, thus resulting in more disturbed intervals. However, the exact functional form for solar wind driving of the magnetosphere has been widely studied, with proposed functional forms varying from the simple half-wave electric rectifier to expressions with a much more complicated dependence upon solar wind parameters. We provide evidence that past empirical results favoring more complicated expressions can statistically emerge from simpler expressions when solar wind parameters are time averaged and that expressions found in past empirical studies can be at least partly explained by the use of time-averaged solar wind parameters having hourly timescales, leading to the pitfall of assigning profound physical meaning to a statistical accident. Suggestions are offered to avoid this pitfall in future investigations. The strongest and most expansive disturbances in the magnetospheric system are magnetic storms. The signature of a geomagnetic storm is the reduction in the strength of Earth’s magnetic field at low latitudes. The conventional explanation for this storm-time geomagnetic depression is a ring shaped current system in the near Earth magnetosphere. In recent years, this conventional view has been called into question by researchers who argue that much of the depression is caused by currents in the more distant region called the magnetotail. Many researchers in the field continue to accept the conventional view. The relative contributions of the current systems are still debated. We construct impulse response functions (IRFs) for storm-time depression to shed light on this controversy. We show that the reduced driving of the geomagnetic index SYM/H (used to measure storm magnitude) during intervals of low density solar wind is due to energy diversion to the ionosphere via burstier events called substorms. As substorm energy is derived from the magnetotail, this reduced driving of storms when substorms are enhanced implies that tail currents are significant to storm-time indices. We also note that the storm-time magnetic depression IRF has a second development several (2-7) hours after the solar wind transfers energy to the magnetosphere, which is more prominent when energy is diverted from the tail to the ionosphere. The IRF of that part of storm-time magnetic depression due (theoretically) to tail currents, as inferred from Auroral Boundary Index (ABI), is shown qualitatively similar to the IRF for SYM/H prior to the second development. We are able to show, by adding functions of AL as an additional IRF driving variable, that this second development is likely due to substorm activity. We interpret this as being consistent with the hypothesis of ionospheric O+ ions enhancing the ring current with a time delay of approximately 2-5 hours. Evaluation of IRFs for sector SMR indices (which resolve storm- time magnetic depression into zones by magnetic local time sectors) reveals a more complicated picture, with evidence for gradual symmetrization of ring current. We model an ideal IRF using our hypothesis and, by comparison to data generated IRFs, show that it presents a plausible model. As Maltsev’s derivation [Maltsev (1996); Maltsev et al. (1996)] of tail current contributions to storm-time magnetic depression depends upon the extent of the equatorward auroral oval, the problem that K-family indices are widely regarded as auroral latitude proxies, rather than storm- time magnetic depression indices, presents itself. We show that the relation of Kp to ABI stems fromthe quasi-logarithmic scaling of Kp, and that storm-time indices, particularly Dcx when corrected for solar wind ram pressure effects, are also a good proxy for ABI when scaled logarithmically. We use ionospheric field aligned current (FAC) maps, provided by APL’s AMPERE project, to generate statistically averaged FAC maps via the machine learning technique of k-means clustering. The region 2 (R2) currents are identified for each cluster and used as a proxy for the equatorward edge of the auroral oval. The magnetic flux in the auroral oval is then used to calculate a predicted tail current contribution to storm-time magnetic depression according to Maltsev’s theory. Re- markable agreement between the predicted and observed median pressure corrected Dcx is found, suggesting that tail contributions are a majority contribution to storm-time magnetic depression

Studies of the Ionosphere-Thermosphere Responses to Multi-Scale Energy Deposition Processes

The Ionosphere-Thermosphere (I-T) system is greatly affected by the magnetospheric energy deposition from above and subjected to forcing from the lower atmosphere simultaneously. A central problem in studying the coupled I-T system is to analyze the multi-scale processes naturally embedded in both ways. Magnetospheric energy input such as auroral precipitation and electric field demonstrates multi-scale structures during magnetic storms, resulting in multi-scale I-T responses when deposited into the I-T system. To better quantify the multi-scale aurora and electric field, we developed a new data assimilation model based on a multi-resolution Gaussian process model to synthesize empirical models and observational data from various sources and provide estimates in regions without observations. The new method mitigates the discrepancy between empirical models and observations by successfully capturing the dynamic evolutions of large-scale and mesoscale auroral and electric field structures. By further incorporating the assimilated aurora and electric fields into Thermosphere Ionosphere Electrodynamics General Circulation Model (TIEGCM) during the 2015 St. Patrick\u27s Day storm, we significantly elevate Joule heating and largely reproduce the global and local I-T responses as observed, including the enhanced electron density and vertical wind. Data assimilation also helps introduce more spatial and temporal variabilities in TIEGCM, which propagate to low-latitude regions through Traveling Atmospheric Disturbance (TAD). In the other direction, to study the atmospheric wave forcing from below and how it impacts the I-T system, we develop a nested-grid extension to TIEGCM to study the Gravity Wave (GW) propagation process and its ionospheric effect during the 2022 Tonga volcano eruption. Such a hybrid-grid design helps to better simulate the variations of a smaller scale than the standard model resolution while reducing computation costs at the same time. With proper seeding at the lower boundary, GW propagation in the thermosphere is successfully reproduced. The resulting Traveling Ionospheric Disturbance (TID) in the ionosphere has a similar speed to observations. The wave spectrum at different altitudes also indicates that the dominant GW has a shorter period and horizontal wavelength at higher altitudes. This dissertation discusses the detailed tool development, including data assimilation and TIEGCM-NG, which enables a better understanding of the influences of multi-scale magnetospheric forcing and lower-atmosphere wave forcing on the I-T system. This work provides a powerful set of tools for a better simulation of space weather

Ionosphere Monitoring with Remote Sensing

This book focuses on the characterization of the physical properties of the Earth’s ionosphere, contributing to unveiling the nature of several processes responsible for a plethora of space weather-related phenomena taking place in a wide range of spatial and temporal scales. This is made possible by the exploitation of a huge amount of high-quality data derived from both remote sensing and in situ facilities such as ionosondes, radars, satellites and Global Navigation Satellite Systems receivers

Review of Environmental Monitoring by Means of Radio Waves in the Polar Regions: From Atmosphere to Geospace

The Antarctic and Arctic regions are Earth's open windows to outer space. They provide unique opportunities for investigating the troposphere–thermosphere–ionosphere–plasmasphere system at high latitudes, which is not as well understood as the mid- and low-latitude regions mainly due to the paucity of experimental observations. In addition, different neutral and ionised atmospheric layers at high latitudes are much more variable compared to lower latitudes, and their variability is due to mechanisms not yet fully understood. Fortunately, in this new millennium the observing infrastructure in Antarctica and the Arctic has been growing, thus providing scientists with new opportunities to advance our knowledge on the polar atmosphere and geospace. This review shows that it is of paramount importance to perform integrated, multi-disciplinary research, making use of long-term multi-instrument observations combined with ad hoc measurement campaigns to improve our capability of investigating atmospheric dynamics in the polar regions from the troposphere up to the plasmasphere, as well as the coupling between atmospheric layers. Starting from the state of the art of understanding the polar atmosphere, our survey outlines the roadmap for enhancing scientific investigation of its physical mechanisms and dynamics through the full exploitation of the available infrastructures for radio-based environmental monitoring

Exploring solar-terrestrial interactions via multiple imaging observers

How does solar wind energy flow through the Earth's magnetosphere, how is it converted and distributed? is the question we want to address. We need to understand how geomagnetic storms and substorms start and grow, not just as a matter of scientific curiosity, but to address a clear and pressing practical problem: space weather, which can influence the performance and reliability of our technological systems, in space and on the ground, and can endanger human life and health. Much knowledge has already been acquired over the past decades, particularly by making use of multiple spacecraft measuring conditions in situ, but the infant stage of space weather forecasting demonstrates that we still have a vast amount of learning to do. A novel global approach is now being taken by a number of space imaging missions which are under development and the first tantalising results of their exploration will be available in the next decade. In this White Paper, submitted to ESA in response to the Voyage 2050 Call, we propose the next step in the quest for a complete understanding of how the Sun controls the Earth's plasma environment: a tomographic imaging approach comprising two spacecraft in highly inclined polar orbits, enabling global imaging of magnetopause and cusps in soft X-rays, of auroral regions in FUV, of plasmasphere and ring current in EUV and ENA (Energetic Neutral Atoms), alongside in situ measurements. Such a mission, encompassing the variety of physical processes determining the conditions of geospace, will be crucial on the way to achieving scientific closure on the question of solar-terrestrial interactions.Peer reviewe

A Study of the Dayside High-Latitude Ionospheric Electrodynamics During Extended Solar Minimum

The high-latitude electric eld fall-o region connects convection in the polar cap to the region where ring currents modify the penetration electric field equatorward of the polar cap boundary. This region is often overlooked because it falls between the limits of low latitude and high-latitude ionospheric models. However, penetrating electric fields cause large changes in ion composition; and therefore, correctly modeling the electric fields and plasma drift in this region aids in correctly specifying the ionosphere. Many ionospheric models use the Kp index as a physical driver, and so the latitude dependence of the plasma drift in the fall-o region was investigated as a function of Kp using Defense Meteorological Satellite Program ion drift data from the 2007{2010 solar minimum. Both the dusk and dawn sectors were analyzed and t to analytical functions describing the fall-o with decreasing latitude. The latitude dependencies were found to dier in the dusk and dawn sectors with a factor of two increase in the expansion of the duskside polar cap radius and auroral region over the dawnside. Additionally, the low-Kp polar cap radius was found to be five degrees smaller than the radius currently used in simple ionospheric models