The Challenge of Machine Learning in Space Weather Nowcasting and Forecasting

The numerous recent breakthroughs in machine learning (ML) make imperative to carefully ponder how the scientific community can benefit from a technology that, although not necessarily new, is today living its golden age. This Grand Challenge review paper is focused on the present and future role of machine learning in space weather. The purpose is twofold. On one hand, we will discuss previous works that use ML for space weather forecasting, focusing in particular on the few areas that have seen most activity: the forecasting of geomagnetic indices, of relativistic electrons at geosynchronous orbits, of solar flares occurrence, of coronal mass ejection propagation time, and of solar wind speed. On the other hand, this paper serves as a gentle introduction to the field of machine learning tailored to the space weather community and as a pointer to a number of open challenges that we believe the community should undertake in the next decade. The recurring themes throughout the review are the need to shift our forecasting paradigm to a probabilistic approach focused on the reliable assessment of uncertainties, and the combination of physics-based and machine learning approaches, known as gray-box.Comment: under revie

A New Tool for CME Arrival Time Prediction using Machine Learning Algorithms: CAT-PUMA

Coronal mass ejections (CMEs) are arguably the most violent eruptions in the solar system. CMEs can cause severe disturbances in interplanetary space and can even affect human activities in many aspects, causing damage to infrastructure and loss of revenue. Fast and accurate prediction of CME arrival time is vital to minimize the disruption that CMEs may cause when interacting with geospace. In this paper, we propose a new approach for partial-/full halo CME Arrival Time Prediction Using Machine learning Algorithms (CAT-PUMA). Via detailed analysis of the CME features and solar-wind parameters, we build a prediction engine taking advantage of 182 previously observed geo-effective partial-/full halo CMEs and using algorithms of the Support Vector Machine. We demonstrate that CAT-PUMA is accurate and fast. In particular, predictions made after applying CAT-PUMA to a test set unknown to the engine show a mean absolute prediction error of ∼5.9 hr within the CME arrival time, with 54% of the predictions having absolute errors less than 5.9 hr. Comparisons with other models reveal that CAT-PUMA has a more accurate prediction for 77% of the events investigated that can be carried out very quickly, i.e., within minutes of providing the necessary input parameters of a CME. A practical guide containing the CAT-PUMA engine and the source code of two examples are available in the Appendix, allowing the community to perform their own applications for prediction using CAT-PUMA

MEMPSEP III. A machine learning-oriented multivariate data set for forecasting the Occurrence and Properties of Solar Energetic Particle Events using a Multivariate Ensemble Approach

We introduce a new multivariate data set that utilizes multiple spacecraft collecting in-situ and remote sensing heliospheric measurements shown to be linked to physical processes responsible for generating solar energetic particles (SEPs). Using the Geostationary Operational Environmental Satellites (GOES) flare event list from Solar Cycle (SC) 23 and part of SC 24 (1998-2013), we identify 252 solar events (flares) that produce SEPs and 17,542 events that do not. For each identified event, we acquire the local plasma properties at 1 au, such as energetic proton and electron data, upstream solar wind conditions, and the interplanetary magnetic field vector quantities using various instruments onboard GOES and the Advanced Composition Explorer (ACE) spacecraft. We also collect remote sensing data from instruments onboard the Solar Dynamic Observatory (SDO), Solar and Heliospheric Observatory (SoHO), and the Wind solar radio instrument WAVES. The data set is designed to allow for variations of the inputs and feature sets for machine learning (ML) in heliophysics and has a specific purpose for forecasting the occurrence of SEP events and their subsequent properties. This paper describes a dataset created from multiple publicly available observation sources that is validated, cleaned, and carefully curated for our machine-learning pipeline. The dataset has been used to drive the newly-developed Multivariate Ensemble of Models for Probabilistic Forecast of Solar Energetic Particles (MEMPSEP; see MEMPSEP I (Chatterjee et al., 2023) and MEMPSEP II (Dayeh et al., 2023) for associated papers)

Variation of proton flux profiles with the observer's latitude in simulated gradual SEP events

We study the variation of the shape of the proton intensity-time profiles in simulated gradual Solar Energetic Particle (SEP) events with the relative observer's position in space with respect to the main direction of propagation of an interplanetary (IP) shock. Using a three-dimensional (3D) magnetohydrodynamic (MHD) code to simulate such a shock, we determine the evolution of the downstream-to-upstream ratios of the plasma variables at its front. Under the assumption of an existing relation between the normalized ratio in speed across the shock front and the injection rate of shock-accelerated particles, we model the transport of the particles and we obtain the proton flux profiles to be measured by a grid of 18 virtual observers located at 0.4 and 1.0 AU, with different latitudes and longitudes with respect to the shock nose. The differences among flux profiles are the result of the way each observer establishes a magnetic connection with the shock front, and we find that changes in the observer's latitude may result in intensity changes of up to one order of magnitude at both radial distances considered here. The peak intensity variation with the radial distance for the pair of observers located at the same angular position is also derived. This is the first time that the latitudinal dependence of the peak intensity with the observer's heliocentric radial distance has been quantified within the framework of gradual SEP event simulations.Comment: 20 pages, 6 Figures, 2 Table

Time Series Mining: Shapelet Discovery, Ensembling, and Applications

Time series is a prominent class of temporal data sequences that has the properties of being equally spaced in time, chronologically ordered, and highly dimensional. Time series classification is an important branch of time series mining. Existing time series classifiers operate either on row data in the time domain or into an alternate data space in the shapelets or frequency domains. Combining time series classifiers, is another powerful technique used to improve the classification accuracy. It was demonstrated that different classifiers can be expert in predicting different subset of classes over others. The challenge lies in learning the expertise of different base learners. In addition, the high dimensionality characteristic of time series data makes it difficult to visualize their distribution. In this thesis we developed a new time series ensembling methods in order to improve the predictive performance, investigated the interpretability of classifiers by leveraging the power of deep learning models and adjusting them to provide visual shapelets as a by-product of the classification task. Finally, we show application through problems of solar energetic particle events prediction

Identifying WIMP dark matter from particle and astroparticle data

One of the most promising strategies to identify the nature of dark matter consists in the search for new particles at accelerators and with so-called direct detection experiments. Working within the framework of simplified models, and making use of machine learning tools to speed up statistical inference, we address the question of what we can learn about dark matter from a detection at the LHC and a forthcoming direct detection experiment. We show that with a combination of accelerator and direct detection data, it is possible to identify newly discovered particles as dark matter, by reconstructing their relic density assuming they are weakly interacting massive particles (WIMPs) thermally produced in the early Universe, and demonstrating that it is consistent with the measured dark matter abundance. An inconsistency between these two quantities would instead point either towards additional physics in the dark sector, or towards a non-standard cosmology, with a thermal history substantially different from that of the standard cosmological model.Comment: 24 pages (+21 pages of appendices and references) and 14 figures. v2: Updated to match JCAP version; includes minor clarifications in text and updated reference

CLEAR Space Weather Center of Excellence: All-Clear Solar Energetic Particle Prediction

The CLEAR Space Weather Center of Excellence (CLEAR center) is a five year project that is funded by the NASA Space Weather Center of Excellence program. The CLEAR center will build a comprehensive prediction framework for solar energetic particles (SEPs) focusing on the timely and accurate prediction of low radiation periods (``all clear forecast") and the occurrence and characteristics of elevated periods. This will be accomplished by integrating empirical, first-principles based and machine learning (ML)-trained prediction models. In this paper, the motivation, overview, and tools of the CLEAR center will be discussed

Tracking and data system support for Surveyor mission 5, volume 3

Surveyor 5 tracking and data system activities evaluated from planning to final flight stage