Model Evaluation Guidelines for Geomagnetic Index Predictions

Geomagnetic indices are convenient quantities that distill the complicated physics of some region or aspect of near‐Earth space into a single parameter. Most of the best‐known indices are calculated from ground‐based magnetometer data sets, such as Dst, SYM‐H, Kp, AE, AL, and PC. Many models have been created that predict the values of these indices, often using solar wind measurements upstream from Earth as the input variables to the calculation. This document reviews the current state of models that predict geomagnetic indices and the methods used to assess their ability to reproduce the target index time series. These existing methods are synthesized into a baseline collection of metrics for benchmarking a new or updated geomagnetic index prediction model. These methods fall into two categories: (1) fit performance metrics such as root‐mean‐square error and mean absolute error that are applied to a time series comparison of model output and observations and (2) event detection performance metrics such as Heidke Skill Score and probability of detection that are derived from a contingency table that compares model and observation values exceeding (or not) a threshold value. A few examples of codes being used with this set of metrics are presented, and other aspects of metrics assessment best practices, limitations, and uncertainties are discussed, including several caveats to consider when using geomagnetic indices.Plain Language SummaryOne aspect of space weather is a magnetic signature across the surface of the Earth. The creation of this signal involves nonlinear interactions of electromagnetic forces on charged particles and can therefore be difficult to predict. The perturbations that space storms and other activity causes in some observation sets, however, are fairly regular in their pattern. Some of these measurements have been compiled together into a single value, a geomagnetic index. Several such indices exist, providing a global estimate of the activity in different parts of geospace. Models have been developed to predict the time series of these indices, and various statistical methods are used to assess their performance at reproducing the original index. Existing studies of geomagnetic indices, however, use different approaches to quantify the performance of the model. This document defines a standardized set of statistical analyses as a baseline set of comparison tools that are recommended to assess geomagnetic index prediction models. It also discusses best practices, limitations, uncertainties, and caveats to consider when conducting a model assessment.Key PointsWe review existing practices for assessing geomagnetic index prediction models and recommend a “standard set” of metricsAlong with fit performance metrics that use all data‐model pairs in their formulas, event detection performance metrics are recommendedOther aspects of metrics assessment best practices, limitations, uncertainties, and geomagnetic index caveats are also discussedPeer Reviewedhttps://deepblue.lib.umich.edu/bitstream/2027.42/147764/1/swe20790_am.pdfhttps://deepblue.lib.umich.edu/bitstream/2027.42/147764/2/swe20790.pdfhttps://deepblue.lib.umich.edu/bitstream/2027.42/147764/3/swe20790-sup-0001-2018SW002067-SI.pd

System Identification and Data‐Driven Forecasting of AE Index and Prediction Uncertainty Analysis Using a New Cloud‐NARX Model

Severe geomagnetic storms caused by the solar wind disturbances have harmful influences on the operation of modern equipment and systems. The modelling and forecasting of AE index are extremely useful to understand the geomagnetic substorms. This study presents a novel cloud‐NARX model to predict AE index 1 hour ahead. The cloud‐NARX model provides AE index forecasting results, with a correlation coefficient of 0.87 on the data of whole year 2015. The benchmarks on the data of the two interested periods of 17‐21 March 2015 and 22‐26 June 2015 are presented. The presented model uses uncertainty ‘cloud’ model and cloud transformation to quantify the uncertainty throughout the structure detection, parameter estimation and model prediction. The new predicted band can be generated to forecast AE index with confidence interval. The proposed method provides a new way to evaluate the model based on uncertainty analysis, revealing the reliability of model and visualize the bias of model prediction

Linear theory of the mirror instability in non-Maxwellian space plasmas

A unified theory of the mirror instability in space plasmas is developed. In the standard quasi-hydrodynamic approach, the most general mirror-mode dispersion relation is derived and the growth rate of the mirror instability is obtained in terms of arbitrary electron and ion velocity distribution functions. Finite electron temperature effects and arbitrary electron temperature anisotropies are included. The new dispersion relation allows the treatment of more general space plasma equilibria such as the Dory-Guest-Harris (DGH) or Kennel- Ashour-Abdalla (KA) loss cone equilibria, as well as distributions with power law velocity dependence that are modeled by the family of kappa-distributions. Under these conditions, we derive the general kinetic mirror instability growth rate including finite electron temperature effects. As for an example of equilibrium particle distribution, we analyze a large class of kappa to suprathermal loss cone distributions in view of application to a variety of space plasmas like the solar wind, magnetosheath, ring current plasma, and the magnetospheres of other planets

A Radio and plasma wave experiment for the Solar orbiter mission

International audienc