Residual analysis methods for space--time point processes with applications to earthquake forecast models in California

Modern, powerful techniques for the residual analysis of spatial-temporal point process models are reviewed and compared. These methods are applied to California earthquake forecast models used in the Collaboratory for the Study of Earthquake Predictability (CSEP). Assessments of these earthquake forecasting models have previously been performed using simple, low-power means such as the L-test and N-test. We instead propose residual methods based on rescaling, thinning, superposition, weighted K-functions and deviance residuals. Rescaled residuals can be useful for assessing the overall fit of a model, but as with thinning and superposition, rescaling is generally impractical when the conditional intensity $\lambda$ is volatile. While residual thinning and superposition may be useful for identifying spatial locations where a model fits poorly, these methods have limited power when the modeled conditional intensity assumes extremely low or high values somewhere in the observation region, and this is commonly the case for earthquake forecasting models. A recently proposed hybrid method of thinning and superposition, called super-thinning, is a more powerful alternative.Comment: Published in at http://dx.doi.org/10.1214/11-AOAS487 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org

Asperity-based earthquake likelihood models for Italy

The Asperity Likelihood Model (ALM) hypothesizes that small-scale spatial variations in the b-value of the Gutenberg-Richter relationship have a central role in forecasting future seismicity. The physical basis of the ALM is the concept that the local b-value is inversely dependent on the applied shear stress. Thus low b-values (b 1.1), which can be found, for example, in creeping sections of faults, suggest a lower probability of large events. To turn this hypothesis into a forecast model for Italy, we first determined the regional b-value (b = 0.93 ±0.01) and compared it with the locally determined b-values at each node of the forecast grid, based on sampling radii ranging from 6 km to 20 km. We used the local b-values if their Akaike Information Criterion scores were lower than those of the regional b-values. We then explored two modifications to this model: in the ALM.IT, we declustered the input catalog for M ≥2 and smoothed the node-wise rates of the declustered catalog with a Gaussian filter. Completeness values for each node were determined using the probability-based magnitude of completeness method. In the second model, the hybrid ALM (HALM), as a «hybrid» between a grid-based and a zoning model, the Italian territory was divided into eight distinct regions that depended on the main tectonic regimes, and the local b-value variability was thus mapped using the regional b-values for each tectonic zone

Asperity-based earthquake likelihood models f or Italy

T he Asperity Likelihood Model (ALM) was first developed for forecasting earthquakes in California (Wiemer and Schorlemmer, SRL, 2007) and is now being tested for performance in the US testing center of the Collaboratory for the Study of Earthquake Predictability (CSEP). T he model hypothesizes that small-scale spatial variations in b-value of the Gutenberg and Richter relationship play a central role in forecasting future seismicity. T he physical basis of the model is the concept that the local b-value depends inversely on applied shear stress. T hus, low b-values (b<0.7) characterize locked patches of faults—asperities—from which future main shocks are more likely to nucleate, whereas high b-vaues (b>1.1), found for example in creeping sections of faults, suggest a lower probability of nucleating large events. Here, we calibrate this model for the Italian testing region, the first region in the CSEP European testing center. Italian seismicity is lower, more distributed, and less fault-centric than seismicity in California. Comparison of forecasts of the same model in different regions is a key element in making progress in the study of earthquake forecast models. We also explore two modified versions of this model: in the ALM.IT model, we in addition decluster the input catalog and smooth the node-wise rates of the declustered catalog with a gaussian filter. Completeness values for each node are determined using the probability-based magnitude of completeness method (Schorlemmer and Woessner,BSSA, 2008). In the HALM (Hybrid Asperity Likelihood Model), a ‘hybrid’ between a grid-based and a zoning model, the Italian territory is divided into 8 distinct regions depending on the main tectonic regime, and the local b-value variabilily is thus mapped using regional b-values for each tectonic zone

pyCSEP:A Python Package For Earthquake Forecast Developers

For government officials and the public to act on real-time forecasts of earthquakes, the seismological community needs to develop confidence in the underlying scientific hypotheses of the forecast generating models by assessing their predictive skill. For this purpose, the Collaboratory for the Study of Earthquake Predictability (CSEP) provides cyberinfrastructure and computational tools to evaluate earthquake forecasts. Here, we introduce pyCSEP, a Python package to help earthquake forecast developers embed model evaluation into the model development process. The package contains the following modules: (1) earthquake catalog access and processing, (2) data models for earthquake forecasts, (3) statistical tests for evaluating earthquake forecasts, and (4) visualization routines. pyCSEP can evaluate earthquake forecasts expressed as expected rates in space-magnitude bins, and simulation-based forecasts that produce thousands of synthetic seismicity catalogs. Most importantly, pyCSEP contains community-endorsed implementations of statistical tests to evaluate earthquake forecasts, and provides well defined file formats and standards to facilitate model comparisons. The toolkit will facilitate integrating new forecasting models into testing centers, which evaluate forecast models and prediction algorithms in an automated, prospective and independent manner, forming a critical step towards reliable operational earthquake forecasting.This research was supported by the Southern California Earthquake Center (Contribution No. 11030). SCEC is funded by NSF Cooperative Agreement EAR-1600087 & USGS Cooperative Agreement G17AC00047. Maximilian J. Werner and Danijel Schorlemmer received funding from the European Union's Horizon 2020 research and innovation program (Number 821115, RISE: Real-Time Earthquake Risk Reduction for a Resilient Europe)

Evaluation of a Decade-Long Prospective Earthquake Forecasting Experiment in Italy

Earthquake forecasting models represent our current understanding of the physics and statistics that govern earthquake occurrence processes. Providing such forecasts as falsifiable statements can help us assess a model’s hypothesis to be, at the least, a plausible conjecture to explain the observations. Prospective testing (i.e., with future data, once the model and experiment have been fully specified) is fundamental in science because it enables confronting a model with completely out‐of‐sample data and zero degrees of freedom. Testing can also help inform decisions regarding the selection of models, data types, or procedures in practical applications, such as Probabilistic Seismic Hazard Analysis. In 2010, a 10‐year earthquake forecasting experiment began in Italy, where researchers collectively agreed on authoritative data sources, testing rules, and formats to independently evaluate a collection of forecasting models. Here, we test these models with ten years of fully prospective data using a multiscore approach to (1) identify the model features that correlate with data‐consistent or ‐inconsistent forecasts; (2) evaluate the stability of the experiment results over time; and (3) quantify the models’ limitations to generate spatial forecasts consistent with earthquake clustering. As each testing metric analyzes only limited properties of a forecast, the proposed synoptic analysis using multiple scores allows drawing more robust conclusions. Our results show that the best‐performing models use catalogs that span over 100 yr and incorporate fault information, demonstrating and quantifying the value of these data types. Model rankings are stable over time, suggesting that a 10‐year period in Italy can provide sufficient data to discriminate between optimal and suboptimal forecasts. Finally, no model can adequately describe spatial clustering, but those including fault information are less inconsistent with the observations. Prospective testing assesses relevant assumptions and hypotheses of earthquake processes truly out‐of‐sample, thus guiding model development and decision‐making to improve society’s earthquake resilience

Prospective CSEP evaluation of 1-Day, 3-month, and 5-Yr earthquake forecasts for Italy

In 2009, the global Collaboratory for the Study of Earthquake Predictability (CSEP) launched three experiments to forecast the distribution of earthquakes in Italy in the subsequent 5 yrs. CSEP solicited forecasts for seismicity tomorrow, in the next three months, and for the entire 5 yrs. In those 5 yrs, the Istituto Nazionale di Geofisica e Vulcanologia (INGV) recorded 83 target earthquakes with local magnitude 3:95 =M <4:95, and 14 larger shocks. The results show that 1-day forecasts are consistent with the number and magnitudes of the target earthquakes, and one version of the epidemic-type aftershock sequence (ETAS) model is also consistent with the spatial distribution; ensemble forecasts, which we created for the 1-day experiment, are consistent with the number, locations, and magnitudes of the target earthquakes, and they perform as well as the best model; none of the 3-month time-independent models produce consistent forecasts; the best 5-yr models account for the fault distribution and the historical seismicity; and 5-yr models based on instrumental seismicity and b-value spatial variation show poor forecasting performance. © 2018 Seismological Society of America. All rights reserved