A parimutuel gambling perspective to compare probabilistic seismicity forecasts

Using analogies to gaming, we consider the problem of comparing multiple probabilistic seismicity forecasts. To measure relative model performance, we suggest a parimutuel gambling perspective which addresses shortcomings of other methods such as likelihood ratio, information gain and Molchan diagrams. We describe two variants of the parimutuel approach for a set of forecasts: head-to-head, in which forecasts are compared in pairs, and round table, in which all forecasts are compared simultaneously. For illustration, we compare the 5-yr forecasts of the Regional Earthquake Likelihood Models experiment for M4.95+ seismicity in Californi

Risk and return: evaluating Reverse Tracing of Precursors earthquake predictions

In 2003, the Reverse Tracing of Precursors (RTP) algorithm attracted the attention of seismologists and international news agencies when researchers claimed two successful predictions of large earthquakes. These researchers had begun applying RTP to seismicity in Japan, California, the eastern Mediterranean and Italy; they have since applied it to seismicity in the northern Pacific, Oregon and Nevada. RTP is a pattern recognition algorithm that uses earthquake catalogue data to declare alarms, and these alarms indicate that RTP expects a moderate to large earthquake in the following months. The spatial extent of alarms is highly variable and each alarm typically lasts 9 months, although the algorithm may extend alarms in time and space. We examined the record of alarms and outcomes since the prospective application of RTP began, and in this paper we report on the performance of RTP to date. To analyse these predictions, we used a recently developed approach based on a gambling score, and we used a simple reference model to estimate the prior probability of target earthquakes for each alarm. Formally, we believe that RTP investigators did not rigorously specify the first two ‘successful' predictions in advance of the relevant earthquakes; because this issue is contentious, we consider analyses with and without these alarms. When we included contentious alarms, RTP predictions demonstrate statistically significant skill. Under a stricter interpretation, the predictions are marginally unsuccessfu

Predictability of repeating earthquakes near Parkfield, California

We analyse sequences of repeating microearthquakes that were identified by applying waveform coherency methods to data from the Parkfield High-Resolution Seismic Network. Because by definition all events in a sequence have similar magnitudes and locations, the temporal behaviour of these sequences is naturally isolated, which, coupled with the high occurrence rates of small events, makes these data ideal for studying interevent time distributions. To characterize the temporal predictability of these sequences, we perform retrospective forecast experiments using hundreds of earthquakes. We apply three variants of a simple algorithm that produces sequence-specific, time-varying hazard functions, and we find that the sequences are predictable. We discuss limitations of these data and, more generally, challenges in identifying repeating events, and we outline the potential implications of our results for understanding the occurrence of large earthquake

Retrospective Evaluation of the Five-Year and Ten-Year CSEP-Italy Earthquake Forecasts

On 1 August 2009, the global Collaboratory for the Study of Earthquake Predictability (CSEP) launched a prospective and comparative earthquake predictability experiment in Italy. The goal of the CSEP-Italy experiment is to test earthquake occurrence hypotheses that have been formalized as probabilistic earthquake forecasts over temporal scales that range from days to years. In the first round of forecast submissions, members of the CSEP-Italy Working Group presented eighteen five-year and ten-year earthquake forecasts to the European CSEP Testing Center at ETH Zurich. We considered the twelve time-independent earthquake forecasts among this set and evaluated them with respect to past seismicity data from two Italian earthquake catalogs. In this article, we present the results of tests that measure the consistency of the forecasts with the past observations. Besides being an evaluation of the submitted time-independent forecasts, this exercise provided insight into a number of important issues in predictability experiments with regard to the specification of the forecasts, the performance of the tests, and the trade-off between the robustness of results and experiment duration. We conclude with suggestions for the future design of earthquake predictability experiments.Comment: 43 pages, 8 figures, 4 table

A prospective earthquake forecast experiment in the western Pacific

Since the beginning of 2009, the Collaboratory for the Study of Earthquake Predictability (CSEP) has been conducting an earthquake forecast experiment in the western Pacific. This experiment is an extension of the Kagan—Jackson experiments begun 15 years earlier and is a prototype for future global earthquake predictability experiments. At the beginning of each year, seismicity models make a spatially gridded forecast of the number of Mw≥ 5.8 earthquakes expected in the next year. For the three participating statistical models, we analyse the first two years of this experiment. We use likelihood-based metrics to evaluate the consistency of the forecasts with the observed target earthquakes and we apply measures based on Student's t-test and the Wilcoxon signed-rank test to compare the forecasts. Overall, a simple smoothed seismicity model (TripleS) performs the best, but there are some exceptions that indicate continued experiments are vital to fully understand the stability of these models, the robustness of model selection and, more generally, earthquake predictability in this region. We also estimate uncertainties in our results that are caused by uncertainties in earthquake location and seismic moment. Our uncertainty estimates are relatively small and suggest that the evaluation metrics are relatively robust. Finally, we consider the implications of our results for a global earthquake forecast experimen

Risk and return: Evaluating Reverse Tracing of Precursors earthquake predictions

ISSN:0956-540XISSN:1365-246

Communicating Time-Varying Seismic Risk during an Earthquake Sequence

ISSN:0895-0695ISSN:1938-205

Simple smoothed seismicity earthquake forecasts for Italy

Several earthquake forecast experiments in Italy have been initiated within the European testing center of the global Collaboratory for the Study of Earthquake Predictability. In preparation for these experiments, we developed space-rate-magnitude forecasts based on a simple model that incorporates the spatial clustering of the seismicity. This model, which we call the simple smoothed seismicity model (TripleS), has a minimal number of free parameters and is based on very few assumptions; therefore, it can be considered as a model of «least information» with which others can be compared. The fundamental TripleS parameter controls the spatial extent of the smoothing, and we selected its value based on an optimization procedure that was applied to retrospective forecast experiments. In this report, we present the motivation for developing TripleS, and describe the construction of forecasts for Italian seismicity. We also discuss the research questions that remain to be answered with respect to TripleS, and more generally, the smoothed seismicity approach to earthquake forecasting

A parimutuel gambling perspective to compare probabilistic seismicity forecasts

ISSN:0956-540XISSN:1365-246