Skill metrics for evaluation and comparison of sea ice models

© The Author(s), 2015. This article is distributed under the terms of the Creative Commons Attribution License. The definitive version was published in Journal of Geophysical Research: Oceans 120 (2015): 5910–5931, doi:10.1002/2015JC010989.Five quantitative methodologies (metrics) that may be used to assess the skill of sea ice models against a control field are analyzed. The methodologies are Absolute Deviation, Root-Mean-Square Deviation, Mean Displacement, Hausdorff Distance, and Modified Hausdorff Distance. The methodologies are employed to quantify similarity between spatial distribution of the simulated and control scalar fields providing measures of model performance. To analyze their response to dissimilarities in two-dimensional fields (contours), the metrics undergo sensitivity tests (scale, rotation, translation, and noise). Furthermore, in order to assess their ability to quantify resemblance of three-dimensional fields, the metrics are subjected to sensitivity tests where tested fields have continuous random spatial patterns inside the contours. The Modified Hausdorff Distance approach demonstrates the best response to tested differences, with the other methods limited by weak responses to scale and translation. Both Hausdorff Distance and Modified Hausdorff Distance metrics are robust to noise, as opposed to the other methods. The metrics are then employed in realistic cases that validate sea ice concentration fields from numerical models and sea ice mean outlook against control data and observations. The Modified Hausdorff Distance method again exhibits high skill in quantifying similarity between both two-dimensional (ice contour) and three-dimensional (ice concentration) sea ice fields. The study demonstrates that the Modified Hausdorff Distance is a mathematically tractable and efficient method for model skill assessment and comparison providing effective and objective evaluation of both two-dimensional and three-dimensional sea ice characteristics across data sets.U.S. National Science Foundation (NSF) Grant Number: PLR-0804017, NASA JPL OVWST, Bureau of Ocean Energy Management (BOEM), FSU Grant Number: M12PC00003, NSF Grant Numbers: projects PLR-0804010 , PLR-1313614 , PLR-1203720, BP/The Gulf of Mexico Research Initiative Grant Number: SA12-12, GoMRI-008, DoD High Performance Computing Modernization Progra

Will Arctic sea ice thickness initialization improve seasonal forecast skill?

Arctic sea ice thickness is thought to be an important predictor of Arctic sea ice extent. However, coupled seasonal forecast systems do not generally use sea ice thickness observations in their initialization and are therefore missing a potentially important source of additional skill. To investigate how large this source is, a set of ensemble potential predictability experiments with a global climate model, initialized with and without knowledge of the sea ice thickness initial state, have been run. These experiments show that accurate knowledge of the sea ice thickness field is crucially important for sea ice concentration and extent forecasts up to 8 months ahead, especially in summer. Perturbing sea ice thickness also has a significant impact on the forecast error in Arctic 2 m temperature a few months ahead. These results suggest that advancing capabilities to observe and assimilate sea ice thickness into coupled forecast systems could significantly increase skill

An examination of business occupier relocation decision making : distinguishing small and large firm behaviour

This paper explores how business occupiers decide whether and where to relocate. It captures the experience and behaviour of a range of sizes and types of business occupier and subjects their decision-making processes to detailed scrutiny. A linear three-stage decision model is used to sequence and structure interviews with individuals who have intimate involvement with the relocation of 28 firms and organizations in Tyne and Wear, in the north-east of England. The 'constant comparative' method is used to analyse the interview data, from which emerges 18 key concepts, comprising 51 characteristic components. Using an axial approach, these are organized into 10 cross-cutting themes that represent the main areas of consideration or influence on the thinking of the people involved in determining whether a firm or organization should relocate and, if so, where to. The resulting analysis finds that organizations adopt varying degrees of sophistication when making relocation decisions; small firms are more inclined to make decisions based on constrained information; larger organizations adopt a more complex approach. Regardless of firm size, key individuals exert considerable influence over the decision-making process and its outcome

Seasonal Arctic sea ice forecasting with probabilistic deep learning

Anthropogenic warming has led to an unprecedented year-round reduction in Arctic sea ice extent. This has far-reaching consequences for indigenous and local communities, polar ecosystems, and global climate, motivating the need for accurate seasonal sea ice forecasts. While physics-based dynamical models can successfully forecast sea ice concentration several weeks ahead, they struggle to outperform simple statistical benchmarks at longer lead times. We present a probabilistic, deep learning sea ice forecasting system, IceNet. The system has been trained on climate simulations and observational data to forecast the next 6 months of monthly-averaged sea ice concentration maps. We show that IceNet advances the range of accurate sea ice forecasts, outperforming a state-of-the-art dynamical model in seasonal forecasts of summer sea ice, particularly for extreme sea ice events. This step-change in sea ice forecasting ability brings us closer to conservation tools that mitigate risks associated with rapid sea ice loss

Towards seasonal Arctic shipping route predictions

The continuing decline in Arctic sea-ice will likely lead to increased human activity and opportunities for shipping in the region, suggesting that seasonal predictions of route openings will become ever more important. Here we present results from a set of 'perfect model' experiments to assess the predictability characteristics of the opening of Arctic sea routes. We find skilful predictions of the upcoming summer shipping season can be made from as early as January, although typically forecasts show lower skill before a May 'predictability barrier'. We demonstrate that in forecasts started from January, predictions of route opening date are twice as uncertain as predicting the closing date and that the Arctic shipping season is becoming longer due to climate change, with later closing dates mostly responsible. We find that predictive skill is state dependent with predictions for high or low ice years exhibiting greater skill than medium ice years. Forecasting the fastest open water route through the Arctic is accurate to within 200 km when predicted from July, a six-fold increase in accuracy compared to forecasts initialised from the previous November, which are typically no better than climatology. Finally we find that initialisation of accurate summer sea-ice thickness information is crucial to obtain skilful forecasts, further motivating investment into sea-ice thickness observations, climate models, and assimilation systems

Seasonal to interannual Arctic sea-ice predictability in current GCMs

We establish the first inter-model comparison of seasonal to interannual predictability of present-day Arctic climate by performing coordinated sets of idealized ensemble predictions with four state-of-the-art global climate models. For Arctic sea-ice extent and volume, there is potential predictive skill for lead times of up to three years, and potential prediction errors have similar growth rates and magnitudes across the models. Spatial patterns of potential prediction errors differ substantially between the models, but some features are robust. Sea-ice concentration errors are largest in the marginal ice zone, and in winter they are almost zero away from the ice edge. Sea-ice thickness errors are amplified along the coasts of the Arctic Ocean, an effect that is dominated by sea-ice advection. These results give an upper bound on the ability of current global climate models to predict important aspects of Arctic climate

Identification of Behaviour in Freely Moving Dogs (Canis familiaris) Using Inertial Sensors

Monitoring and describing the physical movements and body postures of animals is one of the most fundamental tasks of ethology. The more precise the observations are the more sophisticated the interpretations can be about the biology of a certain individual or species. Animal-borne data loggers have recently contributed much to the collection of motion-data from individuals, however, the problem of translating these measurements to distinct behavioural categories to create an ethogram is not overcome yet. The objective of the present study was to develop a “behaviour tracker”: a system composed of a multiple sensor data-logger device (with a tri-axial accelerometer and a tri-axial gyroscope) and a supervised learning algorithm as means of automated identification of the behaviour of freely moving dogs. We collected parallel sensor measurements and video recordings of each of our subjects (Belgian Malinois, N=12; Labrador Retrievers, N=12) that were guided through a predetermined series of standard activities. Seven behavioural categories (lay, sit, stand, walk, trot, gallop, canter) were pre-defined and each video recording was tagged accordingly. Evaluation of the measurements was performed by support vector machine (SVM) classification. During the analysis we used different combinations of independent measurements for training and validation (belonging to the same or different individuals or using different training data size) to determine the robustness of the application. We reached an overall accuracy of above 90% perfect identification of all the defined seven categories of behaviour when both training and validation data belonged to the same individual, and over 80% perfect recognition rate using a generalized training data set of multiple subjects. Our results indicate that the present method provides a good model for an easily applicable, fast, automatic behaviour classification system that can be trained with arbitrary motion patterns and potentially be applied to a wide range of species and situations

Predicting September Arctic Sea Ice: A Multi-Model Seasonal Skill Comparison

Abstract This study quantifies the state-of-the-art in the rapidly growing field of seasonal Arctic sea ice prediction. A novel multi-model dataset of retrospective seasonal predictions of September Arctic sea ice is created and analyzed, consisting of community contributions from 17 statistical models and 17 dynamical models. Prediction skill is compared over the period 2001–2020 for predictions of Pan-Arctic sea ice extent (SIE), regional SIE, and local sea ice concentration (SIC) initialized on June 1, July 1, August 1, and September 1. This diverse set of statistical and dynamical models can individually predict linearly detrended Pan-Arctic SIE anomalies with skill, and a multi-model median prediction has correlation coefficients of 0.79, 0.86, 0.92, and 0.99 at these respective initialization times. Regional SIE predictions have similar skill to Pan-Arctic predictions in the Alaskan and Siberian regions, whereas regional skill is lower in the Canadian, Atlantic, and Central Arctic sectors. The skill of dynamical and statistical models is generally comparable for Pan-Arctic SIE, whereas dynamical models outperform their statistical counterparts for regional and local predictions. The prediction systems are found to provide the most value added relative to basic reference forecasts in the extreme SIE years of 1996, 2007, and 2012. SIE prediction errors do not show clear trends over time, suggesting that there has been minimal change in inherent sea ice predictability over the satellite era. Overall, this study demonstrates that there are bright prospects for skillful operational predictions of September sea ice at least three months in advance.</jats:p