Building Ocean Climate Emulators

The current explosion in machine learning for climate has led to skilled, computationally cheap emulators for the atmosphere. However, the research for ocean emulators remains nascent despite the large potential for accelerating coupled climate simulations and improving ocean forecasts on all timescales. There are several fundamental questions to address that can facilitate the creation of ocean emulators. Here we focus on two questions: 1) the role of the atmosphere in improving the extended skill of the emulator and 2) the representation of variables with distinct timescales (e.g., velocity and temperature) in the design of any emulator. In tackling these questions, we show stable prediction of surface fields for over 8 years, training and testing on data from a high-resolution coupled climate model, using results from four regions of the globe. Our work lays out a set of physically motivated guidelines for building ocean climate emulators

Data-driven super-parameterization using deep learning: Experimentation with multi-scale Lorenz 96 systems and transfer-learning

To make weather/climate modeling computationally affordable, small-scale processes are usually represented in terms of the large-scale, explicitly-resolved processes using physics-based or semi-empirical parameterization schemes. Another approach, computationally more demanding but often more accurate, is super-parameterization (SP), which involves integrating the equations of small-scale processes on high-resolution grids embedded within the low-resolution grids of large-scale processes. Recently, studies have used machine learning (ML) to develop data-driven parameterization (DD-P) schemes. Here, we propose a new approach, data-driven SP (DD-SP), in which the equations of the small-scale processes are integrated data-drivenly using ML methods such as recurrent neural networks. Employing multi-scale Lorenz 96 systems as testbed, we compare the cost and accuracy (in terms of both short-term prediction and long-term statistics) of parameterized low-resolution (LR), SP, DD-P, and DD-SP models. We show that with the same computational cost, DD-SP substantially outperforms LR, and is better than DD-P, particularly when scale separation is lacking. DD-SP is much cheaper than SP, yet its accuracy is the same in reproducing long-term statistics and often comparable in short-term forecasting. We also investigate generalization, finding that when models trained on data from one system are applied to a system with different forcing (e.g., more chaotic), the models often do not generalize, particularly when the short-term prediction accuracy is examined. But we show that transfer-learning, which involves re-training the data-driven model with a small amount of data from the new system, significantly improves generalization. Potential applications of DD-SP and transfer-learning in climate/weather modeling and the expected challenges are discussed

Learning physics-constrained subgrid-scale closures in the small-data regime for stable and accurate LES

We demonstrate how incorporating physics constraints into convolutional neural networks (CNNs) enables learning subgrid-scale (SGS) closures for stable and accurate large-eddy simulations (LES) in the small-data regime (i.e., when the availability of high-quality training data is limited). Using several setups of forced 2D turbulence as the testbeds, we examine the {\it a priori} and {\it a posteriori} performance of three methods for incorporating physics: 1) data augmentation (DA), 2) CNN with group convolutions (GCNN), and 3) loss functions that enforce a global enstrophy-transfer conservation (EnsCon). While the data-driven closures from physics-agnostic CNNs trained in the big-data regime are accurate and stable, and outperform dynamic Smagorinsky (DSMAG) closures, their performance substantially deteriorate when these CNNs are trained with 40x fewer samples (the small-data regime). We show that CNN with DA and GCNN address this issue and each produce accurate and stable data-driven closures in the small-data regime. Despite its simplicity, DA, which adds appropriately rotated samples to the training set, performs as well or in some cases even better than GCNN, which uses a sophisticated equivariance-preserving architecture. EnsCon, which combines structural modeling with aspect of functional modeling, also produces accurate and stable closures in the small-data regime. Overall, GCNN+EnCon, which combines these two physics constraints, shows the best {\it a posteriori} performance in this regime. These results illustrate the power of physics-constrained learning in the small-data regime for accurate and stable LES.Comment: 23 pages, 9 figure

Explaining the physics of transfer learning a data-driven subgrid-scale closure to a different turbulent flow

Transfer learning (TL) is becoming a powerful tool in scientific applications of neural networks (NNs), such as weather/climate prediction and turbulence modeling. TL enables out-of-distribution generalization (e.g., extrapolation in parameters) and effective blending of disparate training sets (e.g., simulations and observations). In TL, selected layers of a NN, already trained for a base system, are re-trained using a small dataset from a target system. For effective TL, we need to know 1) what are the best layers to re-train? and 2) what physics are learned during TL? Here, we present novel analyses and a new framework to address (1)-(2) for a broad range of multi-scale, nonlinear systems. Our approach combines spectral analyses of the systems' data with spectral analyses of convolutional NN's activations and kernels, explaining the inner-workings of TL in terms of the system's nonlinear physics. Using subgrid-scale modeling of several setups of 2D turbulence as test cases, we show that the learned kernels are combinations of low-, band-, and high-pass filters, and that TL learns new filters whose nature is consistent with the spectral differences of base and target systems. We also find the shallowest layers are the best to re-train in these cases, which is against the common wisdom guiding TL in machine learning literature. Our framework identifies the best layer(s) to re-train beforehand, based on physics and NN theory. Together, these analyses explain the physics learned in TL and provide a framework to guide TL for wide-ranging applications in science and engineering, such as climate change modeling.Comment: 21 pages, 6 figure

The impact of short-term incentives on physical activity in a UK behavioural incentives programme

AbstractThis observational study investigates whether the provision of ongoing short-term-incentives for verified physical activity increases and sustains levels of physical activity. We compared UK members at baseline (years 1 and 2) prior to Vitality’s Active Rewards (VAR) intervention commencing (year 3) and follow-up (year 4) for verified, self-reported (encompassing additional physical activities), mortality relative risk and satisfaction with physical activity. Members were categorised into low-active, medium-active and high-active by tertiles of baseline physical activity. Of 11,881 participants, 6477(54.5%) were male, with mean age 39.7(SD 9.8) years. At follow-up, annual active days had increased by 56% overall [60.8(59.7–61.9)–94.8(93.0–96.5)]; 554% in low-active [8.5(8.3–8.7)–47.1(44.7–49.5)]; 205% in medium-active [39.8(39.4–40.2)–81.4(78.7–84.1)] and 17% in high-active members [131.7(129.9–133.5)–153.7(150.7–156.7)] (all p &lt; 0.001). Annual weeks of attaining international physical activity recommendations increased by 19% overall [22.2(42.8%)–26.4(50.8%)] and by 316% for low-active members [4.9(9.5%)–15.5(29.8%)]. Self-reported active minutes/week increased by 45% overall [1423(139.4–145.2)–207.0(201.8–212.3)] and 712% in low-active members [20.1(19.3–21.0)–143.2(134.6–151.9)]. Happiness with exercise levels also increased from 1985(49.4%) to 3414(84.9%) members (all p &lt; 0.001). The relative risk of mortality from a lack of physical activity reduced by 7% for low-active members [from 0.99 to 0.92], 5% for medium-active [0.94–0.89] and 3% for high-active [0.89–0.86](p &lt; 0.001) and by 0.02% for each additional year of age (p = 0.02). This large-scale, real-world, short-term-incentives intervention led to a dramatic increase in physical activity which was sustained for, and still increasing after, two years. If applied at broader level, this approach could considerably aid progress towards WHO targets in its Global Action Plan for Physical Activity.</jats:p

Stable and accurate a posteriori LES of 2D turbulence with convolutional neural networks: Backscatter analysis and generalization via transfer learning

&amp;lt;p&amp;gt;In large eddy simulations (LES), the subgrid-scale effects are modeled by physics-based or data-driven methods. This work develops a convolutional neural network (CNN) to model the subgrid-scale effects of a two-dimensional turbulent flow. The model is able to capture both the inter-scale forward energy transfer and backscatter in both a priori and a posteriori analyses. The LES-CNN model outperforms the physics-based eddy-viscosity models and the previous proposed local artificial neural network (ANN) models in both short-term prediction and long-term statistics. Transfer learning is implemented to generalize the method for turbulence modeling at higher Reynolds numbers. Encoder-decoder network architecture is proposed to generalize the model to a higher computational grid resolution.&amp;lt;/p&amp;gt;</jats:p