6,991 research outputs found
Applying machine learning to improve simulations of a chaotic dynamical system using empirical error correction
Dynamical weather and climate prediction models underpin many studies of the
Earth system and hold the promise of being able to make robust projections of
future climate change based on physical laws. However, simulations from these
models still show many differences compared with observations. Machine learning
has been applied to solve certain prediction problems with great success, and
recently it's been proposed that this could replace the role of
physically-derived dynamical weather and climate models to give better quality
simulations. Here, instead, a framework using machine learning together with
physically-derived models is tested, in which it is learnt how to correct the
errors of the latter from timestep to timestep. This maintains the physical
understanding built into the models, whilst allowing performance improvements,
and also requires much simpler algorithms and less training data. This is
tested in the context of simulating the chaotic Lorenz '96 system, and it is
shown that the approach yields models that are stable and that give both
improved skill in initialised predictions and better long-term climate
statistics. Improvements in long-term statistics are smaller than for single
time-step tendencies, however, indicating that it would be valuable to develop
methods that target improvements on longer time scales. Future strategies for
the development of this approach and possible applications to making progress
on important scientific problems are discussed.Comment: 26p, 7 figures To be published in Journal of Advances in Modeling
Earth System
Data-Driven Forecasting of High-Dimensional Chaotic Systems with Long Short-Term Memory Networks
We introduce a data-driven forecasting method for high-dimensional chaotic
systems using long short-term memory (LSTM) recurrent neural networks. The
proposed LSTM neural networks perform inference of high-dimensional dynamical
systems in their reduced order space and are shown to be an effective set of
nonlinear approximators of their attractor. We demonstrate the forecasting
performance of the LSTM and compare it with Gaussian processes (GPs) in time
series obtained from the Lorenz 96 system, the Kuramoto-Sivashinsky equation
and a prototype climate model. The LSTM networks outperform the GPs in
short-term forecasting accuracy in all applications considered. A hybrid
architecture, extending the LSTM with a mean stochastic model (MSM-LSTM), is
proposed to ensure convergence to the invariant measure. This novel hybrid
method is fully data-driven and extends the forecasting capabilities of LSTM
networks.Comment: 31 page
Data-assisted reduced-order modeling of extreme events in complex dynamical systems
Dynamical systems with high intrinsic dimensionality are often characterized
by extreme events having the form of rare transitions several standard
deviations away from the mean. For such systems, order-reduction methods
through projection of the governing equations have limited applicability due to
the large intrinsic dimensionality of the underlying attractor but also the
complexity of the transient events. An alternative approach is data-driven
techniques that aim to quantify the dynamics of specific modes utilizing
data-streams. Several of these approaches have improved performance by
expanding the state representation using delayed coordinates. However, such
strategies are limited in regions of the phase space where there is a small
amount of data available, as is the case for extreme events. In this work, we
develop a blended framework that integrates an imperfect model, obtained from
projecting equations into a subspace that still contains crucial dynamical
information, with data-streams through a recurrent neural network (RNN)
architecture. In particular, we employ the long-short-term memory (LSTM), to
model portions of the dynamics which cannot be accounted by the equations. The
RNN is trained by analyzing the mismatch between the imperfect model and the
data-streams, projected in the reduced-order space. In this way, the
data-driven model improves the imperfect model in regions where data is
available, while for locations where data is sparse the imperfect model still
provides a baseline for the prediction of the system dynamics. We assess the
developed framework on two challenging prototype systems exhibiting extreme
events and show that the blended approach has improved performance compared
with methods that use either data streams or the imperfect model alone. The
improvement is more significant in regions associated with extreme events,
where data is sparse.Comment: Submitted to PLOS ONE on March 8, 201
Wind Power Forecasting Methods Based on Deep Learning: A Survey
Accurate wind power forecasting in wind farm can effectively reduce the enormous impact on grid operation safety when high permeability intermittent power supply is connected to the power grid. Aiming to provide reference strategies for relevant researchers as well as practical applications, this paper attempts to provide the literature investigation and methods analysis of deep learning, enforcement learning and transfer learning in wind speed and wind power forecasting modeling. Usually, wind speed and wind power forecasting around a wind farm requires the calculation of the next moment of the definite state, which is usually achieved based on the state of the atmosphere that encompasses nearby atmospheric pressure, temperature, roughness, and obstacles. As an effective method of high-dimensional feature extraction, deep neural network can theoretically deal with arbitrary nonlinear transformation through proper structural design, such as adding noise to outputs, evolutionary learning used to optimize hidden layer weights, optimize the objective function so as to save information that can improve the output accuracy while filter out the irrelevant or less affected information for forecasting. The establishment of high-precision wind speed and wind power forecasting models is always a challenge due to the randomness, instantaneity and seasonal characteristics
Data based identification and prediction of nonlinear and complex dynamical systems
We thank Dr. R. Yang (formerly at ASU), Dr. R.-Q. Su (formerly at ASU), and Mr. Zhesi Shen for their contributions to a number of original papers on which this Review is partly based. This work was supported by ARO under Grant No. W911NF-14-1-0504. W.-X. Wang was also supported by NSFC under Grants No. 61573064 and No. 61074116, as well as by the Fundamental Research Funds for the Central Universities, Beijing Nova Programme.Peer reviewedPostprin
Long Term Predictive Modeling on Big Spatio-Temporal Data
In the era of massive data, one of the most promising research fields involves the analysis of large-scale Spatio-temporal databases to discover exciting and previously unknown but potentially useful patterns from data collected over time and space. A modeling process in this domain must take temporal and spatial correlations into account, but with the dimensionality of the time and space measurements increasing, the number of elements potentially contributing to a target sharply grows, making the target\u27s long-term behavior highly complex, chaotic, highly dynamic, and hard to predict. Therefore, two different considerations are taken into account in this work: one is about how to identify the most relevant and meaningful features from the original Spatio-temporal feature space; the other is about how to model complex space-time dynamics with sensitive dependence on initial and boundary conditions.
First, identifying strongly related features and removing the irrelevant or less important features with respect to a target feature from large-scale Spatio-temporal data sets is a critical and challenging issue in many fields, including the evolutionary history of crime hot spots, uncovering weather patterns, predicting floodings, earthquakes, and hurricanes, and determining global warming trends. The optimal sub-feature-set that contains all the valuable information is called the Markov Boundary. Unfortunately, the existing feature selection methods often focus on identifying a single Markov Boundary when real-world data could have many feature subsets that are equally good boundaries. In our work, we design a new multiple-Markov-boundary-based predictive model, Galaxy, to identify the precursors to heavy precipitation event clusters and predict heavy rainfall with a long lead time. We applied Galaxy to an extremely high-dimensional meteorological data set and finally determined 15 Markov boundaries related to heavy rainfall events in the Des Moines River Basin in Iowa. Our model identified the cold surges along the coast of Asia as an essential precursor to the surface weather over the United States, a finding which was later corroborated by climate experts.
Second, chaotic behavior exists in many nonlinear Spatio-temporal systems, such as climate dynamics, weather prediction, and the space-time dynamics of virus spread. A reliable solution for these systems must handle their complex space-time dynamics and sensitive dependence on initial and boundary conditions. Deep neural networks\u27 hierarchical feature learning capabilities in both spatial and temporal domains are helpful for nonlinear Spatio-temporal dynamics modeling. However, sensitive dependence on initial and boundary conditions is still challenging for theoretical research and many critical applications. This study proposes a new recurrent architecture, error trajectory tracing, and accompanying training regime, Horizon Forcing, for prediction in chaotic systems.
These methods have been validated on real-world Spatio-temporal data sets, including one meteorological dataset, three classics, chaotic systems, and four real-world time series prediction tasks with chaotic characteristics. Experiments\u27 results show that each proposed model could outperform the performance of current baseline approaches
- …