Data-driven discovery of coordinates and governing equations

The discovery of governing equations from scientific data has the potential to transform data-rich fields that lack well-characterized quantitative descriptions. Advances in sparse regression are currently enabling the tractable identification of both the structure and parameters of a nonlinear dynamical system from data. The resulting models have the fewest terms necessary to describe the dynamics, balancing model complexity with descriptive ability, and thus promoting interpretability and generalizability. This provides an algorithmic approach to Occam's razor for model discovery. However, this approach fundamentally relies on an effective coordinate system in which the dynamics have a simple representation. In this work, we design a custom autoencoder to discover a coordinate transformation into a reduced space where the dynamics may be sparsely represented. Thus, we simultaneously learn the governing equations and the associated coordinate system. We demonstrate this approach on several example high-dimensional dynamical systems with low-dimensional behavior. The resulting modeling framework combines the strengths of deep neural networks for flexible representation and sparse identification of nonlinear dynamics (SINDy) for parsimonious models. It is the first method of its kind to place the discovery of coordinates and models on an equal footing.Comment: 25 pages, 6 figures; added acknowledgment

Recurrence-based time series analysis by means of complex network methods

Complex networks are an important paradigm of modern complex systems sciences which allows quantitatively assessing the structural properties of systems composed of different interacting entities. During the last years, intensive efforts have been spent on applying network-based concepts also for the analysis of dynamically relevant higher-order statistical properties of time series. Notably, many corresponding approaches are closely related with the concept of recurrence in phase space. In this paper, we review recent methodological advances in time series analysis based on complex networks, with a special emphasis on methods founded on recurrence plots. The potentials and limitations of the individual methods are discussed and illustrated for paradigmatic examples of dynamical systems as well as for real-world time series. Complex network measures are shown to provide information about structural features of dynamical systems that are complementary to those characterized by other methods of time series analysis and, hence, substantially enrich the knowledge gathered from other existing (linear as well as nonlinear) approaches.Comment: To be published in International Journal of Bifurcation and Chaos (2011

Data based identification and prediction of nonlinear and complex dynamical systems

We thank Dr. R. Yang (formerly at ASU), Dr. R.-Q. Su (formerly at ASU), and Mr. Zhesi Shen for their contributions to a number of original papers on which this Review is partly based. This work was supported by ARO under Grant No. W911NF-14-1-0504. W.-X. Wang was also supported by NSFC under Grants No. 61573064 and No. 61074116, as well as by the Fundamental Research Funds for the Central Universities, Beijing Nova Programme.Peer reviewedPostprin

Climate dynamics and fluid mechanics: Natural variability and related uncertainties

The purpose of this review-and-research paper is twofold: (i) to review the role played in climate dynamics by fluid-dynamical models; and (ii) to contribute to the understanding and reduction of the uncertainties in future climate-change projections. To illustrate the first point, we focus on the large-scale, wind-driven flow of the mid-latitude oceans which contribute in a crucial way to Earth's climate, and to changes therein. We study the low-frequency variability (LFV) of the wind-driven, double-gyre circulation in mid-latitude ocean basins, via the bifurcation sequence that leads from steady states through periodic solutions and on to the chaotic, irregular flows documented in the observations. This sequence involves local, pitchfork and Hopf bifurcations, as well as global, homoclinic ones. The natural climate variability induced by the LFV of the ocean circulation is but one of the causes of uncertainties in climate projections. Another major cause of such uncertainties could reside in the structural instability in the topological sense, of the equations governing climate dynamics, including but not restricted to those of atmospheric and ocean dynamics. We propose a novel approach to understand, and possibly reduce, these uncertainties, based on the concepts and methods of random dynamical systems theory. As a very first step, we study the effect of noise on the topological classes of the Arnol'd family of circle maps, a paradigmatic model of frequency locking as occurring in the nonlinear interactions between the El Nino-Southern Oscillations (ENSO) and the seasonal cycle. It is shown that the maps' fine-grained resonant landscape is smoothed by the noise, thus permitting their coarse-grained classification. This result is consistent with stabilizing effects of stochastic parametrization obtained in modeling of ENSO phenomenon via some general circulation models.Comment: Invited survey paper for Special Issue on The Euler Equations: 250 Years On, in Physica D: Nonlinear phenomen

SOM-VAE: Interpretable Discrete Representation Learning on Time Series

High-dimensional time series are common in many domains. Since human cognition is not optimized to work well in high-dimensional spaces, these areas could benefit from interpretable low-dimensional representations. However, most representation learning algorithms for time series data are difficult to interpret. This is due to non-intuitive mappings from data features to salient properties of the representation and non-smoothness over time. To address this problem, we propose a new representation learning framework building on ideas from interpretable discrete dimensionality reduction and deep generative modeling. This framework allows us to learn discrete representations of time series, which give rise to smooth and interpretable embeddings with superior clustering performance. We introduce a new way to overcome the non-differentiability in discrete representation learning and present a gradient-based version of the traditional self-organizing map algorithm that is more performant than the original. Furthermore, to allow for a probabilistic interpretation of our method, we integrate a Markov model in the representation space. This model uncovers the temporal transition structure, improves clustering performance even further and provides additional explanatory insights as well as a natural representation of uncertainty. We evaluate our model in terms of clustering performance and interpretability on static (Fashion-)MNIST data, a time series of linearly interpolated (Fashion-)MNIST images, a chaotic Lorenz attractor system with two macro states, as well as on a challenging real world medical time series application on the eICU data set. Our learned representations compare favorably with competitor methods and facilitate downstream tasks on the real world data.Comment: Accepted for publication at the Seventh International Conference on Learning Representations (ICLR 2019