Impact of noise on a dynamical system: prediction and uncertainties from a swarm-optimized neural network

In this study, an artificial neural network (ANN) based on particle swarm optimization (PSO) was developed for the time series prediction. The hybrid ANN+PSO algorithm was applied on Mackey--Glass chaotic time series in the short-term $x(t+6)$. The performance prediction was evaluated and compared with another studies available in the literature. Also, we presented properties of the dynamical system via the study of chaotic behaviour obtained from the predicted time series. Next, the hybrid ANN+PSO algorithm was complemented with a Gaussian stochastic procedure (called {\it stochastic} hybrid ANN+PSO) in order to obtain a new estimator of the predictions, which also allowed us to compute uncertainties of predictions for noisy Mackey--Glass chaotic time series. Thus, we studied the impact of noise for several cases with a white noise level ($\sigma_{N}$) from 0.01 to 0.1.Comment: 11 pages, 8 figure

Data based identification and prediction of nonlinear and complex dynamical systems

We thank Dr. R. Yang (formerly at ASU), Dr. R.-Q. Su (formerly at ASU), and Mr. Zhesi Shen for their contributions to a number of original papers on which this Review is partly based. This work was supported by ARO under Grant No. W911NF-14-1-0504. W.-X. Wang was also supported by NSFC under Grants No. 61573064 and No. 61074116, as well as by the Fundamental Research Funds for the Central Universities, Beijing Nova Programme.Peer reviewedPostprin

Bayesian Recurrent Neural Network Models for Forecasting and Quantifying Uncertainty in Spatial-Temporal Data

Recurrent neural networks (RNNs) are nonlinear dynamical models commonly used in the machine learning and dynamical systems literature to represent complex dynamical or sequential relationships between variables. More recently, as deep learning models have become more common, RNNs have been used to forecast increasingly complicated systems. Dynamical spatio-temporal processes represent a class of complex systems that can potentially benefit from these types of models. Although the RNN literature is expansive and highly developed, uncertainty quantification is often ignored. Even when considered, the uncertainty is generally quantified without the use of a rigorous framework, such as a fully Bayesian setting. Here we attempt to quantify uncertainty in a more formal framework while maintaining the forecast accuracy that makes these models appealing, by presenting a Bayesian RNN model for nonlinear spatio-temporal forecasting. Additionally, we make simple modifications to the basic RNN to help accommodate the unique nature of nonlinear spatio-temporal data. The proposed model is applied to a Lorenz simulation and two real-world nonlinear spatio-temporal forecasting applications

Can we identify non-stationary dynamics of trial-to-trial variability?"

Identifying sources of the apparent variability in non-stationary scenarios is a fundamental problem in many biological data analysis settings. For instance, neurophysiological responses to the same task often vary from each repetition of the same experiment (trial) to the next. The origin and functional role of this observed variability is one of the fundamental questions in neuroscience. The nature of such trial-to-trial dynamics however remains largely elusive to current data analysis approaches. A range of strategies have been proposed in modalities such as electro-encephalography but gaining a fundamental insight into latent sources of trial-to-trial variability in neural recordings is still a major challenge. In this paper, we present a proof-of-concept study to the analysis of trial-to-trial variability dynamics founded on non-autonomous dynamical systems. At this initial stage, we evaluate the capacity of a simple statistic based on the behaviour of trajectories in classification settings, the trajectory coherence, in order to identify trial-to-trial dynamics. First, we derive the conditions leading to observable changes in datasets generated by a compact dynamical system (the Duffing equation). This canonical system plays the role of a ubiquitous model of non-stationary supervised classification problems. Second, we estimate the coherence of class-trajectories in empirically reconstructed space of system states. We show how this analysis can discern variations attributable to non-autonomous deterministic processes from stochastic fluctuations. The analyses are benchmarked using simulated and two different real datasets which have been shown to exhibit attractor dynamics. As an illustrative example, we focused on the analysis of the rat's frontal cortex ensemble dynamics during a decision-making task. Results suggest that, in line with recent hypotheses, rather than internal noise, it is the deterministic trend which most likely underlies the observed trial-to-trial variability. Thus, the empirical tool developed within this study potentially allows us to infer the source of variability in in-vivo neural recordings

Dynamical system analysis and forecasting of deformation produced by an earthquake fault

We present a method of constructing low-dimensional nonlinear models describing the main dynamical features of a discrete 2D cellular fault zone, with many degrees of freedom, embedded in a 3D elastic solid. A given fault system is characterized by a set of parameters that describe the dynamics, rheology, property disorder, and fault geometry. Depending on the location in the system parameter space we show that the coarse dynamics of the fault can be confined to an attractor whose dimension is significantly smaller than the space in which the dynamics takes place. Our strategy of system reduction is to search for a few coherent structures that dominate the dynamics and to capture the interaction between these coherent structures. The identification of the basic interacting structures is obtained by applying the Proper Orthogonal Decomposition (POD) to the surface deformations fields that accompany strike-slip faulting accumulated over equal time intervals. We use a feed-forward artificial neural network (ANN) architecture for the identification of the system dynamics projected onto the subspace (model space) spanned by the most energetic coherent structures. The ANN is trained using a standard back-propagation algorithm to predict (map) the values of the observed model state at a future time given the observed model state at the present time. This ANN provides an approximate, large scale, dynamical model for the fault.Comment: 30 pages, 12 figure

Measuring information-transfer delays

In complex networks such as gene networks, traffic systems or brain circuits it is important to understand how long it takes for the different parts of the network to effectively influence one another. In the brain, for example, axonal delays between brain areas can amount to several tens of milliseconds, adding an intrinsic component to any timing-based processing of information. Inferring neural interaction delays is thus needed to interpret the information transfer revealed by any analysis of directed interactions across brain structures. However, a robust estimation of interaction delays from neural activity faces several challenges if modeling assumptions on interaction mechanisms are wrong or cannot be made. Here, we propose a robust estimator for neuronal interaction delays rooted in an information-theoretic framework, which allows a model-free exploration of interactions. In particular, we extend transfer entropy to account for delayed source-target interactions, while crucially retaining the conditioning on the embedded target state at the immediately previous time step. We prove that this particular extension is indeed guaranteed to identify interaction delays between two coupled systems and is the only relevant option in keeping with Wiener’s principle of causality. We demonstrate the performance of our approach in detecting interaction delays on finite data by numerical simulations of stochastic and deterministic processes, as well as on local field potential recordings. We also show the ability of the extended transfer entropy to detect the presence of multiple delays, as well as feedback loops. While evaluated on neuroscience data, we expect the estimator to be useful in other fields dealing with network dynamics

Chaotic price dynamics of agricultural commodities

Traditionally, commodity prices have been analyzed and modeled in the context of linear generating processes. The purpose of this dissertation is to address the adequacy of this work through examination of the critical assumption of independence in the residual process of linearly specified models. As an alternative, a test procedure is developed and utilized to demonstrate the appropriateness of applying generalized conditional heteroscedastic time series models (GARCH) to agricultural commodity prices. In addition, a distinction is made between testing for independence and testing for chaos in commodity prices. The price series of interest derive from the major international agricultural commodity markets, sampled monthly over the period 1960--1994. The results of the present analysis suggest that for bananas, beef, coffee, soybeans, wool and wheat seasonally adjusted growth rates, ARCH-GARCH models account for some of the non-linear dependence in these commodity price series. As an alternative to the ARCH-GARCH models, several neural network models were estimated and in some cases outperformed the ARCH family of models in terms of forecast ability. This further demonstrated the nonlinearity present in these time series. Although, further examination is needed, all prices were found to be non-linearly dependent. It was determined by use of different statistical measures for testing for deterministic chaos that wheat prices may be an example of such behavior. Therefore, their may be something to be gained in terms of short-run forecast accuracy by using semi-parametric modeling approaches as applied to wheat prices