Reconstructing complex system dynamics from time series: a method comparison

Modeling complex systems with large numbers of degrees of freedom has become a grand challenge over the past decades. In many situations, only a few variables are actually observed in terms of measured time series, while the majority of variables - which potentially interact with the observed ones - remain hidden. A typical approach is then to focus on the comparably few observed, macroscopic variables, assuming that they determine the key dynamics of the system, while the remaining ones are represented by noise. This naturally leads to an approximate, inverse modeling of such systems in terms of stochastic differential equations (SDEs), with great potential for applications from biology to finance and Earth system dynamics. A well-known approach to retrieve such SDEs from small sets of observed time series is to reconstruct the drift and diffusion terms of a Langevin equation from the data-derived Kramers-Moyal (KM) coefficients. For systems where interactions between the observed and the unobserved variables are crucial, the Mori-Zwanzig formalism (MZ) allows to derive generalized Langevin equations that contain non-Markovian terms representing these interactions. In a similar spirit, the empirical model reduction (EMR) approach has more recently been introduced. In this work we attempt to reconstruct the dynamical equations of motion of both synthetical and real-world processes, by comparing these three approaches in terms of their capability to reconstruct the dynamics and statistics of the underlying systems. Through rigorous investigation of several synthetical and real-world systems, we confirm that the performance of the three methods strongly depends on the intrinsic dynamics of the system at hand. For instance, statistical properties of systems exhibiting weak history-dependence but strong state-dependence of the noise forcing, can be approximated better by the KM method than by the MZ and EMR approaches. In such situations, the KM method is of a considerable advantage since it can directly approximate the state-dependent noise. However, limitations of the KM approximation arise in cases where non-Markovian effects are crucial in the dynamics of the system. In these situations, our numerical results indicate that methods that take into account interactions between observed and unobserved variables in terms of non-Markovian closure terms (i.e., the MZ and EMR approaches), perform comparatively better. © 2020 The Author(s). Published by IOP Publishing Ltd on behalf of the Institute of Physics and Deutsche Physikalische Gesellschaf

Reconstructing Dynamical Systems From Stochastic Differential Equations to Machine Learning

Die Modellierung komplexer Systeme mit einer großen Anzahl von Freiheitsgraden ist in den letzten Jahrzehnten zu einer großen Herausforderung geworden. In der Regel werden nur einige wenige Variablen komplexer Systeme in Form von gemessenen Zeitreihen beobachtet, während die meisten von ihnen - die möglicherweise mit den beobachteten Variablen interagieren - verborgen bleiben. In dieser Arbeit befassen wir uns mit dem Problem der Rekonstruktion und Vorhersage der zugrunde liegenden Dynamik komplexer Systeme mit Hilfe verschiedener datengestützter Ansätze. Im ersten Teil befassen wir uns mit dem umgekehrten Problem der Ableitung einer unbekannten Netzwerkstruktur komplexer Systeme, die Ausbreitungsphänomene widerspiegelt, aus beobachteten Ereignisreihen. Wir untersuchen die paarweise statistische Ähnlichkeit zwischen den Sequenzen von Ereigniszeitpunkten an allen Knotenpunkten durch Ereignissynchronisation (ES) und Ereignis-Koinzidenz-Analyse (ECA), wobei wir uns auf die Idee stützen, dass funktionale Konnektivität als Stellvertreter für strukturelle Konnektivität dienen kann. Im zweiten Teil konzentrieren wir uns auf die Rekonstruktion der zugrunde liegenden Dynamik komplexer Systeme anhand ihrer dominanten makroskopischen Variablen unter Verwendung verschiedener stochastischer Differentialgleichungen (SDEs). In dieser Arbeit untersuchen wir die Leistung von drei verschiedenen SDEs - der Langevin-Gleichung (LE), der verallgemeinerten Langevin-Gleichung (GLE) und dem Ansatz der empirischen Modellreduktion (EMR). Unsere Ergebnisse zeigen, dass die LE bessere Ergebnisse für Systeme mit schwachem Gedächtnis zeigt, während sie die zugrunde liegende Dynamik von Systemen mit Gedächtniseffekten und farbigem Rauschen nicht rekonstruieren kann. In diesen Situationen sind GLE und EMR besser geeignet, da die Wechselwirkungen zwischen beobachteten und unbeobachteten Variablen in Form von Speichereffekten berücksichtigt werden. Im letzten Teil dieser Arbeit entwickeln wir ein Modell, das auf dem Echo State Network (ESN) basiert und mit der PNF-Methode (Past Noise Forecasting) kombiniert wird, um komplexe Systeme in der realen Welt vorherzusagen. Unsere Ergebnisse zeigen, dass das vorgeschlagene Modell die entscheidenden Merkmale der zugrunde liegenden Dynamik der Klimavariabilität erfasst.Modeling complex systems with large numbers of degrees of freedom have become a grand challenge over the past decades. Typically, only a few variables of complex systems are observed in terms of measured time series, while the majority of them – which potentially interact with the observed ones - remain hidden. Throughout this thesis, we tackle the problem of reconstructing and predicting the underlying dynamics of complex systems using different data-driven approaches. In the first part, we address the inverse problem of inferring an unknown network structure of complex systems, reflecting spreading phenomena, from observed event series. We study the pairwise statistical similarity between the sequences of event timings at all nodes through event synchronization (ES) and event coincidence analysis (ECA), relying on the idea that functional connectivity can serve as a proxy for structural connectivity. In the second part, we focus on reconstructing the underlying dynamics of complex systems from their dominant macroscopic variables using different Stochastic Differential Equations (SDEs). We investigate the performance of three different SDEs – the Langevin Equation (LE), Generalized Langevin Equation (GLE), and the Empirical Model Reduction (EMR) approach in this thesis. Our results reveal that LE demonstrates better results for systems with weak memory while it fails to reconstruct underlying dynamics of systems with memory effects and colored-noise forcing. In these situations, the GLE and EMR are more suitable candidates since the interactions between observed and unobserved variables are considered in terms of memory effects. In the last part of this thesis, we develop a model based on the Echo State Network (ESN), combined with the past noise forecasting (PNF) method, to predict real-world complex systems. Our results show that the proposed model captures the crucial features of the underlying dynamics of climate variability

Explicit numerical approximations for stochastic differential equations in finite and infinite horizons : truncation methods, convergence in pth moment, and stability

Solving stochastic differential equations (SDEs) numerically, explicit Euler-Maruyama (EM) schemes are used most frequently under global Lipschitz conditions for both drift and diffusion coefficients. In contrast, without imposing the global Lipschitz conditions, implicit schemes are often used for SDEs but require additional computational effort; along another line, tamed EM schemes and truncated EM schemes have been developed recently. Taking advantages of being explicit and easily implementable, truncated EM schemes are proposed in this paper. Convergence of the numerical algorithms is studied, and pth moment boundedness is obtained. Furthermore, asymptotic properties of the numerical solutions such as the exponential stability in pth moment and stability in distribution are examined. Several examples are given to illustrate our findings

Linear state models for volatility estimation and prediction

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.This thesis concerns the calibration and estimation of linear state models for forecasting stock return volatility. In the first two chapters I present aspects of financial modelling theory and practice that are of particular relevance to the theme of this present work. In addition to this I review the literature concerning these aspects with a particular emphasis on the area of dynamic volatility models. These chapters set the scene and lay the foundations for subsequent empirical work and are a contribution in themselves. The structure of the models employed in the application chapters 4,5 and 6 is the state-space structure, or alternatively the models are known as unobserved components models. In the literature these models have been applied in the estimation of volatility, both for high frequency and low frequency data. As opposed to what has been carried out in the literature I propose the use of these models with Gaussian components. I suggest the implementation of these for high frequency data for short and medium term forecasting. I then demonstrate the calibration of these models and compare medium term forecasting performance for different forecasting methods and model variations as well as that of GARCH and constant volatility models. I then introduce implied volatility measurements leading to two-state models and verify whether this derivative-based information improves forecasting performance. In chapter 6I compare different unobserved components models' specification and forecasting performance. The appendices contain the extensive workings of the parameter estimates' standard error calculations

Colloque International Book of Abstracts Edité par K. Boukhetala et J.F Dupuy

International audienceLe colloque international Modélisation Stochastique et Statistique, dans sa troisième édition, est une rencontre scientifique de haut niveau regroupant chercheurs uni-versitaires et experts praticiens du calcul stochastique et statistique et ses applications dans les domaines so-cio économique, industriel et environnemental. Par les différents thèmes abordés, cette rencontre constituera un cadre idéal pour débattre et discuter des développe-ments récents. Un atelier sur le calcul numérique sto-chastique intensif et ses applications sera animé par des spécialistes en la matière. D'autre part, cette rencontre offrira une occasion de rapprochement entre académi-ciens et professionnels en matière d'échange d'idées et d'expériences dans le domaine de l'utilisation de l'ou-til stochastique et statistique en analyse, modélisation, simulation et prospection pour l'aide à la prise de déci-sion

Information processing in biology

To survive, organisms must respond appropriately to a variety of challenges posed by a dynamic and uncertain environment. The mechanisms underlying such responses can in general be framed as input-output devices which map environment states (inputs) to associated responses (output. In this light, it is appealing to attempt to model these systems using information theory, a well developed mathematical framework to describe input-output systems. Under the information theoretical perspective, an organism’s behavior is fully characterized by the repertoire of its outputs under different environmental conditions. Due to natural selection, it is reasonable to assume this input-output mapping has been fine tuned in such a way as to maximize the organism’s fitness. If that is the case, it should be possible to abstract away the mechanistic implementation details and obtain the general principles that lead to fitness under a certain environment. These can then be used inferentially to both generate hypotheses about the underlying implementation as well as predict novel responses under external perturbations. In this work I use information theory to address the question of how biological systems generate complex outputs using relatively simple mechanisms in a robust manner. In particular, I will examine how communication and distributed processing can lead to emergent phenomena which allow collective systems to respond in a much richer way than a single organism could