Spectral identification of networks using sparse measurements

We propose a new method to recover global information about a network of interconnected dynamical systems based on observations made at a small number (possibly one) of its nodes. In contrast to classical identification of full graph topology, we focus on the identification of the spectral graph-theoretic properties of the network, a framework that we call spectral network identification. The main theoretical results connect the spectral properties of the network to the spectral properties of the dynamics, which are well-defined in the context of the so-called Koopman operator and can be extracted from data through the Dynamic Mode Decomposition algorithm. These results are obtained for networks of diffusively-coupled units that admit a stable equilibrium state. For large networks, a statistical approach is considered, which focuses on spectral moments of the network and is well-suited to the case of heterogeneous populations. Our framework provides efficient numerical methods to infer global information on the network from sparse local measurements at a few nodes. Numerical simulations show for instance the possibility of detecting the mean number of connections or the addition of a new vertex using measurements made at one single node, that need not be representative of the other nodes' properties.Comment: 3

Reconstructing Dynamical Systems From Stochastic Differential Equations to Machine Learning

Die Modellierung komplexer Systeme mit einer großen Anzahl von Freiheitsgraden ist in den letzten Jahrzehnten zu einer großen Herausforderung geworden. In der Regel werden nur einige wenige Variablen komplexer Systeme in Form von gemessenen Zeitreihen beobachtet, während die meisten von ihnen - die möglicherweise mit den beobachteten Variablen interagieren - verborgen bleiben. In dieser Arbeit befassen wir uns mit dem Problem der Rekonstruktion und Vorhersage der zugrunde liegenden Dynamik komplexer Systeme mit Hilfe verschiedener datengestützter Ansätze. Im ersten Teil befassen wir uns mit dem umgekehrten Problem der Ableitung einer unbekannten Netzwerkstruktur komplexer Systeme, die Ausbreitungsphänomene widerspiegelt, aus beobachteten Ereignisreihen. Wir untersuchen die paarweise statistische Ähnlichkeit zwischen den Sequenzen von Ereigniszeitpunkten an allen Knotenpunkten durch Ereignissynchronisation (ES) und Ereignis-Koinzidenz-Analyse (ECA), wobei wir uns auf die Idee stützen, dass funktionale Konnektivität als Stellvertreter für strukturelle Konnektivität dienen kann. Im zweiten Teil konzentrieren wir uns auf die Rekonstruktion der zugrunde liegenden Dynamik komplexer Systeme anhand ihrer dominanten makroskopischen Variablen unter Verwendung verschiedener stochastischer Differentialgleichungen (SDEs). In dieser Arbeit untersuchen wir die Leistung von drei verschiedenen SDEs - der Langevin-Gleichung (LE), der verallgemeinerten Langevin-Gleichung (GLE) und dem Ansatz der empirischen Modellreduktion (EMR). Unsere Ergebnisse zeigen, dass die LE bessere Ergebnisse für Systeme mit schwachem Gedächtnis zeigt, während sie die zugrunde liegende Dynamik von Systemen mit Gedächtniseffekten und farbigem Rauschen nicht rekonstruieren kann. In diesen Situationen sind GLE und EMR besser geeignet, da die Wechselwirkungen zwischen beobachteten und unbeobachteten Variablen in Form von Speichereffekten berücksichtigt werden. Im letzten Teil dieser Arbeit entwickeln wir ein Modell, das auf dem Echo State Network (ESN) basiert und mit der PNF-Methode (Past Noise Forecasting) kombiniert wird, um komplexe Systeme in der realen Welt vorherzusagen. Unsere Ergebnisse zeigen, dass das vorgeschlagene Modell die entscheidenden Merkmale der zugrunde liegenden Dynamik der Klimavariabilität erfasst.Modeling complex systems with large numbers of degrees of freedom have become a grand challenge over the past decades. Typically, only a few variables of complex systems are observed in terms of measured time series, while the majority of them – which potentially interact with the observed ones - remain hidden. Throughout this thesis, we tackle the problem of reconstructing and predicting the underlying dynamics of complex systems using different data-driven approaches. In the first part, we address the inverse problem of inferring an unknown network structure of complex systems, reflecting spreading phenomena, from observed event series. We study the pairwise statistical similarity between the sequences of event timings at all nodes through event synchronization (ES) and event coincidence analysis (ECA), relying on the idea that functional connectivity can serve as a proxy for structural connectivity. In the second part, we focus on reconstructing the underlying dynamics of complex systems from their dominant macroscopic variables using different Stochastic Differential Equations (SDEs). We investigate the performance of three different SDEs – the Langevin Equation (LE), Generalized Langevin Equation (GLE), and the Empirical Model Reduction (EMR) approach in this thesis. Our results reveal that LE demonstrates better results for systems with weak memory while it fails to reconstruct underlying dynamics of systems with memory effects and colored-noise forcing. In these situations, the GLE and EMR are more suitable candidates since the interactions between observed and unobserved variables are considered in terms of memory effects. In the last part of this thesis, we develop a model based on the Echo State Network (ESN), combined with the past noise forecasting (PNF) method, to predict real-world complex systems. Our results show that the proposed model captures the crucial features of the underlying dynamics of climate variability

The Kuramoto model in complex networks

181 pages, 48 figures. In Press, Accepted Manuscript, Physics Reports 2015 Acknowledgments We are indebted with B. Sonnenschein, E. R. dos Santos, P. Schultz, C. Grabow, M. Ha and C. Choi for insightful and helpful discussions. T.P. acknowledges FAPESP (No. 2012/22160-7 and No. 2015/02486-3) and IRTG 1740. P.J. thanks founding from the China Scholarship Council (CSC). F.A.R. acknowledges CNPq (Grant No. 305940/2010-4) and FAPESP (Grants No. 2011/50761-2 and No. 2013/26416-9) for financial support. J.K. would like to acknowledge IRTG 1740 (DFG and FAPESP).Peer reviewedPreprin

Information Theory in Molecular Evolution: From Models to Structures and Dynamics

This Special Issue collects novel contributions from scientists in the interdisciplinary field of biomolecular evolution. Works listed here use information theoretical concepts as a core but are tightly integrated with the study of molecular processes. Applications include the analysis of phylogenetic signals to elucidate biomolecular structure and function, the study and quantification of structural dynamics and allostery, as well as models of molecular interaction specificity inspired by evolutionary cues

A Network Theoretical Approach to Real-World Problems: Application of the K-Core Algorithm to Various Systems

The study of complex networks is, at its core, an exploration of the mechanisms that control the world in which we live at every scale, from particles no bigger than a grain of sand and amino acids that comprise proteins, to social networks, ecosystems, and even countries. Indeed, we find that, regardless of the physical size of the network\u27s components, we may apply principles of complex network theory, thermodynamics, and statistical mechanics to not only better understand these specific networks, but to formulate theories which may be applied to problems on a more general level. This thesis explores several networks at vastly different scales, ranging from the microscopic (amino acids and frictional packed particles) to the macroscopic (human subjects asked to view a set of videos) to the massive (real ecosystems and the financial ecosystem (Haldane 2011, May 2008) of stocks in the S&P500 stock index). The networks are discussed in chronological order of analysis. We begin with a review of k-core theory, including its applications to certain dynamical systems, as this is an important concept to understand for the next two sections. A discussion of the network structure (specifically, a k-shell decomposition) of both ecological and financial dynamic networks, and the implications of this structure for determining a network\u27s tipping point of collapse, follows. Third, this same k-shell structure is examined for networks of frictional particles approaching a jamming transition, where it is seen that the jamming transition is a k-core transition given by random network theory. Lastly comes a thermodynamical examination of human eye-tracking networks built from data of subjects asked to watch the commercials of the 2014 Super Bowl Game; we determine, using a Maximum Entropy approach, that the collective behavior of this small sample can be used to predict population-wide preferences. The behavior of all of these networks are explained using aspects of network theoretical and statistical mechanics frameworks and can be extended beyond the specific networks analyzed herein

Synchronization in dynamical networks:synchronizability, neural network models and EEG analysis

Complex dynamical networks are ubiquitous in many fields of science from engineering to biology, physics, and sociology. Collective behavior, and in particular synchronization,) is one of the most interesting consequences of interaction of dynamical systems over complex networks. In this thesis we study some aspects of synchronization in dynamical networks. The first section of the study discuses the problem of synchronizability in dynamical networks. Although synchronizability, i.e. the ease by which interacting dynamical systems can synchronize their activity, has been frequently used in research studies, there is no single interpretation for that. Here we give some possible interpretations of synchronizability and investigate to what extent they coincide. We show that in unweighted dynamical networks different interpretations of synchronizability do not lie in the same line, in general. However, in networks with high degrees of synchronization properties, the networks with properly assigned weights for the links or the ones with well-performed link rewirings, the different interpretations of synchronizability go hand in hand. We also show that networks with nonidentical diffusive connections whose weights are assigned using the connection-graph-stability method are better synchronizable compared to networks with identical diffusive couplings. Furthermore, we give an algorithm based on node and edge betweenness centrality measures to enhance the synchronizability of dynamical networks. The algorithm is tested on some artificially constructed dynamical networks as well as on some real-world networks from different disciplines. In the second section we study the synchronization phenomenon in networks of Hindmarsh-Rose neurons. First, the complete synchronization of Hindmarsh-Rose neurons over Newman-Watts networks is investigated. By numerically solving the differential equations of the dynamical network as well as using the master-stability-function method we determine the synchronizing coupling strength for diffusively coupled Hindmarsh-Rose neurons. We also consider clustered networks with dense intra-cluster connections and sparse inter-cluster links. In such networks, the synchronizability is more influenced by the inter-cluster links than intra-cluster connections. We also consider the case where the neurons are coupled through both electrical and chemical connections and obtain the synchronizing coupling strength using numerical calculations. We investigate the behavior of interacting locally synchronized gamma oscillations. We construct a network of minimal number of neurons producing synchronized gamma oscillations. By simulating giant networks of this minimal module we study the dependence of the spike synchrony on some parameters of the network such as the probability and strength of excitatory/inhibitory couplings, parameter mismatch, correlation of thalamic input and transmission time-delay. In the third section of the thesis we study the interdependencies within the time series obtained through electroencephalography (EEG) and give the EEG specific maps for patients suffering from schizophrenia or Alzheimer's disease. Capturing the collective coherent spatiotemporal activity of neuronal populations measured by high density EEG is addressed using measures estimating the synchronization within multivariate time series. Our EEG power analysis on schizophrenic patients, which is based on a new parametrization of the multichannel EEG, shows a relative increase of power in alpha rhythm over the anterior brain regions against its reduction over posterior regions. The correlations of these patterns with the clinical picture of schizophrenia as well as discriminating of the schizophrenia patients from normal control subjects supports the concept of hypofrontality in schizophrenia and renders the alpha rhythm as a sensitive marker of it. By applying a multivariate synchronization estimator, called S-estimator, we reveal the whole-head synchronization topography in schizophrenia. Our finding shows bilaterally increased synchronization over temporal brain regions and decreased synchronization over the postcentral/parietal brain regions. The topography is stable over the course of several months as well as over all conventional EEG frequency bands. Moreover, it correlates with the severity of the illness characterized by positive and negative syndrome scales. We also reveal the EEG features specific to early Alzheimer's disease by applying multivariate phase synchronization method. Our analyses result in a specific map characterized by a decrease in the values of phase synchronization over the fronto-temporal and an increase over temporo-parieto-occipital region predominantly of the left hemisphere. These abnormalities in the synchronization maps correlate with the clinical scores associated to the patients and are able to discriminate patients from normal control subjects with high precision

Discerning nonlinear brain dynamics from EEG:an application to autistic spectrum disorder in young children

A challenging goal in neuroscience is that of identifying specific brain patterns characterising autistic spectrum disorder (ASD). Genetic studies, together with investigations based on magnetic resonance imaging (MRI) and functional MRI, support the idea that distinctive structural features could exist in the ASD brain. In the developing brains of babies and small children, structural differences could provide the basis for different brain connectivity, giving rise to macroscopic effects detectable by e.g. electroencephalography (EEG). A significant body of research has already been conducted in this direction, mainly computing spectral power and coherence. Perhaps due to methodological limitations, together with high variability within and between the cohorts investigated, results have not been in complete agreement, and it is therefore still the case that the diagnosis of ASD is based on behavioural tests and interviews. This thesis describes a step-by-step characterisation and comparison of brain dynamics from ASD and neurotypical subjects, based on the analysis of multi-probe EEG time-series from male children aged 3-5 years. The methods applied are all ones that take explicit account of the intrinsically non-linear, open, and time-variable nature of the system. Time-frequency representations were first computed from the time-series to evaluate the spectral power and to categorise the ranges encompassing different activities as low-frequency (LF, 0.8-3.5 Hz), mid-range-frequency (MF, 3.5-12 Hz) or high-frequency (HF, 12-48 Hz). The spatial pathways for the propagation of neuronal activity were then investigated by calculation of wavelet phase coherence. Finally, deeper insight into brain connectivity was achieved by computation of the dynamical cross-frequency coupling between triplets of spatially distributed phases. In doing so, dynamical Bayesian inference was used to find the coupling parameters between the oscillators in the spatially-distributed network. The sets of parameters extracted by this means allowed evaluation of the strength of particular coupling components of the triplet LF, MF→HF, and enabled reconstruction of the coupling functions. By investigation of the form of the coupling functions, the thesis goes beyond conventional measures like the directionality and strength of an interaction, and reveals subtler features of the underlying mechanism. The measured power distributions highlight differences between ASD and typically developing children in the preferential frequency range for local synchronisation of neuronal activity: the relative power is generally higher at LF and HF, and lower at MF, in the ASD case. The phase coherence maps from ASD subjects also exhibited differences, with lower connectivity at LF and MF in the frontal and fronto-occipital pairs, and higher coherence at high frequencies for central links. There was higher inter-subject variability in a comparison of the forms of coupling functions in the ASD group; and a weaker coupling in their theta-gamma range, which can be linked with the cognitive features of the disorder. In conclusion, the approach developed in this thesis gave promising preliminary results, suggesting that a biomarker for ASD could be defined in terms of the described patterns of functional and effective connectivity computed from EEG measurements

Network Synchronization and Control Based on Inverse Optimality : A Study of Inverter-Based Power Generation

This thesis dwells upon the synthesis of system-theoretical tools to understand and control the behavior of nonlinear networked systems. This work is at the crossroads of three topics: synchronization in coupled high-order oscillators, inverse optimal control and the application of inverter-based power systems. The control and stability of power systems leverages the theoretical results obtained for synchronization in coupled high-order oscillators and inverse optimal control.First, we study the dynamics of coupled high-order nonlinear oscillators. These are characterized by their rotational invariance, meaning that their dynamics remain unchanged following a static shift of their angles. We provide sufficient conditions for local frequency synchronization based on both direct, indirect Lyapunov methods and center manifold theory. Second, we study inverse optimal control problems, embedded in networked settings. In this framework, we depart from a given stabilizing control law, with an associated control Lyapunov function and reverse engineer the cost functional to guarantee the optimality of the controller. In this way, inverse optimal control generates a whole family of optimal controllers corresponding to different cost functions. This provides analytically explicit and numerically feasible solutions in closed-form. This approach circumvents the complexity of solving partial differential equations descending from dynamic programming and Bellman's principle of optimality. We show this to be the case also in the presence of disturbances in the dynamics and the cost. In networks, the controller obtained from inverse optimal control has a topological structure (e.g., it is distributed) and thus feasible for implementation. The tuning is analogous to that of linear quadratic regulators.Third, motivated by the pressing changes witnessed by the electrical grid toward renewable energy generation, we consider power system stability and control as the main application of this thesis. In particular, we apply our theoretical findings to study a network of power electronic inverters. We first propose a controller we term the matching controller, a control strategy that, based on DC voltage measurements, endows the inverters with an oscillatory behavior at a common desired frequency. In closed-loop with the matching control, inverters can be considered as nonlinear oscillators. Our study of the dynamics of nonlinear oscillator network provides feasible physical conditions that ask for damping on DC- and AC-side of each converter, that are sufficient for system-wide frequency synchronization.Furthermore, we showcase the usefulness of inverse optimal control for inverter-based generation at two different settings to synthesize robust angle controllers with respect to common disturbances in the grid and provable stability guarantees. All the controllers proposed in this thesis, provide the electrical grid with important services, namely power support whenever needed, as well as power sharing among all inverters