Data based identification and prediction of nonlinear and complex dynamical systems

We thank Dr. R. Yang (formerly at ASU), Dr. R.-Q. Su (formerly at ASU), and Mr. Zhesi Shen for their contributions to a number of original papers on which this Review is partly based. This work was supported by ARO under Grant No. W911NF-14-1-0504. W.-X. Wang was also supported by NSFC under Grants No. 61573064 and No. 61074116, as well as by the Fundamental Research Funds for the Central Universities, Beijing Nova Programme.Peer reviewedPostprin

Dynamics of Genetic Circuits with Molecule Partitioning Errors in Cell Division and RNA-RNA Interactions

Many signaling and regulatory molecules within cells exist in very few copies per cell. Any process affecting even limited numbers of these molecules therefore has the potential to affect the dynamics of the biochemical networks of which they are a part. This sensitivity to small copy-number changes is what allows stochasticity in gene expression to introduce a degree of randomness in what cells do. While this randomness can be suppressed, it does not appear to be so in many biological systems, at least not to the maximum degree possible. This suggests that this randomness is not necessarily detrimental to cell populations, as it can produce qualitatively new behaviours in genetic networks which may be utilized by cells.In this thesis, two other mechanisms are investigated which, through their interaction with low copy-number molecules, are able to produce qualitatively different dynamics in genetic networks: the stochastic partitioning of molecules in cell division, and the direct interaction of two low copy-number molecules. For this, a novel simulator of chemical kinetics is first presented, designed to simulate the dynamics of genetic circuits inside growing populations of cells. It is then used to study a genetic switch where one repressive link is formed by direct interaction between RNA molecules. This arrangement was found to decouple the stability of the two noisy attractors of the network and the speeds of the state transitions. In other words, it allows the network to have two equally-stable noisy attractors, but differing state transition speeds.Next, the cell-to-cell diversity in RNA numbers (as quantified by the normalized variance) of a single gene over time in a growing model cell population was studied as a function of the division synchrony. In the model, synchronous cell divisions introduce transient increases in the cell-to-cell diversity in RNA numbers of the population, a prediction which was verified using single-molecule measurements of RNA numbers. Finally, the effects of the stochastic partitioning of regulatory molecules in cell division on the dynamics of two genetic circuits, a switch and a clock, were studied. Of these two circuits, the switch has the most dramatic changes in its dynamics, brought on by the inevitable negative correlation in molecule numbers that sister cells inherit. This negative correlation can allow a cell population to partition the phenotypes of the individual cells with less variance than a binomial distribution.These results advance our understanding of the different behaviours that can be produced in genetic circuits due to these two mechanisms. Since they produce unique behaviours, these mechanisms, and combinations thereof, are expected to be used for specialized purposes in natural genetic circuits. Further, since the downstream effects of these mechanisms may be more predictable than, e.g., modifying promoter sequences, they may also be useful in the design and implementation of future synthetic genetic circuits with specific behaviours.<br/

Environment-sensing Mechanisms of Gene Expression and their Effects on the Dynamics of Genetic Circuits across Cell Generations

In genetic circuits, the constituent genes do not interact only between themselves, they are also affected by regulatory molecules of the host cells that support the circuits’ operation and by the environmental conditions. These factors, along with the intrinsic noise in gene expression, affect the functioning of the circuits. As such, to understand the structure of natural circuits and to engineer functional synthetic circuits, one needs to characterize thoroughly how external factors and perturbations from the environment may affect their behavior.This thesis focused on two cellular mechanisms through which the dynamics of gene expression becomes environment dependent: the intake of gene expression regulatory molecules from the media and the σ factor competition. The first mechanism determines the dynamics by which inducer molecules in the media enter the cell cytoplasm and trigger or repress the expression of the target gene. The second mechanism allows cells to change its gene expression profile to adapt to specific stress conditions.Following the characterization of the effects of these mechanisms on the expression dynamics of individual genes from live, single cell measurements, we then performed in silico assessments on how these effects at the single gene level propagate to the circuit level. Here, the dynamics of genetic circuits was observed in both non-dividing and dividing cell populations, where errors in the partitioning of molecules in cell division occur and introduce significant variance between sister cells.From these studies, with the knowledge on the factors of the host cells and their environment sensing mechanisms, more predictive models of the circuits’ dynamics are expected to emerge. The models would further help in identifying what circuit composition, properties of the host strains and environmental conditions are needed for the circuits to exhibit the desired behavior

Transient and stochastic dynamics in cellular processes

This Thesis studies different cellular and cell population processes driven by non-linear and stochastic dynamics. The problems addressed here gravitate around the concepts of transient dynamics and relaxation from a perturbed to a steady state. In this regard, in all processes studied, stochastic fluctuations, either intrinsically present in or externally applied to these systems play an important and constructive role, by either driving the systems out of equilibrium, interfering with the underlying deterministic laws, or establishing suitable levels of heterogeneity. The first part of the Thesis is committed the analysis of genetically regulated transient cellular processes. Here, we analyse, from a theoretical standpoint, three genetic circuits with pulsed excitable dynamics. We show that all circuits can work in two different excitable regimes, in contrast to what was previously speculated. We also study how, in the presence of molecular noise, these excitable circuits can generate periodic polymodal pulses due to the combination of two noise induced phenomena: stabilisation of an unstable spiral point and coherence resonance. We also studied an excitable genetic mechanism for the regulation of the transcriptional fluctuations observed in some pluripotency factors in Embryonic Stem cells. In the embryo, pluripotency is a transient cellular state and the exit of cells from it seems to be associated with transcriptional fluctuations. In regard to pluripotency control, we also propose a novel mechanism based on the post-translational regulation of a small set of four pluripotency factors. We have validated the theoretical model, based on the formation of binary complexes among these factors, with quantitative experimental data at the single-cell level. The model suggests that the pluripotency state does not depend on the cellular levels of a single factor, but rather on the equilibrium of correlations between the different proteins. In addition, the model is able to anticipate the phenotype of several mutant cell types and suggests that the regulatory function of the protein interactions is to buffer the transcriptional activity of Oc4, a key pluripotency factor. In the second part of the Thesis we studied the behaviour of a computational cell signalling network of the human fibroblast in the presence of external fluctuations and signals. The results obtained here indicate that the network responds in a nontrivial manner to background chatter, both intrinsically and in the presence of external periodic signals. We show that these responses are consequence of the rerouting of the signal to different network information-transmission paths that emerge as noise is modulated. Finally, we also study the cell population dynamics during the formation of microbial biofilms, wrinkled pellicles of bacteria glued by an extracellular matrix that are one of the simplest cases of self-organised multicellular structures. In this Thesis we develop a spatiotemporal model of cellular growth and death that accounts for the experimentally observed patterns of massive bacterial death that precede wrinkle formation in biofilms. These localised patterns focus mechanical forces during biofilm expansion and trigger the formation of the characteristic ridges. In this sense, the proposed model suggests that the death patterns emerge from the mobility changes in bacteria due to the production of extracellular matrix and the spatially inhomogeneous cellular growth. An important prediction of the model is that matrix productions is crucial for the appearance of the patterns and, therefore for winkle formation. We have also experimentally validated validated this prediction with matrix deficient bacterial strains, which show neither death patterns nor wrinkles.En aquesta Tesi s’estudien diferents processos intracel·lulars i de poblacions cel·lulars regits per dinàmica estocàstica i no lineal. El problemes biològics tractats graviten al voltant el concepte de dinàmica transitòria i de relaxació d’un estat dinàmic pertorbat a l’estat estacionari. En aquest sentit, en tots els processos estudiats, les fluctuacions estocàstiques, presents intrínsecament o aplicades de forma externa, hi tenen un paper constructiu, ja sigui empenyent els sistemes fora de l’equilibri, interferint amb les lleis deterministes subjacents, o establint els nivells d’heterogeneïtat necessaris. La primera part de la Tesi es dedica a l’estudi de processos cel·lulars transitoris regulats genèticament. En ella analitzem des d’un punt de vista teòric tres circuits genètics de control de polsos excitables i, contràriament al que s’havia especulat anteriorment, establim que tots ells poden treballar en dos tipus de règim excitable. Analitzem també com, en presència de soroll molecular, aquests circuits excitables poden generar polsos periòdics i multimodals degut a la combinació de dos fenòmens induïts per soroll: l’estabilització estocàstica d’estats inestables i la ressonància de coherència. D’altra banda, estudiem com un mecanisme genètic excitable pot ser el responsable de regular a nivell transcripcional les fluctuacions que s’observen experimentalment en alguns factors de pluripotència en cèl·lules mare embrionàries. En l’embrió, la pluripotència és un estat cel·lular transitori i la sortida de les cèl·lules d’aquest sembla que està associada a fluctuacions transcripcionals. En relació al control de la pluripotència, presentem també un nou mecanisme basat en la regulació post-traduccional d’un petit conjunt de 4 factors de pluripotència. El model teòric proposat, basat en la formació de complexos entre els diferents factors de pluripotència, l’hem validat mitjançant experiments quantitatius en cèl·lules individuals. El model postula que l’estat de pluripotència no depèn dels nivells cel·lulars d’un únic factor, sinó d’un equilibri de correlacions entre diverses proteïnes. A més, prediu el fenotip de cèl·lules mutants i suggereix que la funció reguladora de les interaccions entre les quatre proteïnes és la d’esmorteir l’activitat transcripcional d’Oct4, un dels principals factors de pluripotència. En el segon apartat de la Tesi estudiem el comportament d’una xarxa computacional de senyalització cel·lular de fibroblast humà en presència de senyals externs fluctuants i cíclics. Els resultats obtinguts mostren que la xarxa respon de forma no trivial a les fluctuacions ambientals, fins i tot en presència d’una senyal externa. Diferents nivells de soroll permeten modular la resposta de la xarxa, mitjançant la selecció de rutes alternatives de transmissió de la informació. Finalment, estudiem la dinàmica de poblacions cel·lulars durant la formació de biofilms, pel·lícules arrugades d’aglomerats de bacteris que conformen un dels exemples més simples d’estructures multicel·lulars autoorganitzades. En aquesta Tesi presentem un model espai-temporal de creixement i mort cel·lular motivat per l’evidència experimental sobre l’aparició de patrons de mort massiva de bacteris previs a la formació de les arrugues dels biofilms. Aquests patrons localitzats concentren les forces mecàniques durant l’expansió del biofilm i inicien la formació de les arrugues característiques. En aquest sentit, el model proposat explica com es formen els patrons de mort a partir dels canvis de mobilitat dels bacteris deguts a la producció de matriu extracel·lular combinats amb un creixement espacialment heterogeni. Una important predicció del model és que la producció de matriu és un procés clau per a l’aparició dels patrons i, per tant de les arrugues. En aquest aspecte, els nostres resultats experimentals en bacteris mutants que no produeixen components essencials de la matriu, confirmen les prediccions

The cultural epigenetics of psychopathology: The missing heritability of complex diseases found?

We extend a cognitive paradigm for gene expression based on the asymptotic limit theorems of information theory to the epigenetic epidemiology of mental disorders. In particular, we recognize the fundamental role culture plays in human biology, another heritage mechanism parallel to, and interacting with, the more familiar genetic and epigenetic systems. We do this via a model through which culture acts as another tunable epigenetic catalyst that both directs developmental trajectories, and becomes convoluted with individual ontology, via a mutually-interacting crosstalk mediated by a social interaction that is itself culturally driven. We call for the incorporation of embedding culture as an essential component of the epigenetic regulation of human mental development and its dysfunctions, bringing what is perhaps the central reality of human biology into the center of biological psychiatry. Current US work on gene-environment interactions in psychiatry must be extended to a model of gene-environment-culture interaction to avoid becoming victim of an extreme American individualism that threatens to create paradigms particular to that culture and that are, indeed, peculiar in the context of the world's cultures. The cultural and epigenetic systems of heritage may well provide the 'missing' heritability of complex diseases now under so much intense discussion

Measuring information-transfer delays

In complex networks such as gene networks, traffic systems or brain circuits it is important to understand how long it takes for the different parts of the network to effectively influence one another. In the brain, for example, axonal delays between brain areas can amount to several tens of milliseconds, adding an intrinsic component to any timing-based processing of information. Inferring neural interaction delays is thus needed to interpret the information transfer revealed by any analysis of directed interactions across brain structures. However, a robust estimation of interaction delays from neural activity faces several challenges if modeling assumptions on interaction mechanisms are wrong or cannot be made. Here, we propose a robust estimator for neuronal interaction delays rooted in an information-theoretic framework, which allows a model-free exploration of interactions. In particular, we extend transfer entropy to account for delayed source-target interactions, while crucially retaining the conditioning on the embedded target state at the immediately previous time step. We prove that this particular extension is indeed guaranteed to identify interaction delays between two coupled systems and is the only relevant option in keeping with Wiener’s principle of causality. We demonstrate the performance of our approach in detecting interaction delays on finite data by numerical simulations of stochastic and deterministic processes, as well as on local field potential recordings. We also show the ability of the extended transfer entropy to detect the presence of multiple delays, as well as feedback loops. While evaluated on neuroscience data, we expect the estimator to be useful in other fields dealing with network dynamics

Permutation entropy and its main biomedical and econophysics applications: a review

Entropy is a powerful tool for the analysis of time series, as it allows describing the probability distributions of the possible state of a system, and therefore the information encoded in it. Nevertheless, important information may be codified also in the temporal dynamics, an aspect which is not usually taken into account. The idea of calculating entropy based on permutation patterns (that is, permutations defined by the order relations among values of a time series) has received a lot of attention in the last years, especially for the understanding of complex and chaotic systems. Permutation entropy directly accounts for the temporal information contained in the time series; furthermore, it has the quality of simplicity, robustness and very low computational cost. To celebrate the tenth anniversary of the original work, here we analyze the theoretical foundations of the permutation entropy, as well as the main recent applications to the analysis of economical markets and to the understanding of biomedical systems.Facultad de Ingenierí

Stochastic 0-dimensional Biogeochemical Flux Model: Effect of temperature fluctuations on the dynamics of the biogeochemical properties in a marine ecosystem

We present a new stochastic model, based on a 0-dimensional version of the well known biogeochemical flux model (BFM), which allows to take into account the temperature random fluctuations present in natural systems and therefore to describe more realistically the dynamics of real marine ecosystems. The study presents a detailed analysis of the effects of randomly varying temperature on the lower trophic levels of the food web and ocean biogeochemical processes. More in detail, the temperature is described as a stochastic process driven by an additive self-correlated Gaussian noise. Varying both correlation time and intensity of the noise source, the predominance of different plankton populations is observed, with regimes shifted towards the coexistence or the exclusion of some populations. Finally a Fourier analysis carried out on the time series of the plankton populations shows how the ecosystem responds to the seasonal driving for different values of the noise intensit

The effect of noise on dynamics and the influence of biochemical systems

Understanding a complex system requires integration and collective analysis of data from many levels of organisation. Predictive modelling of biochemical systems is particularly challenging because of the nature of data being plagued by noise operating at each and every level. Inevitably we have to decide whether we can reliably infer the structure and dynamics of biochemical systems from present data. Here we approach this problem from many fronts by analysing the interplay between deterministic and stochastic dynamics in a broad collection of biochemical models. In a classical mathematical model we first illustrate how this interplay can be described in surprisingly simple terms; we furthermore demonstrate the advantages of a statistical point of view also for more complex systems. We then investigate strategies for the integrated analysis of models characterised by different organisational levels, and trace the propagation of noise through such systems. We use this approach to uncover, for the first time, the dynamics of metabolic adaptation of a plant pathogen throughout its life cycle and discuss the ecological implications. Finally, we investigate how reliably we can infer model parameters of biochemical models. We develop a novel sensitivity/inferability analysis framework that is generally applicable to a large fraction of current mathematical models of biochemical systems. By using this framework to quantify the effect of parametric variation on system dynamics, we provide practical guidelines as to when and why certain parameters are easily estimated while others are much harder to infer. We highlight the limitations on parameter inference due to model structure and qualitative dynamical behaviour, and identify candidate elements of control in biochemical pathways most likely of being subjected to regulation