Numerical analysis of stochastic biochemical reaction networks

Numerical solution of the chemical master equation for stochastic reaction networks typically suffers from the state space explosion problem due to the curse of dimensionality and from stiffness due to multiple time scales. The dimension of the state space equals the number of molecular species involved in the reaction network and the size of the system of differential equations equals the number of states in the corresponding continuous-time Markov chain, which is usually enormously huge and often even infinite. Thus, efficient numerical solution approaches must be able to handle huge, possibly infinite and stiff systems of differential equations efficiently. In this thesis, we present efficient techniques for the numerical analysis of the biochemical reaction networks. We present an approximate numerical integration approach that combines a dynamical state space truncation procedure with efficient numerical integration schemes for systems of ordinary differential equations including adaptive step size selection based on local error estimates. We combine our dynamical state space truncation with the method of conditional moments, and present the implementation details and numerical results. We also incorporate ideas from importance sampling simulations into a non-simulative numerical method that approximates transient rare event probabilities based on a dynamical truncation of the state space. Finally, we present a maximum likelihood method for the estimation of the model parameters given noisy time series measurements of molecular counts. All approaches presented in this thesis are implemented as part of the tool STAR, which allows to model and simulate the biochemical reaction networks. The efficiency and accuracy is demonstrated by numerical examples.Numerische Lösungen der chemischen Master-Gleichung für stochastische Reaktionsnetzwerke leiden typischerweise an dem Zustandsraumexplosionsproblem aufgrund der hohen Dimensionalität und der Steifigkeit durch mehrfache Zeitskalen. Die Dimension des Zustandsraumes entspricht der Anzahl der molekularen Spezies von dem Reaktionsnetzwerk und die Größe des Systems von Differentialgleichungen entspricht der Anzahl der Zustände in der entsprechenden kontinuierlichen Markov-Kette, die in der Regel enorm gross und oft sogar unendlich gross ist. Daher müssen numerische Methoden in der Lage sein, riesige, eventuell unendlich grosse und steife Systeme von Differentialgleichungen effizient lösen zu können. In dieser Arbeit beschreiben wir effiziente Methoden für die numerische Analyse biochemischer Reaktionsnetzwerke. Wir betrachten einen inexakten numerischen Integrationsansatz, bei dem eine dynamische Zustandsraumbeschneidung und ein Verfahren mit einem effizienten numerischen Integrationsschema für Systeme von gewöhnlichen Differentialgleichungen benutzt werden. Wir kombinieren unsere dynamische Zustandsraumbeschneidungsmethode mit der Methode der bedingten Momente und beschreiben die Implementierungdetails und numerischen Ergebnisse. Wir benutzen auch Ideen des importance sampling für eine nicht-simulative numerische Methode, die basierend auf der Zustandsraumbeschneidung die Wahrscheinlichkeiten von seltenen Ereignissen berechnen kann. Schließlich beschreiben wir eine Maximum-Likelihood-Methode für die Schätzung der Modellparameter bei verrauschten Zeitreihenmessungen von molekularen Anzahlen. Alle in dieser Arbeit beschriebenen Ansätze sind in dem Software-Tool STAR implementiert, das erlaubt, biochemische Reaktionsnetzwerke zu modellieren und zu simulieren. Die Effizienz und die Genauigkeit werden durch numerische Beispiele gezeigt

Automatic Moment-Closure Approximation of Spatially Distributed Collective Adaptive Systems

Spatially distributed collective adaptive systems are an important class of systems that pose significant challenges to modeling due to the size and complexity of their state spaces. This problem is acute when the dynamic behavior of the system must be captured, such as to predict system performance. In this article, we present an abstraction technique that automatically derives a moment-closure approximation of the dynamic behavior of a spatially distributed collective adaptive system from a discrete representation of the entities involved. The moment-closure technique is demonstrated to give accurate estimates of dynamic behavior, although the number of ordinary differential equations generated for the second-order joint moments can grow large in some cases. For these cases, we propose a rigorous model reduction technique and demonstrate its use to substantially reduce the computational effort with only limited impact on the accuracy if the reduction threshold is set appropriately. All techniques reported in this article are implemented in a tool that is freely available for download

On the analysis of stochastic timed systems

The formal methods approach to develop reliable and efficient safety- or performance-critical systems is to construct mathematically precise models of such systems on which properties of interest, such as safety guarantees or performance requirements, can be verified automatically. In this thesis, we present techniques that extend the reach of exhaustive and statistical model checking to verify reachability and reward-based properties of compositional behavioural models that support quantitative aspects such as real time and randomised decisions. We present two techniques that allow sound statistical model checking for the nondeterministic-randomised model of Markov decision processes. We investigate the relationship between two different definitions of the model of probabilistic timed automata, as well as potential ways to apply statistical model checking. Stochastic timed automata allow nondeterministic choices as well as nondeterministic and stochastic delays, and we present the first exhaustive model checking algorithm that allows their analysis. All the approaches introduced in this thesis are implemented as part of the Modest Toolset, which supports the construction and verification of models specified in the formal modelling language Modest. We conclude by applying this language and toolset to study novel distributed control strategies for photovoltaic microgenerators

Potential based prediction markets: a machine learning perspective

A prediction market is a special type of market which offers trades for securities associated with future states that are observable at a certain time in the future. Recently, prediction markets have shown the promise of being an abstract framework for designing distributed, scalable and self-incentivized machine learning systems which could then apply to large scale problems. However, existing designs of prediction markets are far from achieving such machine learning goal, due to (1) the limited belief modelling power and also (2) an inadequate understanding of the market dynamics. This work is thus motivated by improving and extending current prediction market design in both aspects. This research is focused on potential based prediction markets, that is, prediction markets that are administered by potential (or cost function) based market makers (PMM). To improve the market’s modelling power, we first propose the partially-observable potential based market maker (PoPMM), which generalizes the standard PMM such that it allows securities to be defined and evaluated on future states that are only partially-observable, while also maintaining the key properties of the standard PMM. Next, we complete and extend the theory of generalized exponential families (GEFs), and use GEFs to free the belief models encoded in the PMM/PoPMM from always being in exponential families. To have a better understanding of the market dynamics and its link to model learning, we discuss the market equilibrium and convergence in two main settings: convergence driven by traders, and convergence driven by the market maker. In the former case, we show that a market-wise objective will emerge from the traders’ personal objectives and will be optimized through traders’ selfish behaviours in trading. We then draw intimate links between the convergence result to popular algorithms in convex optimization and machine learning. In the latter case, we augment the PMM with an extra belief model and a bid-ask spread, and model the market dynamics as an optimal control problem. This convergence result requires no specific models on traders, and is suitable for understanding the markets involving less controllable traders

On the analysis of stochastic timed systems

The formal methods approach to develop reliable and efficient safety- or performance-critical systems is to construct mathematically precise models of such systems on which properties of interest, such as safety guarantees or performance requirements, can be verified automatically. In this thesis, we present techniques that extend the reach of exhaustive and statistical model checking to verify reachability and reward-based properties of compositional behavioural models that support quantitative aspects such as real time and randomised decisions. We present two techniques that allow sound statistical model checking for the nondeterministic-randomised model of Markov decision processes. We investigate the relationship between two different definitions of the model of probabilistic timed automata, as well as potential ways to apply statistical model checking. Stochastic timed automata allow nondeterministic choices as well as nondeterministic and stochastic delays, and we present the first exhaustive model checking algorithm that allows their analysis. All the approaches introduced in this thesis are implemented as part of the Modest Toolset, which supports the construction and verification of models specified in the formal modelling language Modest. We conclude by applying this language and toolset to study novel distributed control strategies for photovoltaic microgenerators.Formale Methoden erlauben die Entwicklung verlässlicher und performanter sicherheits- oder zeitkritischer Systeme, indem auf mathematisch präzisen Modellen relevante Eigenschaften wie Sicherheits- oder Performance-Garantien automatisch verifiziert werden. In dieser Dissertation stellen wir Methoden vor, mit denen die Anwendbarkeit der klassischen und statistischen Modellprüfung (model checking) zur Verifikation von Erreichbarkeits- und Nutzenseigenschaften auf kompositionellen Verhaltensmodellen, die quantitative Aspekte wie zufallsbasierte Entscheidungen und Echtzeitverhalten enthalten, erweitert wird. Wir zeigen zwei Methoden auf, die eine korrekte statistische Modellprüfung von Markov-Entscheidungsprozessen erlauben. Wir untersuchen den Zusammenhang zwischen zwei Definitionen des Modells des probabilistischen Zeitautomaten sowie mögliche Wege, die statistische Modellprüfung auf diese Art Modelle anzuwenden. Stochastische Zeitautomaten erlauben nichtdeterministische Entscheidungen sowie nichtdeterministische und stochastische Wartezeiten; wir stellen den ersten Algorithmus für die klassische Modellprüfung dieser Automaten vor. Alle Techniken, die wir in dieser Dissertation behandeln, sind als Teil des Modest Toolsets, welches die Erstellung und Verifikation von Modellen mittels der formalen Modellierungssprache Modest erlaubt, implementiert. Wir verwenden diese Sprache und Tools, um neuartige verteilte Steuerungsalgorithmen für Photovoltaikanlagen zu untersuchen

On the analysis of stochastic timed systems

The formal methods approach to develop reliable and efficient safety- or performance-critical systems is to construct mathematically precise models of such systems on which properties of interest, such as safety guarantees or performance requirements, can be verified automatically. In this thesis, we present techniques that extend the reach of exhaustive and statistical model checking to verify reachability and reward-based properties of compositional behavioural models that support quantitative aspects such as real time and randomised decisions. We present two techniques that allow sound statistical model checking for the nondeterministic-randomised model of Markov decision processes. We investigate the relationship between two different definitions of the model of probabilistic timed automata, as well as potential ways to apply statistical model checking. Stochastic timed automata allow nondeterministic choices as well as nondeterministic and stochastic delays, and we present the first exhaustive model checking algorithm that allows their analysis. All the approaches introduced in this thesis are implemented as part of the Modest Toolset, which supports the construction and verification of models specified in the formal modelling language Modest. We conclude by applying this language and toolset to study novel distributed control strategies for photovoltaic microgenerators.Formale Methoden erlauben die Entwicklung verlässlicher und performanter sicherheits- oder zeitkritischer Systeme, indem auf mathematisch präzisen Modellen relevante Eigenschaften wie Sicherheits- oder Performance-Garantien automatisch verifiziert werden. In dieser Dissertation stellen wir Methoden vor, mit denen die Anwendbarkeit der klassischen und statistischen Modellprüfung (model checking) zur Verifikation von Erreichbarkeits- und Nutzenseigenschaften auf kompositionellen Verhaltensmodellen, die quantitative Aspekte wie zufallsbasierte Entscheidungen und Echtzeitverhalten enthalten, erweitert wird. Wir zeigen zwei Methoden auf, die eine korrekte statistische Modellprüfung von Markov-Entscheidungsprozessen erlauben. Wir untersuchen den Zusammenhang zwischen zwei Definitionen des Modells des probabilistischen Zeitautomaten sowie mögliche Wege, die statistische Modellprüfung auf diese Art Modelle anzuwenden. Stochastische Zeitautomaten erlauben nichtdeterministische Entscheidungen sowie nichtdeterministische und stochastische Wartezeiten; wir stellen den ersten Algorithmus für die klassische Modellprüfung dieser Automaten vor. Alle Techniken, die wir in dieser Dissertation behandeln, sind als Teil des Modest Toolsets, welches die Erstellung und Verifikation von Modellen mittels der formalen Modellierungssprache Modest erlaubt, implementiert. Wir verwenden diese Sprache und Tools, um neuartige verteilte Steuerungsalgorithmen für Photovoltaikanlagen zu untersuchen