This thesis examines Recursive Markov Chains (RMCs), their natural extensions and
connection to other models. RMCs can model in a natural way probabilistic procedural
programs and other systems that involve recursion and probability. An RMC
is a set of ordinary finite state Markov Chains that are allowed to call each other recursively
and it describes a potentially infinite, but countable, state ordinary Markov
Chain. RMCs generalize in a precise sense several well studied probabilistic models
in other domains such as natural language processing (Stochastic Context-Free Grammars),
population dynamics (Multi-Type Branching Processes) and in queueing theory
(Quasi-Birth-Death processes (QBDs)). In addition, RMCs can be extended to a
controlled version called Recursive Markov Decision Processes (RMDPs) and also a
game version referred to as Recursive (Simple) Stochastic Games (RSSGs). For analyzing
RMCs, RMDPs, RSSGs we devised highly optimized numerical algorithms and
implemented them in a tool called PReMo (Probabilistic Recursive Models analyzer).
PReMo allows computation of the termination probability and expected termination
time of RMCs and QBDs, and a restricted subset of RMDPs and RSSGs. The input
models are described by the user in specifically designed simple input languages. Furthermore,
in order to analyze the worst and best expected running time of probabilistic
recursive programs we study models of RMDPs and RSSGs with positive rewards
assigned to each of their transitions and provide new complexity upper and lower
bounds of their analysis. We also establish some new connections between our models
and models studied in queueing theory. Specifically, we show that (discrete time)
QBDs can be described as a special subclass of RMCs and Tree-like QBDs, which are a
generalization of QBDs, are equivalent to RMCs in a precise sense. We also prove that
for a given QBD we can compute (in the unit cost RAM model) an approximation of
its termination probabilities within i bits of precision in time polynomial in the size of
the QBD and linear in i. Specifically, we show that we can do this using a decomposed
Newton’s method
