2,450 research outputs found
Stationary distributions of continuous-time Markov chains: a review of theory and truncation-based approximations
Computing the stationary distributions of a continuous-time Markov chain involves solving a set of linear equations. In most cases of interest, the number of equations is infinite or too large, and cannot be solved analytically or numerically. Several approximation schemes overcome this issue by truncating the state space to a manageable size. In this review, we first give a comprehensive theoretical account of the stationary distributions and their relation to the long-term behaviour of the Markov chain, which is readily accessible to non-experts and free of irreducibility assumptions made in standard texts. We then review truncation-based approximation schemes paying particular attention to their convergence and to the errors they introduce, and we illustrate their performance with an example of a stochastic reaction network of relevance in biology and chemistry. We conclude by elaborating on computational trade-offs associated with error control and some open questions
Techniques for the Fast Simulation of Models of Highly dependable Systems
With the ever-increasing complexity and requirements of highly dependable systems, their evaluation during design and operation is becoming more crucial. Realistic models of such systems are often not amenable to analysis using conventional analytic or numerical methods. Therefore, analysts and designers turn to simulation to evaluate these models. However, accurate estimation of dependability measures of these models requires that the simulation frequently observes system failures, which are rare events in highly dependable systems. This renders ordinary Simulation impractical for evaluating such systems. To overcome this problem, simulation techniques based on importance sampling have been developed, and are very effective in certain settings. When importance sampling works well, simulation run lengths can be reduced by several orders of magnitude when estimating transient as well as steady-state dependability measures. This paper reviews some of the importance-sampling techniques that have been developed in recent years to estimate dependability measures efficiently in Markov and nonMarkov models of highly dependable system
A Web Aggregation Approach for Distributed Randomized PageRank Algorithms
The PageRank algorithm employed at Google assigns a measure of importance to
each web page for rankings in search results. In our recent papers, we have
proposed a distributed randomized approach for this algorithm, where web pages
are treated as agents computing their own PageRank by communicating with linked
pages. This paper builds upon this approach to reduce the computation and
communication loads for the algorithms. In particular, we develop a method to
systematically aggregate the web pages into groups by exploiting the sparsity
inherent in the web. For each group, an aggregated PageRank value is computed,
which can then be distributed among the group members. We provide a distributed
update scheme for the aggregated PageRank along with an analysis on its
convergence properties. The method is especially motivated by results on
singular perturbation techniques for large-scale Markov chains and multi-agent
consensus.Comment: To appear in the IEEE Transactions on Automatic Control, 201
On analog quantum algorithms for the mixing of Markov chains
The problem of sampling from the stationary distribution of a Markov chain
finds widespread applications in a variety of fields. The time required for a
Markov chain to converge to its stationary distribution is known as the
classical mixing time. In this article, we deal with analog quantum algorithms
for mixing. First, we provide an analog quantum algorithm that given a Markov
chain, allows us to sample from its stationary distribution in a time that
scales as the sum of the square root of the classical mixing time and the
square root of the classical hitting time. Our algorithm makes use of the
framework of interpolated quantum walks and relies on Hamiltonian evolution in
conjunction with von Neumann measurements.
There also exists a different notion for quantum mixing: the problem of
sampling from the limiting distribution of quantum walks, defined in a
time-averaged sense. In this scenario, the quantum mixing time is defined as
the time required to sample from a distribution that is close to this limiting
distribution. Recently we provided an upper bound on the quantum mixing time
for Erd\"os-Renyi random graphs [Phys. Rev. Lett. 124, 050501 (2020)]. Here, we
also extend and expand upon our findings therein. Namely, we provide an
intuitive understanding of the state-of-the-art random matrix theory tools used
to derive our results. In particular, for our analysis we require information
about macroscopic, mesoscopic and microscopic statistics of eigenvalues of
random matrices which we highlight here. Furthermore, we provide numerical
simulations that corroborate our analytical findings and extend this notion of
mixing from simple graphs to any ergodic, reversible, Markov chain.Comment: The section concerning time-averaged mixing (Sec VIII) has been
updated: Now contains numerical plots and an intuitive discussion on the
random matrix theory results used to derive the results of arXiv:2001.0630
Faster quantum mixing for slowly evolving sequences of Markov chains
Markov chain methods are remarkably successful in computational physics,
machine learning, and combinatorial optimization. The cost of such methods
often reduces to the mixing time, i.e., the time required to reach the steady
state of the Markov chain, which scales as , the inverse of the
spectral gap. It has long been conjectured that quantum computers offer nearly
generic quadratic improvements for mixing problems. However, except in special
cases, quantum algorithms achieve a run-time of , which introduces a costly dependence on the Markov chain size
not present in the classical case. Here, we re-address the problem of mixing of
Markov chains when these form a slowly evolving sequence. This setting is akin
to the simulated annealing setting and is commonly encountered in physics,
material sciences and machine learning. We provide a quantum memory-efficient
algorithm with a run-time of ,
neglecting logarithmic terms, which is an important improvement for large state
spaces. Moreover, our algorithms output quantum encodings of distributions,
which has advantages over classical outputs. Finally, we discuss the run-time
bounds of mixing algorithms and show that, under certain assumptions, our
algorithms are optimal.Comment: 20 pages, 2 figure
Asymptotic Expansions for Stationary Distributions of Perturbed Semi-Markov Processes
New algorithms for computing of asymptotic expansions for stationary
distributions of nonlinearly perturbed semi-Markov processes are presented. The
algorithms are based on special techniques of sequential phase space reduction,
which can be applied to processes with asymptotically coupled and uncoupled
finite phase spaces.Comment: 83 page
- …