Tijms, H.C.

DSpace at VU

Faculteit der Economische Wetenschappen en Econometrie 
Serie Research Memoranda 
Computational Probability: Old Ideas 
Ne ver Die 
Henk C. Tijms 
Research Memorandum 1991-58. 
August 1991 
vrije Universiteit amsterdam 

COMPUTATIONAL PROBABILITY: OLD IDEAS NEVER DIE 
Henk Tijms 
Dept. of Econometrics 
Vrije University 
1081 HV Amsterdam, The Netheriands 
^TBS? 
This paper puts on the stage old ideas that are still very useful in compu-
tational probability problems arising in telecommunication and teletraffic 
applications. Basic computational problems to be discussed are the calcula-
tion of equilibrium and transient solutions for Markovian systems and the 
calculation of overfow probabilities in finite-buffer systems. 
1 
1. Introduction 
Basic problems in computational probability are: 
1. Solving the equilibrium equations of an infinite Markov chain. 
2. Approximating overflow probabilities in finite-buffer queues. 
3. Calculating transient solutions for Markovian systems. 
Computational probability is more than getting numerical answers. The 
essence of computational probability is to have probabilistic ideas which 
make the computations transparent and natural. This paper puts on the stage 
old probabilistic ideas that are not very known to the teletraffic communi-
ty but are often very useful for computational purposes. 
2. Equilibrium distributions for an infinite 
Markov chain 
A frequently arising problem is the computation of the (unique) equilibrium 
distribution {n } of an infinite Markov chain with state space I={0,1,...}: 
rtj - C o Vu • j=0'1--
where the p ' s denote the one-step transition probabilities of the Markov 
chain. The generally used approach is to choose a sufficiently large in-
teger M so that (hopefully) £ n s e with e a very small number and next 
to solve the truncated system of linear equations: 
n = L » P . j=0,l,.. . ,M-l, j ^1=0 r i j J 
zf1 « = i -
^ j = o j 
The drawbacks of this approach are: 
a. M may be very large. 
b. Convergence problems may arise for the iterative methods that have to be 
used when M is large. 
Also, it is somewhat disconcerting that we need a brute-force approximation 
to solve numerically the infinite-state model. Usually we introducé in-
finite-state models to obtain mathematical simplification, and now in its 
numerical analysis using a brute-force truncation we are proceeding in the 
reverse direction. 
However, in most practical situations the infinite Markov chain sat is-
fies regularity conditions that enables us to apply an old idea going at 
least back to Everett [2]: 
Use that the state probabilities ir drop geo-
metrically f ast to zero as j increases. 
That is, for some constant 0<T<1, 
2 
TT 
* T for j large enough. (1) 
n 
J-i 
Conditions under which this asymptotic expansion holds are discussed in 
Tijms and Van de Coevering [5]. Roughly, it is required that Iï(z)=y^_ ir z 
can be represented as A(z)/B(z) with B(z) analytic outside the unit circle. 
Then 1/x is the smallest zero of B(z) outside the unit circle. The asympto-
tic result (1) implies that, for a sufficiently large integer N, 
n « 7r T for all j^N. 
j N 
Substituting this into the equilibrium equations of the infinite Markov 
chain leads to the following finite system of linear equations: 
ir = 'T; i r p + 7 r p , j=0,l N-1 
j ^ 1 = 0 V i j N *Nj J 
^J=0 . 
7T 
•j + TZ = h 
where p = 7; x p . This reduction of the infinite system of equilib-
Nj i J i = N ij 
rium equations to a finite system of linear equations is not based on brute 
force, but makes essential use of properties of the infinite-state model. 
From a practical point of view the question is of course: Is the size of 
this finite system smaller that that of the finite system obtained by 
brute-force truncation? The answer is in the affirmative. It is an empiri-
cal finding that for practical purposes the asymptotic expansion (1) can be 
used already for remarkably small values of j . How large N should be chosen 
has to be determined experimentally and depends of course on the specific 
application and the required accuracy in the s tate probabilities. It is our 
experience that an appropriate value for N is typically in the order of 1-
200. Consequently, the resulting system of linear equations can routinely 
be solved by Gaussian elimination methods for which reliable and fast codes 
are widely available. 
It is hoped that the approach using the geometrie tail behavior of the 
s tate probabilities will be used more in practice. It is a simple numerical 
approach that can be easily used by the nonspecialist using Standard soft-
ware. We successfully applied the approach to many basic queueing models 
including the M/D/c queue and the D/G/l queue. 
3 . Overflow probabilities in f inite-buffer queues 
A practical problem of considerable interest is the calculation of (very 
small) overflow probabilities in finite-buffer queues. Let us first con-
sider this problem for a queueing system in which customers arrive singly. 
A common heuristic for obtaining the overflow probability in the finite-
buffer queue with a maximum number of K customers allowed is: 
n « T nla\ 
overflow ^j=K j 
(00) 
where n is the equilibrium probability that in the corresponding ïn-
3 
finite-buffer queue a customer finds upon arrival j other customers 
present. Here it is assumed that the offered traffic intensity is less than 
1. That is, the overflow probability is approximated by the steady-state 
probability that in the infinite-buffer queue a customer finds upon arrival 
K or more other customers present. This approximation may perform rather 
unsatisfactorily. A much improved approximation was derived in Tijms [6] by 
using the following approximation idea: 
Assume that the first K-l state probabilities (of the interior states) 
in the finite-buffer queue are proportional to the first K-l state 
probabilities in the corresponding infinite-buffer queue. 
This approximation idea results from an old question of J.W. Cohen (private 
communication) whether there are other queueing systems than the M/G/l/K 
queue and the M/M/c/K queue for which the proportionality result for the 
state probabilities holds. The improved heuristic for the overflow proba-
bility in the finite-buffer queue is given by: 
a-P) ^_K n 
n ~ j K J 
(00) 
overflow , Y-JOO (oo) 
1 - p Y~ 11 
where p is the offered traffic intensity. Numerical investigations in Tijms 
[6] indicate that the new heuristic performs very well for practical pur-
poses. The heuristic provides an estimate that is typically of the same 
order of magnitude as the exact value of the overflow probability. This is 
what is needed for a heuristic used for dimensioning the buffer size. 
Also, an extension of the heuristic to finite-buffer queues with batch 
arrivals was suggested in Tijms [6]. Denoting by {f$ } the batch-size dis-
tribution, the extended heuristic is: 
II 
(I-P) C K «J (») 
overflow , x-fa (oo) 
1
 "
 P
 h^ QJ 
with 
(00) j (00) 
(oo) 
where n now denotes the steady-state probability that in the correspond-
ing infinite-buffer queue an arriving batch sees j other customers. 
4. Transient solutions for Markovian systems 
How to compute transient solutions for Markovian systems is a very impor-
tant question that arises in numerous problems in teletraffic, telecommuni-
cation and reliability. Supposing a continuous-time Markov chain {X(t),t£0} 
with discrete state space I, basic problems are: 
a. Compute the transient probabilities 
p (t) = P{X(t)=j|X(0)=i}. 
4 
b. Compute first-passage time probabilites 
f (t) = P {system reaches the set A of 
IA 
states bef ore time t|X(0)=i>. 
c. Compute cumulative expected rewards 
E [^ r(X(u))du|X(0)=i] 
when a reward at ra te r(j) is incurred whenever 
the process is in state j . 
In f act, the problems b and c can be reduced to problem a (e.g. for problem 
b make the state of the set A absorbing). 
The usual approach to answer above questions is to set up a system of 
linear differential equations and to solve them using Runge-Kutta methods. 
However, there is a better approach called Jensen's method. This method is 
also known under the names randomization method and uniformization method. 
The name Jensen's method was recently proposed in Grassmann [3]. This 
method that was already introduced in 1953 by Jensen [4] has recently 
become very popular in operations research and computer science and has 
seen many interesting applications. Remarkably, the use of Jensen's method 
is not widespread in the field of teletraffic theory, though it is a power-
ful and beautiful method. What is the idea behind the method? Suppose for 
the moment that the leaving rates v of the states of the continuous-time 
Markov chain are identical, say v =v for all i. Then, using that the number 
of transitions in a time interval of length t is Poisson distributed with 
mean vt, the transient probabilities are easy to compute from: 
P..<t> = £ L e 
n 
-Vt (Vt) (n) 
P, ij ^ n ^ n! Mj 
where p is the probability that in n state transitions the process goes 
from state i to state j . The n-step transition probabilities can be recur-
sively computed. However, in general the leaving rates v are not identi-
cal. To overcome this difficulty, the old idea of Jensen [4] is: 
Make the leaving rates identical by introducing fictitious transitions 
leaving the state of the system unchanged. 
That is, take a uniform leaving ra te 
v = max v i i 
and modify the one-step transition probabilities p (with the convention 
p =0) accordingly 
v 
t i 
P iJ 
V p i j ' J*1 
v 
i - ^ . J-I. 
5 
The method of Jensen has as advantages: 
- insightful and easy to program 
- numerically stable (roundoff errors can be easily controlled) 
- infinite or very large state spaces can often be handled by a dynamic 
implementation of the method. 
The latter point needs some explanation. Notice first that the infinite 
series for p (t) can be truncated on beforehand for a given value of t by 
choosing M so that 
o^o -vt (vt)n (n) r^o -vt (v t ) n 
L e —;— P - L e —i— - e 
n^=M n! *1J Si=M n! 
for some precision e. Next this bound M may be used in dynamically adjust-
ing the set of states that can be reached in n(<M) transitions from the 
given initial s tate i. 
It is our experience that Jensen's method tends to outperform Runge-
Kutta methods, see also Grassmann [3]. The method is not only superior from 
a computational point of view, but its probabilistic transparency also 
allows to attack problems that are otherwise be difficult to handle. For 
example, the paper of De Sousa e Silva and Gail [1] uses Jensen's method to 
give a very nice algorithm to compute the probability distribution of the 
sojourn time in a given set of states of a continuous-time Markov chain. 
REFERENCES 
[1] De Sousa e Silva, E. and Gail, H.R. (1986), Calculating cumulative 
operational time distributions of repairable computer systerns, IEEE 
Transactions on Computers 35, 322-332. 
[2] Everett, J.L. (1954), State probabilities in congestion problems 
characterized by constant holding times, Operations Research 1, 279-
285. 
[3] Grassmann, W.K. (1991), Finding transient solutions in Markovian event 
systems through randomization. In: The First International Conference 
on the Numerical Solution of Markov Chains, ed. W.J. Stewart, Marcel 
Dekker, New York. 
[4] Jensen, A. (1953), Markoff chains as an aid in the study of Markoff 
processes, Skandinavisk Aktuarietidskrift 36, 87-91. 
[5] Tijms, H.C. and Van de Coevering, M.C.T. (1991), A simple numerical 
approach for infinite Markov chains, Probability in the Engineering 
and Informational Sciences 5, 285-295. 
[6] Tijms, H.C. (1992), Heuristics for finite-buffer queues, Probability 
in the Engineering and Information Sciences 6 (to appear). 
HT917bT2.chl 


Computational probability

VU Research Portal

VU Research PortalComputational probabilityTijms, H.C.1991document versionPublisher's PDF, also known as Version of recordLink to publication in VU Research Portalcitation for published version (APA)Tijms, H. C. (1991). Computational probability. (Serie Research Memoranda; No. 1991-58). Faculty ofEconomics and Business Administration, Vrije Universiteit Amsterdam.General rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright ownersand it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.            • Users may download and print one copy of any publication from the public portal for the purpose of private study or research.            • You may not further distribute the material or use it for any profit-making activity or commercial gain            • You may freely distribute the URL identifying the publication in the public portal ?Take down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediatelyand investigate your claim.E-mail address:vuresearchportal.ub@vu.nlDownload date: 22. May. 2021Faculteit der Economische Wetenschappen en Econometrie Serie Research Memoranda Computational Probability: Old Ideas Ne ver Die Henk C. Tijms Research Memorandum 1991-58. August 1991 vrije Universiteit amsterdam COMPUTATIONAL PROBABILITY: OLD IDEAS NEVER DIE Henk Tijms Dept. of Econometrics Vrije University 1081 HV Amsterdam, The Netheriands ^TBS? This paper puts on the stage old ideas that are still very useful in compu-tational probability problems arising in telecommunication and teletraffic applications. Basic computational problems to be discussed are the calcula-tion of equilibrium and transient solutions for Markovian systems and the calculation of overfow probabilities in finite-buffer systems. 1 1. Introduction Basic problems in computational probability are: 1. Solving the equilibrium equations of an infinite Markov chain. 2. Approximating overflow probabilities in finite-buffer queues. 3. Calculating transient solutions for Markovian systems. Computational probability is more than getting numerical answers. The essence of computational probability is to have probabilistic ideas which make the computations transparent and natural. This paper puts on the stage old probabilistic ideas that are not very known to the teletraffic communi-ty but are often very useful for computational purposes. 2. Equilibrium distributions for an infinite Markov chain A frequently arising problem is the computation of the (unique) equilibrium distribution {n } of an infinite Markov chain with state space I={0,1,...}: rtj - C o Vu • j=0'1--where the p ' s denote the one-step transition probabilities of the Markov chain. The generally used approach is to choose a sufficiently large in-teger M so that (hopefully) £ n s e with e a very small number and next to solve the truncated system of linear equations: n = L » P . j=0,l,.. . ,M-l, j ^1=0 r i j J zf1 « = i -^ j = o j The drawbacks of this approach are: a. M may be very large. b. Convergence problems may arise for the iterative methods that have to be used when M is large. Also, it is somewhat disconcerting that we need a brute-force approximation to solve numerically the infinite-state model. Usually we introducé in-finite-state models to obtain mathematical simplification, and now in its numerical analysis using a brute-force truncation we are proceeding in the reverse direction. However, in most practical situations the infinite Markov chain sat is-fies regularity conditions that enables us to apply an old idea going at least back to Everett [2]: Use that the state probabilities ir drop geo-metrically f ast to zero as j increases. That is, for some constant 0<T<1, 2 TT * T for j large enough. (1) n J-i Conditions under which this asymptotic expansion holds are discussed in Tijms and Van de Coevering [5]. Roughly, it is required that Iï(z)=y^_ ir z can be represented as A(z)/B(z) with B(z) analytic outside the unit circle. Then 1/x is the smallest zero of B(z) outside the unit circle. The asympto-tic result (1) implies that, for a sufficiently large integer N, n « 7r T for all j^N. j N Substituting this into the equilibrium equations of the infinite Markov chain leads to the following finite system of linear equations: ir = 'T; i r p + 7 r p , j=0,l N-1 j ^ 1 = 0 V i j N *Nj J ^J=0 . 7T •j + TZ = h where p = 7; x p . This reduction of the infinite system of equilib-Nj i J i = N ij rium equations to a finite system of linear equations is not based on brute force, but makes essential use of properties of the infinite-state model. From a practical point of view the question is of course: Is the size of this finite system smaller that that of the finite system obtained by brute-force truncation? The answer is in the affirmative. It is an empiri-cal finding that for practical purposes the asymptotic expansion (1) can be used already for remarkably small values of j . How large N should be chosen has to be determined experimentally and depends of course on the specific application and the required accuracy in the s tate probabilities. It is our experience that an appropriate value for N is typically in the order of 1-200. Consequently, the resulting system of linear equations can routinely be solved by Gaussian elimination methods for which reliable and fast codes are widely available. It is hoped that the approach using the geometrie tail behavior of the s tate probabilities will be used more in practice. It is a simple numerical approach that can be easily used by the nonspecialist using Standard soft-ware. We successfully applied the approach to many basic queueing models including the M/D/c queue and the D/G/l queue. 3 . Overflow probabilities in f inite-buffer queues A practical problem of considerable interest is the calculation of (very small) overflow probabilities in finite-buffer queues. Let us first con-sider this problem for a queueing system in which customers arrive singly. A common heuristic for obtaining the overflow probability in the finite-buffer queue with a maximum number of K customers allowed is: n « T nla\ overflow ^j=K j (00) where n is the equilibrium probability that in the corresponding ïn-3 finite-buffer queue a customer finds upon arrival j other customers present. Here it is assumed that the offered traffic intensity is less than 1. That is, the overflow probability is approximated by the steady-state probability that in the infinite-buffer queue a customer finds upon arrival K or more other customers present. This approximation may perform rather unsatisfactorily. A much improved approximation was derived in Tijms [6] by using the following approximation idea: Assume that the first K-l state probabilities (of the interior states) in the finite-buffer queue are proportional to the first K-l state probabilities in the corresponding infinite-buffer queue. This approximation idea results from an old question of J.W. Cohen (private communication) whether there are other queueing systems than the M/G/l/K queue and the M/M/c/K queue for which the proportionality result for the state probabilities holds. The improved heuristic for the overflow proba-bility in the finite-buffer queue is given by: a-P) ^_K n n ~ j K J (00) overflow , Y-JOO (oo) 1 - p Y~ 11 where p is the offered traffic intensity. Numerical investigations in Tijms [6] indicate that the new heuristic performs very well for practical pur-poses. The heuristic provides an estimate that is typically of the same order of magnitude as the exact value of the overflow probability. This is what is needed for a heuristic used for dimensioning the buffer size. Also, an extension of the heuristic to finite-buffer queues with batch arrivals was suggested in Tijms [6]. Denoting by {f$ } the batch-size dis-tribution, the extended heuristic is: II (I-P) C K «J (») overflow , x-fa (oo) 1 " P h^ QJ with (00) j (00) (oo) where n now denotes the steady-state probability that in the correspond-ing infinite-buffer queue an arriving batch sees j other customers. 4. Transient solutions for Markovian systems How to compute transient solutions for Markovian systems is a very impor-tant question that arises in numerous problems in teletraffic, telecommuni-cation and reliability. Supposing a continuous-time Markov chain {X(t),t£0} with discrete state space I, basic problems are: a. Compute the transient probabilities p (t) = P{X(t)=j|X(0)=i}. 4 b. Compute first-passage time probabilites f (t) = P {system reaches the set A of IA states bef ore time t|X(0)=i>. c. Compute cumulative expected rewards E [^ r(X(u))du|X(0)=i] when a reward at ra te r(j) is incurred whenever the process is in state j . In f act, the problems b and c can be reduced to problem a (e.g. for problem b make the state of the set A absorbing). The usual approach to answer above questions is to set up a system of linear differential equations and to solve them using Runge-Kutta methods. However, there is a better approach called Jensen's method. This method is also known under the names randomization method and uniformization method. The name Jensen's method was recently proposed in Grassmann [3]. This method that was already introduced in 1953 by Jensen [4] has recently become very popular in operations research and computer science and has seen many interesting applications. Remarkably, the use of Jensen's method is not widespread in the field of teletraffic theory, though it is a power-ful and beautiful method. What is the idea behind the method? Suppose for the moment that the leaving rates v of the states of the continuous-time Markov chain are identical, say v =v for all i. Then, using that the number of transitions in a time interval of length t is Poisson distributed with mean vt, the transient probabilities are easy to compute from: P..<t> = £ L e n -Vt (Vt) (n) P, ij ^ n ^ n! Mj where p is the probability that in n state transitions the process goes from state i to state j . The n-step transition probabilities can be recur-sively computed. However, in general the leaving rates v are not identi-cal. To overcome this difficulty, the old idea of Jensen [4] is: Make the leaving rates identical by introducing fictitious transitions leaving the state of the system unchanged. That is, take a uniform leaving ra te v = max v i i and modify the one-step transition probabilities p (with the convention p =0) accordingly v t i P iJ V p i j ' J*1 v i - ^ . J-I. 5 The method of Jensen has as advantages: - insightful and easy to program - numerically stable (roundoff errors can be easily controlled) - infinite or very large state spaces can often be handled by a dynamic implementation of the method. The latter point needs some explanation. Notice first that the infinite series for p (t) can be truncated on beforehand for a given value of t by choosing M so that ôo -vt (vt)n (n) r̂o -vt (v t ) n L e —;— P - L e —i— - e n̂=M n! *1J Si=M n! for some precision e. Next this bound M may be used in dynamically adjust-ing the set of states that can be reached in n(<M) transitions from the given initial s tate i. It is our experience that Jensen's method tends to outperform Runge-Kutta methods, see also Grassmann [3]. The method is not only superior from a computational point of view, but its probabilistic transparency also allows to attack problems that are otherwise be difficult to handle. For example, the paper of De Sousa e Silva and Gail [1] uses Jensen's method to give a very nice algorithm to compute the probability distribution of the sojourn time in a given set of states of a continuous-time Markov chain. REFERENCES [1] De Sousa e Silva, E. and Gail, H.R. (1986), Calculating cumulative operational time distributions of repairable computer systerns, IEEE Transactions on Computers 35, 322-332. [2] Everett, J.L. (1954), State probabilities in congestion problems characterized by constant holding times, Operations Research 1, 279-285. [3] Grassmann, W.K. (1991), Finding transient solutions in Markovian event systems through randomization. In: The First International Conference on the Numerical Solution of Markov Chains, ed. W.J. Stewart, Marcel Dekker, New York. [4] Jensen, A. (1953), Markoff chains as an aid in the study of Markoff processes, Skandinavisk Aktuarietidskrift 36, 87-91. [5] Tijms, H.C. and Van de Coevering, M.C.T. (1991), A simple numerical approach for infinite Markov chains, Probability in the Engineering and Informational Sciences 5, 285-295. [6] Tijms, H.C. (1992), Heuristics for finite-buffer queues, Probability in the Engineering and Information Sciences 6 (to appear). HT917bT2.chl 

English

http://dare.ubvu.vu.nl/bitstream/handle/1871/12305/Scanjob%20199100058.pdf?sequence=1&isAllowed=y

Computational probability

Abstract

Similar works

Full text

Available Versions

DSpace at VU

VU Research Portal