Recurrent events in a Markov chain

Straightforward and direct methods are presented for finding the probabilities of a recurrent event defined over a t-state, rth-order Markov chain. A special case of this problem is that of the determination of the probabilities of events in a Bernoulli sequence. The recurrent event may be either simple or compound, nonoverlapping or overlapping. Consideration is given to stationary and nonstationary Markov chains.The probabilities determined are for:(a)the occurrence of exactly k events in n trials(b)the occurrence of the kth event on the n trial(c)the occurrence of one or more events in n trialsTo solve for the above probabilities a transformation is made of the general problem stated to the simpler problem of finding the corresponding probabilities for a set of states defined over an s state, first-order Markov chain. The probabilities of the set of states of the reduced problem are found by the use of forward differential equations developed in the past for a continuous time stochastic birth process. Backward differential equations can also be used.The results are applied to radar detection. They are also applicable to beam splitting problems and the detection of nuclear tracks