In the present paper a method to determine the probability distribution of the maximum value process (MVP) of a Markov process is proposed. In this method, an augmented vector process of a physical process and its MVP is constructed. The joint probability density function is then calculated by the path integral solution (PIS), and further the probability density function of the MVP can be obtained as the marginal probability density. A numerical example is shown to validate the proposed method.Financial supports from the National Natural Science Foundation of China (NSFC Grant Nos. 11672209, 51538010 and the National Distinguished Youth Fund of NSFC with Grant No.51725804), the NSFC-DFG joint program (11761131014) and the International Joint Research Program of Shanghai Municipal Government (Grant No. 18160712800) are highly appreciated

Chen, Jian-Bing

Lyu, Meng-Ze

SNU Open Repository and Archive

13th International Conference on Applications of Statistics and Probability in Civil Engineering, ICASP13 Seoul, South Korea, May 26-30, 2019  1 A New Approach for Capturing the Probability Density Function of the Maximum Value of a Markov Process Meng-Ze Lyu Ph.D. Student, State Key Laboratory of Disaster Reduction in Civil Engineering, Tongji University, Shanghai,  P. R. China Jian-Bing Chen Professor, State Key Laboratory of Disaster Reduction in Civil Engineering, Tongji University, Shanghai,  P. R. China ABSTRACT: In the present paper a method to determine the probability distribution of the maximum value process (MVP) of a Markov process is proposed. In this method, an augmented vector process of a physical process and its MVP is constructed. The joint probability density function is then calculated by the path integral solution (PIS), and further the probability density function of the MVP can be obtained as the marginal probability density. A numerical example is shown to validate the proposed method.  Key words: Maximum value process (MVP), Dynamics reliability, Path integral, Markov process  1. INTRODUCTION Stochastic dynamical systems or stochastic processes are studied and applied widely in various science and engineering fields. The probability distribution of the extreme value of a stochastic process, e.g., the response of stochastic dynamical systems, is of important concern, in particular in dynamical reliability of structures in civil/marine engineering (Li & Chen, 2009; Naess & Moan, 2013; Melchers & Beck, 2018). As a matter of fact, more than half a century ago, the probability distribution of the extreme value of a series of events or discrete time series was well studied (Weibull, 1951; Gumbel 1958; Ang & Tang, 1984). However, for the extreme value distributions (EVD) of stochastic processes in continuous time domain, much less understanding has been achieved. Only limited results for some special stochastic processes were available (Newland, 1993; Finkenstädt & Rootzén, 2004; Naess & Moan, 2013). The methods that can capture the probability distribution of extreme value of generic stochastic processes were unavailable till the end of last century. The probability density evolution method (PDEM) (Li & Chen, 2009; Chen & Zhang, 2013) provides a potential tool. Based on the generalized probability density evolution equation (GDEE) (Li & Chen, 2008), the EVD can be evaluated through constructing a virtual stochastic process (Chen & Li, 2007). The structural global dynamic reliability can then be evaluated (Li et al, 2007). An alternative approach is to apply an absorbing boundary condition at a certain threshold and calculate the remain probability density (Redner, 2001). However, again only for some special processes the analytical solutions of the remain probability density are available, e.g., the Brown motion process (Dudley, 1989; Kou & Zhong, 2016), the process with purely time dependent drift and diffusion (Molini et al, 2011), and so on. For general processes, even Markov processes, how to capture its remain probability density analytically at a certain threshold is still a challenging problem.  13th International Conference on Applications of Statistics and Probability in Civil Engineering, ICASP13 Seoul, South Korea, May 26-30, 2019  2 The study on maximum value process (MVP) of the Markov process is another perspective on this issue. In the present paper, an augmented vector Markov process is constructed by combining the MVP with the underlying Markov process. The path integral solution (PIS) is then adopted to yield the probability distribution of this augmented process. A numerical example is illustrated to demonstrate the effectiveness of the proposed method. 2. MAXIMUM VALUE PROCESS AND PATH INTEGRAL SOLUTION OF PROBABILITY DENSITY FUNCTION For a continuous stochastic process , the corresponding MVP can be defined as   (1) Clearly,  is a stochastic process, and is non-decreasing monotonous. Such a time-dependent maximum value is of great engineering significance. For instance, the probability without any excursion of the system, i.e., the reliability, is   (2) where  denotes the probability of the bracketed event. Obviously, the reliability is equivalent to (Chen & Li 2007)   (3) where  is the PDF of the MVP  defined in Eq. (1). Unfortunately, as discussed in the preceding section, for a general stochastic process, it is difficult to obtain the analytical solution of the probability density function (PDF) of MVP, except several very special cases (Dudley, 1989; Redner, 2001; Molini et al, 2011; Kou & Zhong, 2016). In the present paper, a new method will be developed for the determination of the PDF of MVP of a Markov process. To make it clear, for convenience, consider a Markov process  governed by the following state equation, namely, the Itô SDE  (4) where  and  are two functions satisfying appropriate regulation conditions;  is a Wiener process with  and ;  is the intensity. It is known that the stochastic process  governed by the Itô SDE [Eq. (4)] is a Markov process (Gardiner, 1985).  The MVP  itself is not a Markov process. However, the augmented vector process  is a Markov vector process. Though the analytical solution for the augmented Markov vector process  is still unavailable for general problems, numerical methods are feasible and will be elaborated by PIS. According to the Chapman-Kolmogorov equation   (5) in which  are three arbitrary different time instants,  is the transition probability density (TPD) from the state  at time  to the state  at time . Similar meaning apply to the other symbols. According to Eq. (5), the recursive format of PIS to obtain the joint PDF of  can be written as   (6) 13th International Conference on Applications of Statistics and Probability in Civil Engineering, ICASP13 Seoul, South Korea, May 26-30, 2019  3 where  is the joint PDF of ;  is an arbitrary time increment;  is the TPD given by  (7) where  is the Heaviside step function;  is Dirac’s δ function. Once the joint PDF  is obtained, the PDF of MVP can be further obtained by   (8)Simultaneously, the state PDF can also be obtained as a marginal PDF   (9) 3. NUMERICAL EXAMPLE In this section, a numerical example is studied to illustrate the approach for determining the PDF of the MVP of Markov processes. Consider a nonlinear, additive white noise excited, one-dimensional diffusion process  with the following Itô SDE (Er, 2000)  (10) where  are coefficients;  is a Brownian motion process with intensity . As , the stationary PDF of  has the expression (Er, 2000)   (11) where  is the normalization constant given by   (12) In a small time increment , the TPD of  can be written as   (13) The MVP  of  is defined by Eq. (1). Then, the TPD of the augmented vector process  can be obtained as   (14) Therefore, the PDF  of  can be obtained by the proposed method. Then, the PDF  of  can be obtained by a marginal integral. In this case, the initial value is taken as ; the intensity of white noise is ; the coefficients are ; the time step is ; the solving domain is  and ; and the grid size is . The PDF surfaces of  and  against time are shown, respectively, in Figure 1.  13th International Conference on Applications of Statistics and Probability in Civil Engineering, ICASP13 Seoul, South Korea, May 26-30, 2019  4  (a) PDF surface of the underlying process ;  (b) PDF surface of the MVP . Figure 1: The PDF surfaces and their contours of  and  of a scalar diffusion process.  At the time , the comparison between the numerical result and the stationary analytical solution of PDF of  is shown in Figure 2. Again perfect agreement is observed between the analytical and numerical results.    (a) In linear coordinates;  (b) In logarithmic coordinates. Figure 2: The comparison between the numerical and stationary analytical solution of PDF of a scalar diffusion process  at .  To compare with the numerical solution of PDF of  by the proposed method, MCS is carried out.  samples of  are conducted in the MCS. The comparison between the PDF and cumulative distribution function (CDF) of  at the time  by the proposed method and MCS, respectively, are shown in Figure 3. Again, fairly good agreement between the results by the two methods is observed. Obvious skewness is also observed from Figure 3(a). It is also noted that the accuracy of the tail of CDF is at least in the order of magnitude of 10-5.   (a) PDF;  p p13th International Conference on Applications of Statistics and Probability in Civil Engineering, ICASP13 Seoul, South Korea, May 26-30, 2019  5  (b) CDF (in linear coordinates);  (c) CDF (in logarithmic coordinates). Figure 3: The comparison between PIS and MCS of PDF and CDF of  of a scalar diffusion process at .  4. CONCLUDING REMARKS In this paper, the theoretical and numerical investigations on the maximum value process (MVP) of Markov process are carried out. The MVP of a Markov process, combined with the underlying process, constitutes an augmented vector Markov process. The numerical solution of its PDF can then be obtained by the PIS. The proposed method is verified by a numerical example, where the numerical solution obtained by PIS agrees well with the analytical solution and MCS. It is shown that the results obtained by PIS are of high accuracy even in the tail. The basic idea can be extended to more complex systems. 5. ACKNOWLEDGEMENTS Financial supports from the National Natural Science Foundation of China (NSFC Grant Nos. 11672209, 51538010 and the National Distinguished Youth Fund of NSFC with Grant No.51725804), the NSFC-DFG joint program (11761131014) and the International Joint Research Program of Shanghai Municipal Government (Grant No. 18160712800) are highly appreciated. 6. REFERENCES Ang, A. H. S., and Tang, W. H. (1984). “Probability Concepts in Engineering Planning and Design”. Vol 2, John Wiley and Sons, New York. Chen, J. B., and Li, J. (2007). “The extreme value distribution and dynamic reliability analysis of nonlinear structures with uncertain parameters” Structural Safety, 29, 77-93. Chen, J. B., and Zhang, S. H. (2013). “Improving point selection in cubature by a new discrepancy” SIAM Journal on Scientific Computing, 35(5), A2121-A2149. Dudley, R. M. (1989). “Real Analysis and Probability” The Press Syndicate of the University of Cambridge, Cambridge. Er, G. K. (2000). “Exponential closure method for some randomly excited non-linear systems” International Journal of Non-Linear Mechanics, 35, 69-78. Finkenstädt, B., and Rootzén, H. (2004). “Extreme Values in Finance, Telecommunications, and the Environment” Chapman and Hall/CRC, Boca Raton. Gardiner, C. W. (1985). “Handbook of Stochastic Methods for Physics” 2nd Ed., Springer-Verlag, Berlin. Gumbel, E. J. (1958). “Statistics of Extremes” Columbia University Press, New York. Kou, S., and Zhong, H. (2016). “First-passage times of two-dimensional Brownian motion” Advanced Applied Probability, 48, 1045-1060. Li, J., Chen, J. B., and Fan, W. L. (2007). “The equivalent extreme-value event and evaluation of the structural system reliability” Structural Safety, 29, 112-131. Li, J., and Chen, J. B. (2008). “The principle of preservation of probability and the generalized density evolution equation” Structural Safety, 30, 65-77. 13th International Conference on Applications of Statistics and Probability in Civil Engineering, ICASP13 Seoul, South Korea, May 26-30, 2019  6 Li, J., and Chen, J. B. (2009). “Stochastic Dynamics of Structure” John Wiley & Sons (Asia) Pte Ltd, Singapore. Melchers, R. E., and Beck, A. T. (2018). “Structural Reliability Analysis and Prediction” 3rd Ed., John Wiley & Sons, Chichester. Monili, A., Talkner, P., Katul, G. G., and Porporato, A. (2011). “First passage time statistics of Brownian motion with purely time dependent drift and diffusion” Physica A, 390, 1841-1852. Naess, A., and Moan, T. (2013). “Stochastic Dynamics of Marine Structures” Cambridge University Press, Cambrige. Newland, D. E. (1993). “An Introduction to Random Vibration, Spectral Analysis & Wavelet Analysis” 3rd Ed., Longman, London & New York. Redner, S. (2001). “A Guide to First-Passage Processes” Cambridge University Press, Cambridge. Weibull, W. (1951). “A statistical distribution function of wide applicability” ASME Journal of Applied Mechanics, Sept., 293-297. 

A New Approach for Capturing the Probability Density Function of the Maximum Value of a Markov Process

https://s-space.snu.ac.kr/bitstream/10371/153335/1/127.pdf

A New Approach for Capturing the Probability Density Function of the Maximum Value of a Markov Process

Abstract

Similar works

Full text

Available Versions

SNU Open Repository and Archive