For many complex probabilistic problems involving continuous random variables, it is impossible to find a closed-form solution. Consequently, it is necessary to approximate a given continuous probability distribution with a discrete one. Several techniques are
available for producing a discrete approximation to a continuous random variable, based on rather di↵erent criteria. In this work, we will examine and compare these main methods, by considering a well-known parametric distribution. Applications to real problems will finally be suggested

A. Barbiero

Asmerilda Hitaj

English

AIR Universita degli studi di Milano

International Conference onMathematicalAnalysis andApplications in Science andEngineeringICMA2SC’24ISEP Porto-Portugal, June 20 - 22, 2024Choosing among different discrete approximations ofa continuous random variableAlessandro Barbiero1⋆ Asmerilda Hitaj21 Università degli Studi di Milano, Italy2 Università degli Studi dell’Insubria, Italy1⋆alessandro.barbiero@unimi.itAbstract: For many complex probabilistic problems involving continuous random variables,it is impossible to find a closed-form solution. Consequently, it is necessary to approximatea given continuous probability distribution with a discrete one. Several techniques areavailable for producing a discrete approximation to a continuous random variable, based onrather different criteria. In this work, we will examine and compare these main methods, byconsidering a well-known parametric distribution. Applications to real problems will finallybe suggested.keywords: discretization; moment-matching; quantization; representative points.MSC2020: 62E15; 62E17.1 IntroductionFor many complex probabilistic problems involving continuous random variables, it is impossibleto find a closed-form solution. Consequently, it is necessary to approximate a given continuousprobability distribution with a discrete one. For a continuous random variableX with cumulativedistribution function F (x), one has to construct a discrete random variable X̃ defined on S ={x1, x2, . . . , xk}, with probability mass function P (X̃ = xi) = pi, i = 1, . . . , k, ensuring thatX̃ preserves the properties of X “as much as possible”. Various discretization methods can beemployed for determining the xi and the pi, relying on different criteria [1]. The most widelyaccepted criterion involves matching as many moments as possible of X. Alternatively, one canoptimally select the values xi in the sense of minimizing the expected mean squared error withX (optimal quantization). Another approach in based on minimizing the Cramér-von Misesdistance between F (x) and the step-wise cumulative distribution function of X̃. In this work,we will examine and compare these main methods and modifications thereof, by consideringseveral well-known parametric distributions (normal, exponential, etc.). Possible applicationsto real problems will be finally hinted at.2 Results and discussionIt is well-known that a k-point discrete random variable X̃, which is characterized by 2k values -the k support points (x1, x2, . . . , xk) and the corresponding k probabilities (p1, p2, . . . , pk) - canpreserve up to the first 2k − 1 moments of a continuous random variable X. The values of thexi and pi which lead to the exact matching of all the first 2k − 1 moments can be obtained bysolving the corresponding non-linear system of 2k equations in the 2k unknowns:E(X̃) =k∑i=1pi · xri = mr = E(Xr), r = 0, 1, . . . , 2k − 1, (1)where mr is the r-th raw moment of the continuous random variable, and for r = 0 we havethe trivial requirement that the pi must sum up to 1. This can be done by resorting to the so-called Gaussian quadrature procedure and solving for the roots of an orthogonal polynomial [3]or computing the eigenvalues and first component of the orthonormalized eigenvectors of asymmetric tridiagonal matrix [4].Alternatively, a k-point discrete approximation can be obtained by minimizing the following lossfunction, which is actually an expected mean squared distance:LX(x1, x2, . . . , xk) =∫ ∞−∞mini(x− xi)2f(x)dx.After defining a set of k+1 thresholds as δ0 = −∞, δk = ∞ and δi = (xi+xi+1)/2, i = 1, . . . , k−1,the loss function above can be expressed in the formLX(x1, x2, . . . , xk) =k∑i=1∫ δiδi−1(x− xi)2f(x)dx.By minimizing LX with respect to the xi, one can obtain the optimal xi, called the “principalpoints” [4], and the δi; the corresponding probabilities pi are obtained as pi =∫ δiδi−1f(x)dx. Thisapproximation is also known as “optimal quantization” [5]. The resulting rv X̃ always preservesthe first moment, but underestimates the variance of X.[7] suggested a slight modification to “optimal quantization”, in order to match the secondmoment of the continuous distribution (along with its expectation). The optimal k-point discreteapproximation is derived as the solution to a nonlinear optimization problem and can be actuallyregarded as the discrete distribution which is the closest in mean-squared-error terms to theoptimal quantizer, while preserving the first two moments. The probabilities are the same asfor optimal quantization, whereas the modified support values are given byx∗i =σXσX̃(xi − E(X)) + E(X).The three methods illustrated so far require the finiteness of (at least) the first two moments, acondition which is not satisfied by many heavy-tailed distributions used in quantitative finance,just think of stable distributions. In this case, one can perform discretization based on theminimization of some distance between the continuous cumulative distribution function of X,F , and the piece-wise constant cumulative distribution function of X̃, G, e.g,d2(F,G) =∫R[F (x)−G(x)]2w(x)dx,with w(x) a (positive) weighting function. If w(x) = F ′(x) the function above reduces to Cramér-von Mises distance, and in this latter case, then the optimal discretization assigns constantweights pi = 1/k to the values xi = F−1(ui), with ui = (2i− 1)/(2k) [7].We now apply the four discretization methods discussed in this section to the exponential randomdistribution, which is characterized by the cumulative distribution function F (x) = 1−e−λx and2the probability density function f(x) = λe−λx, x > 0, where λ > 0 is the rate parameter. Thequantile function is F−1(u) = − ln(1 − u)/λ, 0 < u < 1. We recall that all the positive integermoments of the exponential distribution exist and are equal to mr = r!/λr.Table 1 displays the support values xi and the corresponding probabilities pi for a 7-pointapproximation of an exponential random variable with unit rate parameter. The last two rowsreports the expectation and the variance of the approximating random variable.Table 1: Approximation of an exponential rv with unit parameter through k = 7 points. Leg-end: MM=Moment Matching; OQ=Optimal Quantization; DZ=Drezner and Zerom (2016);minCvM=minimization of Cramér-von Mises distance.MM OQ DZ minCvMxi pi xi pi xi pi xi pi0.193 0.409 0.199 0.348 0.182 0.348 0.074 0.1431.027 0.422 0.657 0.256 0.650 0.256 0.241 0.1432.568 0.147 1.197 0.179 1.201 0.179 0.442 0.1434.900 0.021 1.857 0.115 1.875 0.115 0.693 0.1438.182 0.001 2.705 0.065 2.740 0.065 1.030 0.14312.734 < 0.001 3.893 0.029 3.951 0.029 1.540 0.14319.396 < 0.001 5.893 0.008 5.992 0.008 2.639 0.143E(X̃) 1 E(X̃) 1 E(X̃) 1 E(X̃) 0.951Var(X̃) 1 Var(X̃) 0.961 Var(X̃) 1 Var(X̃) 0.686From Table 1, we can see that the different discretization techniques give rise to discrete randomvariables that can sensibly differ: one can note, in particular, that the range of the approxima-tion induced by moment matching is quite larger than the others, and includes two points with areally small probability; to the contrary opposite, the discretization based on the minimizationof the Cramér-von Mises distance leads to a distribution with the narrowest range and con-stant probabilities. Moreover, this latter approximation underestimates the first two moments,whereas the other three methods by construction preserve the first o the first two moments.3 Conclusions and future workA natural application of discrete approximation of a continuous random variable is scenariogeneration in stochastic programming. Here, the continuous stochastic quantity involved inthe optimization problem is substituted by a random variable sitting on a finite number ofpoints, the so-called “atoms” or “scenarios”. Future work will consider some typical problems instochastic programming, apply the different discretizations discussed in this work, and comparethe resulting solutions.AcknowledgmentsThe corresponding author acknowledges financial support from the Italian Ministry of Universityand Research through the PRIN2022 “The effects of climate change in the evaluation of financialinstruments”.3References[1] R.K. Hammond, J.E. Bickel, Reexamining discrete approximations to continuous distribu-tions, Decision Analysis, 10(1) (2013), 6–25.[2] G. H. Golub, J. H. Welsch, Calculation of Gauss quadrature rules, Mathematics of Com-putation, 23(106) (1969), 221–230.[3] C. F. Dunkl, Y. Xu, Orthogonal polynomials of several variables, Encyclopedia of Mathe-matics and its Applications, vol. 81, Cambridge University Press, Cambridge, UK (2001)[4] B. A. Flury, Principal points, Biometrika, 77(1) (1990), 33–41.[5] S.P. Stuart Lloyd, Least squares quantization in PCM, IEEE Transations on InformationTheory, 28(2) (1982), 129–137.[6] Z. Drezner, D. Zerom, A simple and effective discretization of a continuous random variable,Communications in Statistics - Simulation and Computation, 45(10) (2016), 3798–3810.[7] J. Kennan, A Note on Discrete Approximations of Continuous Distributions. Departmentof Economics, University of Wisconsin, 1180 Observatory Drive, Madison, WI, 2006.4

Choosing among different discrete approximations of a continuous random variable

https://air.unimi.it/bitstream/2434/1085328/4/Abstract_ICMASC-watermark.pdf

Choosing among different discrete approximations of a continuous random variable

Abstract

Similar works

Full text

Available Versions

AIR Universita degli studi di Milano