Kurdyaeva, Tamara

Milias Argeitis, Andreas

English

In this note, we present a complete analytic calculation of variance-based sensitivity indices using GP regression

ARTS repository - University of Groningen

   University of GroningenAnalytical calculation of Sobol sensitivity indices for Gaussian Processes with a squaredexponential covariance functionMilias Argeitis, Andreas; Kurdyaeva, TamaraIMPORTANT NOTE: You are advised to consult the publisher's version (publisher's PDF) if you wish to cite fromit. Please check the document version below.Publication date:2018Link to publication in University of Groningen/UMCG research databaseCitation for published version (APA):Milias Argeitis, A., & Kurdyaeva, T. (2018). Analytical calculation of Sobol sensitivity indices for GaussianProcesses with a squared exponential covariance function.CopyrightOther than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of theauthor(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons).The publication may also be distributed here under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license.More information can be found on the University of Groningen website: https://www.rug.nl/library/open-access/self-archiving-pure/taverne-amendment.Take-down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediatelyand investigate your claim.Downloaded from the University of Groningen/UMCG research database (Pure): http://www.rug.nl/research/portal. For technical reasons thenumber of authors shown on this cover page is limited to 10 maximum.Download date: 21-06-2022Analytical calculation of Sobol sensitivity indices for Gaussian Processeswith a squared exponential covariance functionNotation and modeling assumptions• Column vector of random inputs: X = [X1...Xd]T ∈ Rd, X′ = [X′1...X′d]T ∈ Rd,• Column subvector of X without Xs element: X−s = [X1 . . . Xs−1 Xs+1 . . . Xd]T ∈ Rd−1• Column vector consists of elements X′−s and Xs: X̂ = [X′1 . . . Xs . . . X′d]T ∈ Rd• Random inputs Xs, s = 1, ..., d are mutually independent and each input follows a standard uniformdistribution: Xs ∼ U(0, 1), s = 1, ..., d• Function F represents a black-box model of interest: Y = F(X), where Y is the model output• Observed n training data points are denoted by X̃ = (̃x(1), . . . , x̃(n))• Test n∗ data points are denoted by X∗ =(x(1)∗ , . . . , x(n∗)∗)• Vector of function F values at n training points X̃: ỹ =[̃y(1), . . . , ỹ(n)]T, ỹ(i) = F(̃x(i)), i = 1, ..., n• Gaussian process (GP) parametrization f (x) ∼ GP(0, k(x, x′))k(x, x′) = σ2fd∏j=1exp− (x j − x′j)22 l2j = ∗ ∣∣∣∣∣h j = 12 l−2j∣∣∣∣∣= σ2fd∏j=1exp(−h j(x j − x′j)2)(1)• Posterior GP distribution f∗|X̃, ỹ, X∗ ∼ N(m(X∗), v(X∗)), wherem(X∗) =K(X∗, X̃)[K(X̃, X̃) + σ2I]−1ỹ, (2)v(X∗) =K(X∗, X∗) − K(X∗, X̃)[K(X̃, X̃) + σ2I]−1K(X̃, X∗), (3)where the additive term σ2I reflects the fact that the function values are corrupted by additive noisewith variance σ2. Even if σ2 = 0 (as is the case when the function is exactly evaluated), a smallnoise term is typically inserted to regularize the inversion of the covariance matrix K(X̃, X̃).• K = K(X̃, X̃)• kpq for p = 1, ..., n and q = 1, ..., n denotes the entries of the matrix[K + σ2 I]−1• wp, p = 1, ..., n are the elements of the vector[K + σ2 I]−1ỹ• Expectations and variances with respect to the GP posterior distribution are denoted as E∗ andVar∗, respectively• erf(·) denotes the error function.1Sobol indicesThe estimate of the Sobol indices for the output Y with respect to the input XsŜ s =E∗{Var{E (Y |Xs)}}E∗{Var(Y)}, s = 1, ..., d. (4)Inference for variance in numerator E∗{Var{E (Y |Xs)}}Let us first calculate the numerator in the formula for estimates of Sobol indices (4).E∗{Var{E(Y |Xs)}} = E∗{E (E(Y |Xs))2} − E∗{E(Y)}2 == E∗{E (E(Y |Xs))2} − Var∗{E(Y)} − (E∗{E(Y)})2 == I − II − III (5)First component I from (5): E∗{E (E(Y |Xs))2}We derive the formulas for every component of (5) separately. The first term I requires the computationof the following integrals:E∗{E (E(Y |Xs))2} = E[Var∗{E(Y |Xs)} + (E∗{E(Y |Xs)})2]==1∫0∫[0,1]d−1∫[0,1]d−1v(x, x̂) dx−sdx′−sdxs +1∫0∫[0,1]d−1∫[0,1]d−1m∗(x)m∗(x̂)dx−sdx′−sdxs == L + M. (6)For the simplification we also provide the derivation of the formulas for the individual terms L and M in(6) separately.L =1∫0∫[0,1]d−1∫[0,1]d−1v(x, x̂) dx−sdx′−sdxs3==1∫0∫[0,1]d−1∫[0,1]d−1K(x, x̂) dx−sdx′−sdxs −1∫0∫[0,1]d−1∫[0,1]d−1K(x, X̃)(K + σ2)−1K(X̃, x̂) dx−sdx′−sdxs ==1∫0Z−s(x̂)dxs −1∫0[zT−s(x)(K + σ2)−1z−s(x̂)]dxs (7)Now we divide the calculation of the integrals into several parts to make it easier to followz−s(x) =∫[0,1]d−1K(x, X̃) dx−s(1) =σ2f exp(−hs(xs − x̃s)2) ∏j=−s1∫0exp(−h j(x j − x̃ j)2)dx j == σ2f exp(−hs(xs − x̃s)2) ∏j=−sI j (8)2Next we evaluate the integral defined as I j in (8)I j =1∫0exp(−h j(x j − x̃ j)2)dx j = ∗∣∣∣∣∣a j = √h j(x j − x̃ j)∣∣∣∣∣ = 1√h j√h j(1−x̃ j)∫−√h j x̃ jexp(−a2j)da j ==√π21√h j(erf(√h j(1 − x̃ j))+ erf(√h j x̃ j))=√π2l jerf  1√2l j(1 − x̃ j) + erf  1√2l jx̃ j(9)We can substitute the exact formula of I j (9) into the final formula of z−s in (8)z−s(x) = σ2f exp(−12l2s(xs − x̃s)2) ∏j=−s√π2l jerf  1√2l j(1 − x̃ j) + erf  1√2l jx̃ j (10)Finally we calculate the first double integral in (7) which was marked as Z−sZ−s =∫[0,1]d−1∫[0,1]d−1K(x, x̂)dx−sdx′−s = σ2f∏j=−s1∫01∫0exp(−h j(x j − x′j)2)dx j dx′j = σ2f ∏j=−s1∫0I jdx′j == σ2f∏j=−s√π2l j1∫0erf  1√2l j(1 − x′j) + erf  1√2l jx′j dx′j = σ2f ∏j=−sJ j (11)We present the exact analytic calculation of the J j integral in (11)J j =√π2l j1∫0erf 1√2l j(1 − x′j) dx′j + 1∫0erf 1√2l jx′j dx′j == ∗∣∣∣∣∣∣∣b j = 1√2l j (1 − x′j), c j = 1√2l j x′j∣∣∣∣∣∣∣ ==√π2l j−√2l j0∫1/√2l jerf(b j)db j +√2l j1/√2l j∫0erf(c j)dc j == 2√πl2j1/√2l j∫0erf(t j)dt j == ∗∣∣∣∣∣∣u = erf(t j), dv = dt j, du = 2√π exp(−t2j )dt j, v = t j∣∣∣∣∣∣ == 2√πl2jerf(t j)t j∣∣∣∣1/√2l j0−2√π1/√2l j∫0exp(−t2j)t jdt j == 2√πl2j 1√2l jerf 1√2l j + 1√πexp(−t2j) ∣∣∣∣1/√2l j0 == 2√π2l jerf 1√2l j + 2l2jexp− 12l2j − 1 (12)3Then we can write down a closed form to compute Z−s in (11)Z−s = σ2f∏j=−s2√π2l jerf 1√2l j + 2l2jexp− 12l2j − 1 (13)Finally, the analytic formula for L in (7) is simplified to the following formL = σ2f∏j=−sJ j − σ4fn∑p=1n∑q=1kpq ∏j=−sI j(xp)∏j=−sI j(xq)1∫0exp(−hs[(xs − x̃ps )2 + (xs − x̃qs)2])dxs == σ2f∏j=−sJ j − σ4fn∑p=1n∑q=1kpq ∏j=−sI j(xp)∏j=−sI j(xq) Rpqs (14)The derivation of the integral Rpqs is presented separately:Rpqs =1∫0exp(−hs[(xs − x̃ps )2 + (xs − x̃qs)2])dxs == ∗∣∣∣∣∣a2 + b2 = 12 [(a + b)2 + (a − b)2]∣∣∣∣∣ == exp[−hs2(x̃ps − x̃qs)2] 1∫0exp[−hs2(2xs − x̃ps − x̃qs)2]dxs == ∗∣∣∣∣∣∣∣gs =√hs2(2xs − x̃ps − x̃qs), dgs =√2hsdxs, gs(0) = −√hs2(x̃ps + x̃qs), gs(1) =√hs2(2 − x̃ps − x̃qs)∣∣∣∣∣∣∣ == exp[−hs2(x̃ps − x̃qs)2] 1√2hsgs(1)∫gs(0)exp(−g2s)dgs == exp[−hs2(x̃ps − x̃qs)2] 1√2hs√π2(erf[gs(1)]− erf[gs(0)])==√π2ls exp[−14l2s(x̃ps − x̃qs)2] (erf[12ls(2 − x̃ps − x̃qs)]+ erf[12ls(x̃ps + x̃qs)])(15)This concludes the evaluation of the integral L in (14) with the exact analytic form for the componentsI j(x) in (9), J j in (12) and Rpqs in (15).On the next step to finalize the calculation of (6), we evaluate the second part M in the equation.M =1∫0∫[0,1]d−1∫[0,1]d−1m∗(x)m∗(x̂)dx−sdx′−sdxs ==1∫0[zT−s(x)(K + σ2)−1ỹ zT−s(x̂)(K + σ2)−1ỹ]dxs == σ4fn∑p=1n∑q=1wp wq ∏j=−sI j(xp)∏j=−sI j(xq)1∫0exp(−hs[(xs − x̃ps )2 + (xs − x̃qs)2])dxs == σ4fn∑p=1n∑q=1wp wq ∏j=−sI j(xp)∏j=−sI j(xq)Rpqs (16)4Second component II from (5): Var∗{E(Y)}Let us return to the equation in (5) and focus on the second term IIVar∗{E(Y)} =∫[0,1]d∫[0,1]dk∗(x, x′) dxdx′ ==∫[0,1]d∫[0,1]dK(x, x′)dxdx′ −∫[0,1]d∫[0,1]dK(x, X̃)(K + σ2)−1K(X̃, x′) dxdx′ == Z − zT(K + σ2)−1z (17)Again we recast the calculation in more simple steps and use the formulas that was presented previously.For instance, the computation of the integral z in (17) is reduced to the evaluation of I j from the equation(9).z(x) =∫[0,1]dK(x, X̃)dx = σ2fd∏j=1I j(x) (18)Analogously, to compute Z we address to the integral J j that was calculated in (12)Z =∫[0,1]d∫[0,1]dK(x, x′)dxdx′ = σ2fd∏j=1J j (19)Third component III from (5): E∗{E(Y)}We address now to the calculation of the final third component in (5).E∗{E(Y)} =∫[0,1]dK(x, X̃)dx [K + σ2I]−1 ỹ = zT (K(X̃, X̃) + σ2)−1 ỹ (18)= σ2fd∏j=1I j(K(X̃, X̃) + σ2)−1ỹ(20)This derivation finalize the computation of the numerator from the formula (4), all three components,I, IIand III , from (5) have been derived in analytic form.Inference for variance in denominator E∗{Var(Y)}The final step in the calculation of the estimate for the Sobol indices is the derivation of the analyticformula for the denominator in (4).E∗{var(Y)} = E∗{E(Y2)} − E∗{E(Y)}2 = E{E∗(Y2)} −(Var∗{E(Y)} + (E∗{E(Y)})2)== E(Var∗{Y} + {E∗(Y)}2)−(Var∗{E(Y)} + (E∗{E(Y)})2)= IV − II − III (21)Similar to the previous routine for the numerator in (4), we divide the calculations in several steps. Thereare three components II, III and IV in (21), where II is summarized by formulas (17), (18) and (19),while III can be calculated using (20). The derivation of the fourth component IV is presented in thenext section.5Fourth component IV from (21): E∗{E(Y)}Here we perform the last calculation of the fourth component in (21).E{Var∗{Y}} + E{E∗(Y)}2 =∫[0,1]dK(x, x)dx −∫[0,1]dK(x, X̃)[K + σ2 ]−1K(X̃, x) dx++∫[0,1]d[K(x, X̃)(K + σ2)−1ỹ K(x, X̃)[K + σ2 ]−1ỹ dx]== σ2f − σ4fn∑p=1n∑q=1kpq d∏s=1Rpqs + σ4f n∑p=1n∑q=1wp wq d∏s=1Rpqs , (22)where the derivation for Rpqs is shown in (15).6Final formulas for Sobol indicesHere we present the final formulas for Sobol indices’ estimates Ŝ s defined as:Ŝ s =E∗{Var{E (Y |Xs)}}E∗{Var(Y)}, s = 1, ..., d. (4)We start with the analytic expression for the numerator E∗{Var{E (Y |Xs)}} in 4.E∗{Var{E(Y |Xs)}} = E∗{E (E(Y |Xs))2} − Var∗{E(Y)} − (E∗{E(Y)})2 = I − II − III. (5)For the first term I in 5, we getI : E∗{E (E(Y |Xs))2} = σ2f∏j=−sJ j − σ4fn∑p=1n∑q=1(kpq − wp wq)Rpqs ∏j=−sI j(x̃(p))I j(x̃(q)) .whereJ j = 2 l j√π2erf 1√2l j + l jexp− 12l2j − 1 ,Rpqs =√π2ls exp(−14l2s(x̃(p)s − x̃(q)s)2) (erf[12ls(2 − x̃(p)s − x̃(q)s)]+ erf[12ls(x̃(p)s + x̃(q)s)]),andI j(x̃(p))=√π2l jerf  1√2l j(1 − x̃(p)j) + erf  1√2l jx̃(p)j .The second term II in 5 can then be written asII : Var∗{E(Y)} = σ2fd∏j=1J j − σ4f d∏j=1I j(X̃)′ [K + σ2f I]−1  d∏j=1I j(X̃) .Finally, for the third III term of 5 we getIII : (E∗{E(Y)})2 =σ2f d∏j=1I j(K(X̃, X̃) + σ2)−1ỹ2The expression for the denominator E∗{Var(Y)} of 4 has the following form:E∗{Var(Y)} = E∗{E(Y2)} − Var∗{E(Y)} − (E∗{E(Y)})2 = IV − II − III. (21)As we can see, the formula 21 contains one further term compared to the expression of the numerator5, which is given byIV : E∗{E(Y2)} = σ2f − σ4fn∑p=1n∑q=1kpq d∏s=1Rpqs + σ4f n∑p=1n∑q=1wp wq d∏s=1Rpqs .7

Analytical calculation of Sobol sensitivity indices for Gaussian Processes with a squared exponential covariance function

https://pure.rug.nl/ws/files/65061085/sobols_indices_v2.pdf

Analytical calculation of Sobol sensitivity indices for Gaussian Processes with a squared exponential covariance function

Abstract

Similar works

Full text

Available Versions

ARTS repository - University of Groningen