8 pages, 2 figures, to be published in the conference proceedings of 11th international conference "Computer Data Analysis & Modeling 2016" http://www.ajs.or.at/index.php/ajs/about/editorialPolicies#openAccessPolicyPeer reviewedPublisher PD

Dürre, Alexander

Fried, Roland

Vogel, Daniel

Austrian Journal of Statistics

English

arXiv

Durre, A.

Fried, R.

Vogel, D.

BSU Digital Library

The Spatial Sign Covariance Matrix and its Application for Robust Correlation Estimation

Электронная библиотека БГУ

Research Archive

arXiv:1606.02274v1  [stat.ME]  7 Jun 2016The spatial sign covariance matrix and itsapplication for robust correlation estimationA. Du¨rre1, R. FriedFakulta¨t Statistik, Technische Universita¨t Dortmund44221 Dortmund, GermanyD. VogelInstitute for Complex Systems and Mathematical Biology, University of AberdeenAberdeen AB24 3UE, United KingdomAbstractWe summarize properties of the spatial sign covariance matrix and especiallylook at the relationship between its eigenvalues and those of the shape matrixof an elliptical distribution. The explicit relationship known in the bivariatecase was used to construct the spatial sign correlation coefficient, which is anon-parametric and robust estimator for the correlation coefficient within theelliptical model. We consider a multivariate generalization, which we call themultivariate spatial sign correlation matrix.1 IntroductionLet X1, . . . ,Xn denote a sample of independent p dimensional random variables froma distribution F and s : Rp → Rp with s(x) = x/|x| for x 6= 0 and s(0) = 0 the spatialsign, thenSn(tn,X1, . . . ,Xn) =1nn∑i=1s(Xi − tn)s(Xi − tn)Tdenotes the empirical spatial sign covariance matrix (SSCM) with location tn. Thecanonical choice for the location estimator tn is the spatial medianµn = argminµ∈Rpn∑i=1||Xi − µ||.Beside its nice robustness properties like an asymptotic breakdown-point of 1/2, ithas (under regularity conditions, see [12]) the advantageous feature that it centres thespatial signs, i.e.,1nn∑i=1s(Xi − µn) = 0,so that Sn(µn,X1, . . . ,Xn) is indeed the empirical covariance matrix of the spatialsigns of the data. If tn is (strongly) consistent for a location t ∈ R, it was shown1corresponding author, e-mail: alexander.duerre@udo.edu1in [5] that under mild conditions on F the empirical SSCM is a (strongly) consistentestimator for its population counterpartS(X) = E(s(X− t)s(X− t)T ).There are some nice results if F is within the class of continuous elliptical distributions,which means that F possesses a density of the formf(x) = det(V )−12 g((x− µ)V −1(x− µ))for a location µ ∈ Rp, a symmetric and positive definite shape matrix V ∈ Rp×pand a function g : R → R, which is often called the elliptical generator. Prominentmembers of the elliptical family are the multivariate normal distribution and ellipticalt-distributions (e.g. [2], p. 208). If second moments exists, then µ is the expectationof X ∼ F , and V a multiple of the covariance matrix. The shape matrix V is uniqueonly up to a multiplicative constant. In the following, we consider the trace-normalizedshape matrix V0 = V/tr(V ), which is convenient since S(X) also has trace 1. If F iselliptical, then S(X) and V share the same eigenvectors and the respective eigenvalueshave the same ordering. For this reason, the SSCM has been proposed for robust prin-cipal component analysis (e.g. [13, 15]). In the present article, we study the eigenvaluesof the SSCM.2 Eigenvalues of the SSCMLet λ1 ≥ . . . ≥ λp ≥ 0 denote the eigenvalues of V0 and δ1 ≥ . . . ≥ δp ≥ 0 those ofS(X). Explicit formulae that relate the δi to the λi are only known for p = 2 (see[19, 3]), namelyδi =√λi√λ1 +√λ2, i = 1, 2. (1)Assuming λ2 > 0, we have δ1/δ2 =√λ1/λ2 ≤ λ1/λ2, thus the eigenvalues of the SSCMare closer together than those of the corresponding shape matrix. It is shown in [8]that this holds true for arbitrary p > 2, soλi/λj ≥ δi/δj for 1 ≤ i < j ≤ p (2)as long as λj > 0. There is no explicit map between the eigenvalues known for p > 2.Du¨rre et al. [8] give a representation of δi as one-dimensional integral, which permitsfast and accurate numerical evaluations for arbitrary p,δi =λi2∫∞01(1 + λix)∏pj=1(1 + λjx)12dx, i = 1, . . . , p. (3)We use this formula (implemented in R [17] in the package sscor [9]) to get an impressionhow the eigenvalues of S(X) look like in comparison to those of V0. We first look at ofequidistantly spaced eigenvaluesλi =2ip(p+ 1), i = 1, . . . , p,2●●●0.20 0.30 0.40 0.500.200.300.400.50Eigenvalues of V0Eigenvalues of S(X)●●●●●●●●●●●0.05 0.10 0.150.050.100.15Eigenvalues of V0Eigenvalues of S(X)●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●0.000 0.005 0.010 0.0150.0000.0050.0100.015Eigenvalues of V0Eigenvalues of S(X)Figure 1: Eigenvalues of the SSCM wrt the corresponding eigenvalues of the shapematrix in the equidistant setting p = 3 (left), p = 11 (centre) and p = 101 (right).●●●0.1 0.3 0.5 0.70.10.30.50.7Eigenvalues of V0Eigenvalues of S(X)●●●●●●●●●●●0.0 0.1 0.2 0.3 0.40.00.10.20.30.4Eigenvalues of V0Eigenvalues of S(X)●●●●●●●●●●●●●●●●0.00 0.02 0.04 0.06 0.080.000.020.040.060.08Eigenvalues of V0Eigenvalues of S(X)Figure 2: Eigenvalues of the SSCM wrt the corresponding eigenvalues of shape matrixin the setting of one large eigenvalue for p = 3 (left), p = 11 (centre) and p = 101(right).for different p = 3, 11, 101. The magnitude of the eigenvalues necessarily decreasesas p increases, since∑pi=1 λi =∑pi=1 δi = 1 per definition of V0 and S(X). As one cansee in Figure 1, the eigenvalues of S(X) and V0 approach each other for increasing p.In fact the maximal absolute difference for p = 101 is roughly 2 · 10−4. In the secondscenario, we take p − 1 equidistantly spaced eigenvalues and one eigenvalue 5 timeslarger than the rest, i.e.,λi =ip((p+1)/2+5)−5i = 1, . . . , p− 1,5(p−1)p((p+1)/2+5)−5i = p.This models the case where the dependence is mainly driven by one principle compo-nent. As one can see in Figure 2, the distance between the two largest eigenvalues issmaller for S(X) than for V0. This is not surprising in light of (2). Thus in general, theeigenvalues of the SSCM are less separated than those of V0, which is one reason why3the use of the SSCM for robust principal component analysis has been questioned (e.g.[1, 14]). However, the differences appear to be generally small in higher dimensions.3 Estimation of the correlation matrixEquation (1) can be used to derive an estimator for the correlation coefficient basedon the empirical SSCM: the spatial sign correlation coefficient ρn ([6]). Under mildregularity assumptions this estimator is consistent under elliptical distributions andasymptotically normal with varianceASV(ρn) = (1− ρ2)2 + 12(a+ a−1)(1− ρ2)3/2, (4)where a =√v11/v22 is the ratio of the marginal scales and ρ = v12/√v11v22 is thegeneralized correlation coefficient, which coincides with the usual moment correlationcoefficient if second moments exists. Equation (4) indicates that the variance of ρn isminimal for a = 1, but can get arbitrarily large if a tends to infinity or 0.Therefore a two-step procedure has been proposed, the two-stage spatial sign cor-relation ρσ,n, which first normalizes the data by a robust scale estimator, e.g., themedian absolute deviation (mad), and then computes the spatial sign correlation ofthe transformed data. Under mild conditions (see [7]), this two-step procedure yieldsan asymptotic variance ofASV(ρσ,n) = (1− ρ2)2 + (1− ρ2)3/2, (5)which equals that of ρn for the favourable case of a = 1. Since (5) only depends onthe parameter ρ, the two-stage spatial sign correlation coefficient is very suitable toconstruct robust and non-parametric confidence intervals for the correlation coefficientunder ellipticity. It turns out that these intervals are quite accurate even for rathersmall sample sizes of n = 10 and in fact more accurate then those based on the samplemoment correlation coefficient [7].One can construct an estimator of the correlation matrix R by filling the off-diagonalpositions of the matrix estimate with the bivariate spatial sign correlation coefficientsof all pairs of variables. This was proposed in [6]. Equation (3) allows an alternativeapproach: First standardize the data by a robust scale estimator and compute theSSCM of the transformed data. Then apply a singular value decompositionSn(tn,X1, . . . ,Xn) = Uˆ∆ˆUˆT ,where ∆ˆ contains the ordered eigenvalues δˆ1 ≥ . . . ≥ δˆp. One obtains estimatesλˆ1, . . . , λˆp by inverting (3). Although theoretical results are yet to be established,4we found in our simulations that the following fix point algorithmλˆ(0)i = δi, i = 1, . . . , p,λ˜(k+1)i = 2δˆi(∫∞01(1 + λˆ(k)i x)∏pj=1(1 + λˆ(k)j x)12dx,)−1, i = 1, . . . , p, k = 1, 2, . . .λˆ(k+1)i = λ˜(k+1)i(p∑j=1λ˜j(k+1))−1, i = 1, . . . , p, k = 1, 2, . . .works reliably and converges fast. Let Λˆ denote the diagonal matrix containing λˆ1, . . . , λˆp,then Vˆ = Uˆ ΛˆUˆT is a suitable estimator for for the shape of the standardized data andRˆ with rˆij = vˆij/√vˆiivˆjj an estimator for the correlation matrix, which we call themultivariate spatial sign correlation matrix. Contrary to the pairwise approach, themultivariate spatial sign correlation matrix is positive semi-definite by construction.Theoretical properties of the new estimator are not straightforward to establish. Bya small simulation study we want to get an impression of its efficiency. We compare thevariances of the moment correlation, the pairwise as well as the multivariate spatial signcorrelation under several elliptical distributions: normal, Laplace and t distributionswith 5 and 10 degrees of freedom. The latter three generate heavier tails than thenormal distribution. The Laplace distribution is obtained by the elliptical generatorg(x) = cp exp(−√|x|/2), where cp is the appropriate integration constant dependingon p (e.g. [2], p. 209).We take the identity matrix as shape matrix and compare the variances of an off-diagonal element of the matrix estimates for different dimensions p = 2, 3, 5, 10, 50and sample sizes n = 100, 1000. We use the R packages mvtnorm [10] and MNM [16]for the data generation. The results based on 10000 runs are summarized in Table 1.Except for the moment correlation at the t5 distribution, the results for n = 100 andn = 1000 are very similar. Note that the variance of the moment correlation decreasesat the Laplace distribution as the dimension p increases, but not so for the otherdistributions considered. The lower dimensional marginals of the Laplace distributionare, contrary to the normal and the t-distributions, not Laplace distributed (see [11]),and the kurtosis of the one-dimensional marginals of the Laplace distribution in factdecreases as p increases.Equation (5) yields an asymptotic variance of 2 for the pairwise spatial sign corre-lation matrix elements regardless of the specific elliptical generator, which can also beobserved in the simulation results. The moment correlation is twice as efficient undernormality, but has a higher variance at heavy tailed distributions. For uncorrelated t5distributed random variables, the spatial sign correlation outperforms the moment cor-relation. Looking at the multivariate spatial sign correlation, we see a strong increaseof efficiency for larger p. For p = 50 the variance is comparable to that of the momentcorrelation. Since the asymptotic variance of the SSCM does not depend on the ellipti-cal generator, this is expected to also hold for the multivariate spatial sign correlation,and we find this confirmed by the simulations. The multivariate spatial sign correla-tion is more efficient than the moment correlation even under slightly heavier tails formoderately large p.5n 100 1000p 2 3 5 10 50 2 3 5 10 50Ncor 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0sscor pairwise 1.9 1.9 1.9 1.9 1.9 2.0 2.0 2.0 2.0 2.0sscor multivariate 1.9 1.6 1.4 1.2 1.0 2.0 1.7 1.4 1.2 1.0t10cor 1.3 1.3 1.3 1.3 1.3 1.3 1.3 1.3 1.4 1.3sscor pairwise 2.0 1.9 1.9 2.0 1.9 2.0 2.0 2.0 2.0 2.0sscor multivariate 2.0 1.7 1.3 1.2 1.0 2.0 1.7 1.4 1.2 1.0t5cor 2.0 2.1 2.1 2.1 2.1 2.6 2.6 2.6 2.6 2.6sscor pairwise 2.0 2.0 1.9 2.0 1.9 2.1 2.0 2.0 2.0 2.0sscor multivariate 2.0 1.7 1.4 1.2 1.1 2.1 1.7 1.4 1.2 1.0Lcor 1.6 1.5 1.3 1.2 1.1 1.6 1.5 1.3 1.2 1.1sscor pairwise 1.9 1.9 1.9 2.0 2.0 2.0 2.0 2.0 2.0 2.0sscor multivariate 1.9 1.6 1.4 1.2 1.1 2.0 1.7 1.4 1.2 1.1Table 1: Simulated variances (multiplied by√n) of one off-diagonal element of thecorrelation matrix estimate based on the moment correlation (cor), the pairwise spatialsign correlation (sscor pairwise) and the multivariate spatial sign correlation matrix(sscor multivariate) for spherical normal (N), t5, t10, and Laplace (L) distribution,several dimensions p and sample sizes n = 100, 1000.An increase of efficiency for larger p is not uncommon for robust scatter estimators.It can be observed amongst others forM-estimators, the Tyler shape matrix, the MCD,and S-estimators (e.g. [4, 18]). All of these are affine equivariant estimators, requiringn > p. This is not necessary for the spatial sign correlation matrix. One may expectthat the efficiency gain for large p is at the expense of robustness, in particular alarger maximum bias curve. Further research will be necessary to thoroughly explorethe robustness properties and efficiency of the multivariate spatial sign correlationestimator.References[1] Bali J.L., Boente G., Tyler D.E., Wang J.L. (2011). Robust functional principalcomponents: A projection-pursuit approach. The Annals of Statistics. Vol. 39,pp. 2852-2882.[2] Bilodeau M., Brenner D. (1999). Theory of Multivariate Statistics. Springer, NewYork.[3] Croux C., Dehon C., Yadine, A. (2010). The k-step spatial sign covariance matrix.Advances in data analysis and classification. Vol. 4, pp. 137-150.6[4] Croux C., Haesbroeck G. (1999). Influence function and efficiency of the minimumcovariance determinant scatter matrix estimator. Journal of Multivariate Analysis.Vol. 71, pp. 161-190.[5] Du¨rre A., Vogel D., Tyler D.E. (2014). The spatial sign covariance matrix withunknown location. Journal of Multivariate Analysis. Vol. 130, pp. 107-117.[6] Du¨rre A., Vogel D., Fried R. (2015). Spatial sign correlation. Journal of Multivari-ate Analysis. Vol. 135, pp. 89-105.[7] Du¨rre A., Vogel, D. (2016). Asymptotics of the two-stage spatial sign correlation.Journal of Multivariate Analysis. Vol. 144, pp. 54-67.[8] Du¨rre A., Tyler D.E., Vogel, D. (2016). On the eigenvalues of the spatial signcovariance matrix in more than two dimensions. Statistics & Probability Letters.Vol. 111, pp. 80-85.[9] Du¨rre A., Vogel D. (2016). sscor: Robust Correlation Estimation and TestingBased on Spatial Signs. R package version 0.2.[10] Genz A, Bretz F., Miwa T., Mi X., Leisch F., Scheipl F., Bornkamp B., Maech-ler M., Hothorn T. (2016), mvtnorm: Multivariate Normal and t Distributions. Rpackage version 1.0.5.[11] Kano Y. (1994). Consistency property of elliptic probability density functions,Journal of Multivariate Analysis. Vol. 51, pp. 139-147.[12] Kemperman J. H. B. (1987). The median of a finite measure on a Banach space.Statistical Data Analysis Based on the L1-Norm and Related Methods. pp. 217-230.[13] Locantore N., Marron J.S., Simpson D.G., Tripoli N., Zhang J.T., Co-hen K.L. (1999). Robust principal component analysis for functional data. Test.Vol. 8, pp. 1-73.[14] Magyar A.F., Tyler D.E. (2014). The asymptotic inadmissibility of the spatial signcovariance matrix for elliptically symmetric distributions. Biometrika. Vol. 101,pp. 673-688.[15] Marden, J.I. (1999). Some robust estimates of principal components. Statistics &probability letters. Vol. 43, pp. 349-359.[16] Nordhausen K., Oja H. (2011), Multivariate L1 methods: the package MNM.Journal of Statistical Software. Vol. 43, pp. 1-28.[17] R Development Core Team (2016). R: A Language and Environment for StatisticalComputing.7[18] Taskinen S., Croux C., Kankainen A., Ollila E., Oja H. (2006). Influence functionsand efficiencies of the canonical correlation and vector estimates based on scatterand shape matrices. Journal of Multivariate Analysis. Vol. 97, pp. 359-384.[19] Vogel D., Ko¨llmann C., Fried R. (2008). Partial correlation estimates based onsigns. Proceedings of the 1st Workshop on Information Theoretic Methods in Sci-ence and Engineering. Vol. 43, pp. 1-6.8

The spatial sign covariance matrix and its application for robust correlation estimation

https://aura.abdn.ac.uk/bitstream/2164/6150/3/667_1962_2_PB.pdf

The spatial sign covariance matrix and its application for robust correlation estimation

Abstract

Similar works

Full text

Available Versions

BSU Digital Library

Электронная библиотека БГУ

Research Archive