In this note we develop an extension of the Mar\v{c}enko-Pastur theorem to
time series model with temporal correlations. The limiting spectral
distribution (LSD) of the sample covariance matrix is characterised by an
explicit equation for its Stieltjes transform depending on the spectral density
of the time series. A numerical algorithm is then given to compute the density
functions of these LSD's

Grenander

Jianfeng Yao

Jonsson

Marčenko

Silverstein

Wachter

English

arXiv

In this note we develop an extension of the Marčenko-Pastur theorem to time series model with temporal correlations. The limiting spectral distribution (LSD) of the sample covariance matrix is characterised by an explicit equation for its Stieltjes transform depending on the spectral density of the time series. A numerical algorithm is then given to compute the density functions of these LSD's. © 2011 Elsevier B.V.postprin

Yao, J

HKU Scholars Hub

Title A note on a Marenko-Pastur type theorem for time seriesAuthor(s) Yao, JCitation Statistics And Probability Letters, 2012, v. 82 n. 1, p. 22-28Issued Date 2012URL http://hdl.handle.net/10722/143792RightsNOTICE: this is the author’s version of a work that was acceptedfor publication in Statistics & Probability Letters. Changesresulting from the publishing process, such as peer review,editing, corrections, structural formatting, and other qualitycontrol mechanisms may not be reflected in this document.Changes may have been made to this work since it wassubmitted for publication. A definitive version was subsequentlypublished in Statistics & Probability Letters, 2012, v. 82 n. 1, p.22-28. DOI: 10.1016/j.spl.2011.08.011A note on a Marcˇenko-Pastur type theoremfor time seriesJianfeng Yao∗Jianfeng YaoDepartment of Statistics and Actuarial ScienceThe University of Hong KongPokfulam, HONG KONGe-mail: jeffyao@hku.hkAbstract: In this note we develop an extension of the Marcˇenko-Pastur the-orem to time series model with temporal correlations. The limiting spectraldistribution (LSD) of the sample covariance matrix is characterised by anexplicit equation for its Stieltjes transform depending on the spectral den-sity of the time series. A numerical algorithm is then given to compute thedensity functions of these LSD’s.AMS 2000 subject classifications: Primary 62H15; secondary 62H10.Keywords and phrases: High-dimensional time series, High-dimensionalsample covariance matrices, Marcˇenko-Pastur distributions.1. IntroductionLet {Xj}, j = 1, . . . , n be a sequence of p-dimensional real-valued random vectorsand consider the associated empirical covariance matrixSn =1nn∑j=1XjX>j . (1)The study of the empirical spectral distribution (ESD) Fn of Sn, i.e. the distri-bution generated by its (real-valued) eigenvalues, goes back to Wishart in 1920’s.A milestone work by Marcˇenko and Pastur (1967) states that if both sample sizen and data dimension p proportionally grow to infinity such that lim p/n = c forsome positive c > 0 and all the coordinates of all the vectors Xj ’s are i.i.d. withmean zero and variance 1, then Fn converges to a nonrandom distribution. Thislimiting spectral distribution (LSD), named after them as the Marcˇenko-Pastur∗The research of the author was supported partly by a HKU Start-up grant.1imsart-generic ver. 2008/08/29 file: lp-ss.tex date: August 16, 2011J. Yao/Marcˇenko-Pastur theorem for time series 2distribution of index c has a density functionf(x) =12picx√(b− x)(x− a), a ≤ x ≤ b, (2)with a = (1−√c)2 and b = (1+√c)2 defining the support interval and has a pointmass 1− 1/c at the origin if c > 1. Further refinements are made successively bymany researchers including Jonsson (1982), Wachter (1978) and Yin (1986).An important work by Silverstein (1995) aimed at relaxing the independencestructure between the coordinates of the Xj ’s and considered random vectors ofform Yj = T1/2p Xj where (Tp) is a sequence of non-negative definite matrices.Assuming that (Tp) is bounded in spectral norm and the sequence of ESD of(Tp) has a weak limit H, he established a (strong) LSD for the sample covari-ance matrix n−1∑YjY>j and provides a characteristic equation for its Stieltjestransform. Despite a big step made by this generalisation, it still does not coverall possible correlation patterns of coordinates. Pursuing these efforts, a recentwork by Bai and Zhou (2008) pushes a step further Silverstein’s result by allow-ing a very general pattern for correlations between the coordinates of the Xj ’ssatisfying a mild moment conditions.In this work, we extend such Marcˇenko-Pastur type theorems along anotherdirection by considering time series observations instead of an i.i.d. sample. Letus first consider an univariate real-valued linear processzt =∞∑k=0φkεt−k, t ∈ Z, (3)where (εk) is a real-valued and weakly stationary white noise with mean zero andvariance 1. The p-dimensional process (Xt) considered in this paper will be madeby p independent copies of the linear process (zt), i.e. for Xt = (X1t, . . . , Xpt)>,Xit =∞∑k=0φkεi,t−k, t ∈ Z, (4)where the p coordinate processes {(ε1,t, . . . , εp,t)} are independent copies of theunivariate error process {εt} in (3). Let X1, . . . ,Xn be the observations of thetime series at time epochs t = 1, . . . , n. Again we are interested in the ESD ofthe sample covariance matrix Sn in (1).The author should mention that a similar problem has been considered in Jinet al. (2009). However we propose much more general results in this note sincefirstly their results are limited to ARMA-type processes instead of a generallinear process considered here and secondly, they do not find a general equationas the one proposed in Theorem 1 below except for two simplest particular casesof AR(1) and MA(1).imsart-generic ver. 2008/08/29 file: lp-ss.tex date: August 16, 2011J. Yao/Marcˇenko-Pastur theorem for time series 32. A Marcˇenko-Pastur type theorem for linear processesRecall that the Stieltjes transform sµ of a probability measure µ on the real lineis a map from the set C+ of complex numbers with positive imaginary part ontoitself and defined bysµ(z) =∫1x− zµ(dx), z ∈ C+.We always employ an usual convention that for any complex number z,√zdenotes its square root with nonnegative imaginary part.Theorem 1. Assume that the following conditions hold:1. The dimensions p→∞, n→∞ and p/n→ c ∈ (0,∞);2. The error process has a fourth moment: Eε4t <∞;3. The linear filter (φk) is absolutely summable, i.e.∞∑k=0|φk| <∞.Then almost surely the ESD of Sn tends to a non-random probability distributionF . Moreover, the Stieltjes transform s = s(z) of F (as a mapping from C+ intoC+) satisfies the equationz = −1s+12pi∫ 2pi01cs+ {2pif(λ)}−1 dλ , (5)where f(λ) is the spectral density of the linear process (zt):f(λ) =12pi∣∣∣∣∣∞∑k=0φkeikλ∣∣∣∣∣2, λ ∈ [0, 2pi). (6)The proof of the theorem is postponed to Section 4. Let us mention thatalthough the case c = 0 is beyond the scope of Theorem 1, Equation (5) leads inthis case to the solution s(z) = 1/(γ0 − z), that is the LSD would be the Diracmass at γ0 = Var(Xit). This conjectures an extension of Theorem 1 to the so-called “very large p and small n” asymptotics where one assumes p→∞, n→∞and p/n → 0. Indeed, in this scenario taking into account that the populationcovariance matrix of Xt equals γ0Ip, one can expect that the sample eigenvaluesof Sn stay close to the population ones (all equal to γ0). Note that such resultsexist for i.i.d. sequence (Xt) with i.i.d. components (see Bai and Yin, 1988).imsart-generic ver. 2008/08/29 file: lp-ss.tex date: August 16, 2011J. Yao/Marcˇenko-Pastur theorem for time series 4Figure 1. Determination of the support [x1, x2] of the LSD. Left panel c > 1 and right panelc < 1.2.1. Support of the LSD FStarting from Eq. 5 and following the techniques devised in Silverstein and Choi(1995), we can describe precisely the support of the LSD F in previous theorem.Let a and b be respectively the minimum and the maximum of the function2pif over [0, 2pi]. As f is infinitely differentiable and positive everywhere, botha and b are attained and the range of 2pif is exactly the interval [a, b]. Wewill always exclude the situation a = 0 since it corresponds to a special classof linear processes, namely non-invertible ARMA processes, see Grenander andSzego¨ (1958, chap. 9), which has no practical interest for applications. Thereforethe map s 7→ z = g(s) in Eq.(5) has a trace for real-valued s providing s /∈[− 1ac ,− 1bc ]. Figure 1 depicts this map for both c < 1 and c > 1 cases.The following proposition is a straightforward application of results fromSilverstein and Choi (1995) and we then omit its proof.Proposition 1. With the map g : s 7→ z = g(s) in Eq.(5) restricted to reals /∈ [− 1ac ,− 1bc ] (Figure 1) the following holds:1. The LSD F has a compact support [x1, x2] ⊂ [0,∞) on which it has acontinuous density function. In case of c > 1, F has an additional pointmass 1− 1/c at the origin.2. When c > 1, the map g has an unique maximum s1 on (−∞,− 1ac) andan unique minimum s2 on (− 1bc , 0) and we have The edges of the supportinterval are given by these local extrema: x1 = g(s1) and x2 = g(s2).3. When c < 1, the map g has an unique maximum s1 on (0,∞) and anunique minimum s2 on (− 1bc , 0). The edges of the support interval (for theabsolutely continuous component) are again given by these local extrema:x1 = g(s1) and x2 = g(s2).imsart-generic ver. 2008/08/29 file: lp-ss.tex date: August 16, 2011J. Yao/Marcˇenko-Pastur theorem for time series 52.2. Application to an ARMA processFor simplicity, we consider the simplest causal ARMA(1,1) process for the coor-dinates:zt = φzt−1 + εt + θεt−1, t ∈ Z,where |φ| < 1 and θ is real. The aim is to find a simplified form of generalequation (5). We have12pif(λ)=∣∣∣∣1− φeiλ1 + θeiλ∣∣∣∣2 ,andI =12pi∫ 2pi01cs+ {2pif(λ)}−1 dλ =12pii∮|ξ|=11cs+∣∣∣1−φξ1+θξ ∣∣∣2dξξ.By a lengthy but elementary calculation of residues detailed in Section 4, we findI =θcsθ − φ{1− (φ+ θ)(1 + φθ)θ(csθ − φ)(α)√α2 − 4}, (7)withα =cs(1 + θ2) + 1 + φ2csθ − φ , (α) = sgn(=α) . (8)Therefore for an ARMA(1,1) process, the general equation (5) reduces toz = −1s+θcsθ − φ −(φ+ θ)(1 + φθ)(csθ − φ)2(α)√α2 − 4 . (9)Let us mention that it is important to have an explicit formula for the integralin (5) to implement numerical algorithms like the one proposed in Section 3 inorder to compute the density function of the LSD F .Case of an AR(1). For this particular case, we have θ = 0 and α = −(cs +1 + φ2)/φ. As =s > 0, (α) = sgn(−φ). It follows that−(φ+ θ)(1 + φθ)(csθ − φ)2(α)√α2 − 4 =sgn(−φ)−φ1√(cs+1+φ2)2φ2− 4=1|φ|1√(cs+1+φ2)2φ2− 4=1√(cs+ 1 + φ2)2 − 4φ2 .Therefore the Stieltjes transform s of the LSD is solution to a simpler equationz = −1s+1√(cs+ 1 + φ2)2 − 4φ2 . (10)imsart-generic ver. 2008/08/29 file: lp-ss.tex date: August 16, 2011J. Yao/Marcˇenko-Pastur theorem for time series 6It is worth noticing that if we further assume φ = 0, this equation reduces toz = −1/s + 1/(cs + 1) which characterises the standard Marcˇenko-Pastur lawwith i.i.d. coordinates. Furthermore, for the determination of the support [x1, x2]of the LSD, we notice that2pif(λ) =1|1− φeiλ|2 ,so that its extrema are a = 1/(1 + |φ|)2 and b = 1/(1− |φ|)2 (see Figure 1).Case of an MA(1). Here we have φ = 0 andα =1θ(1cs+ 1 + θ2).Hence (α) = − sgn(θ) and it is readily checked out that the Stieltjes transformof the LSD is solution to the equationz = −1s+1cs+1c2s21√(1cs + 1 + θ2)2 − 4θ2 . (11)Again if we further assume θ = 0, this equation reduces to the one for the stan-dard Marcˇenko-Pastur law. Furthermore, for the determination of the support[x1, x2] of the LSD, we notice that2pif(λ) = |1 + θeiλ|2,so that its extrema are a = (1− |θ|)2 and b = (1 + |θ|)2 (see Figure 1).3. A numerical method for computing the LSD density functionIn this section we provide a numerical algorithm for the computation of thedensity function h of the LSD defined in Eq.(5) through its Stieltjes transforms. We haves =1−z +A(s(z))withA(s(z)) =12pi∫ 2pi01cs+ {2pif(λ)}−1 dλ .The algorithm we propose is of fixed-point type.imsart-generic ver. 2008/08/29 file: lp-ss.tex date: August 16, 2011J. Yao/Marcˇenko-Pastur theorem for time series 7Algorithm For a given real x, let ε be small enough positive value and setz = x+ iε.Choose an initial value s0(z) = u + iε and iterate for k ≥ 0 the abovemappingsk+1(z) = {−z +A(sk(z))}−1 ,until convergence and let sK(z) be the final value.Define the estimate of the density function h(x) to beĥ(x) =1pi=sK(z) . It is well-known that this iterated map has good contraction properties guar-anteeing the convergence of the algorithm. There are however two issues whichneed a careful consideration. First the integral operator A is usually approxi-mated by a numeric routine and because of a high number of calls to A, theresulting algorithm is slow. In this aspect analytic formula for A when availableare well acknowledged as Eq.(9) in the case of an ARMA(1,1).A second issue is that overall we first need to determine the support interval[x1, x2] of the density function h. This is handled with the help of description ofxj ’s given in Proposition 1.For four ARMA(1,1) models listed in Table 1, we have used this algorithmwith the map A defined in (9) to get the density plots displayed in Figure 2.Table 1ARMA(1,1) models for density plots. Reference support interval for Marcˇenko-Pastur law ofindex c = 0.2 is [0.306, 2.094].Parameters (φ, θ, c) Estimated support [x1, x2](0.4, 0, 0.2) [0.310, 2.875](0.4, 0.2, 0.2) [0.319, 3.737](0.4, 0.6, 0.2) [0.382, 6.186](0.8, 0.2, 0.2) [0.485, 13.66]Compared to the reference standard Marcˇenko-Pastur law with the samedimension to sample ratio c, the above density functions from ARMA modelsshare a similar shape with however a support interval getting larger and largerwith increasing ARMA coefficients φ and θ.imsart-generic ver. 2008/08/29 file: lp-ss.tex date: August 16, 2011J. Yao/Marcˇenko-Pastur theorem for time series 84. ProofsProof of Theorem 1Recall that the p coordinates of the vectors X1, . . . ,Xn are i.i.d. while the tem-poral covariances Cov(Xis, Xit) are by definition those of (zt): for all 1 ≤ i ≤ p,Cov(Xis, Xit) = Cov(zs, zt) = γt−s, 1 ≤ s, t ≤ n .Let f˜ = 2pif . It follows that the covariance matrix Tn of each coordinate process(Xi1, . . . , Xin) equals to the n-th order Toeplitz matrix associated to f˜ :Tn(s, t) = γt−s, 1 ≤ s, t ≤ n ,andf˜(λ) =∞∑k=−∞γkeikλ , λ ∈ [0, 2pi).We are going to apply Theorem 1.1 of Bai and Zhou (2008) for a stronglimit of the ESD of the sample covariance matrix Sn =1nn∑t=1XtX>t . Under theassumptions made, all the conditions of this theorem are satisfied except that weneed to ensure a weak limit for spectral distributions of (Tn).The function f˜ belongs to the Wiener class, i.e. the sequence of its Fouriercoefficients is absolutely summable. Moreover note that f˜ is infinitely differen-tiable, its minimum a and maximum b are attained. According to the fundamentaleigenvalue distribution theorem of Szego¨ for Toeplitz forms, see Grenander andSzego¨ (1958, sect. 5.2), for any function ϕ continuous on [a, b] and denoting theeigenvalues of Tn by σ(n)1 , . . . , σ(n)n , it holds thatlimn→∞1nn∑k=1ϕ(σ(n)k ) =12pi∫ 2pi0ϕ(f˜(λ))dλ .Consequently, the ESD of Tn (i.e. distribution generated by the σ(k)n ’s) weaklyconverges to a nonrandom distribution H with support [a, b] and defined byH(x) =12pi∫ 2pi01{f˜(λ)≤x}dλ , (12)and we have for ϕ as above,∫ ∞0ϕ(x)dH(x) =12pi∫ 2pi0ϕ(f˜(λ))dλ . (13)imsart-generic ver. 2008/08/29 file: lp-ss.tex date: August 16, 2011J. Yao/Marcˇenko-Pastur theorem for time series 9Furthermore, by application of Theorem 1.1 of Bai and Zhou (2008), it holdsthat the ESD of npSn converges almost surely to a nonrandom probability distri-bution whose Stieltjes transform m solves the equationz = − 1m+1c∫x1 +mxdH(x)= − 1m+12pic∫ 2pi01m+ 1/f˜dλ ,where we have used (13) in the last equation. The equation(5) follows by observ-ing the relation s(z) = 1cm(z/c).Proof of Equation (7)The aim is to evaluate the integralI =12pii∮|ξ|=11cs+∣∣∣1−φξ1+θξ ∣∣∣2dξξ.Let m = cs in this computation of residues. We haveI =12pii∮|ξ|=11 + θ2 + θ(ξ + ξ−1)m {1 + θ2 + θ(ξ + ξ−1)}+ {1 + φ2 − φ(ξ + ξ−1)}dξξ=θmθ − φ12pii∮|ξ|=1{1ξ− (φ+ θ)(1 + φθ)θ(mθ − φ)1ξ2 + 1 + αξ}dξ=θmθ − φ{1− (φ+ θ)(1 + φθ)θ(mθ − φ)12pii∮|ξ|=1dξP (ξ)},withP (ξ) = ξ2 + αξ + 1, α =m(1 + θ2) + 1 + φ2mθ − φ .Let ξ1, ξ2 be the roots of P (ξ) = ξ2 + 1 + αξ. Then1P (ξ)=(1ξ − ξ1 −1ξ − ξ2)1ξ1 − ξ2 .As ξ1ξ2 = 1, only one of the two poles is inside the unit circle. It is readily checkedthat if =α > 0, then |ξ1| < |ξ2| and12pii∮|ξ|=1dξP (ξ)=1ξ1 − ξ2 =1√α2 − 4 .imsart-generic ver. 2008/08/29 file: lp-ss.tex date: August 16, 2011J. Yao/Marcˇenko-Pastur theorem for time series 10Otherwise we have |ξ1| > |ξ2| and the integral has an opposite sign. Summarisingboth cases we get12pii∮|ξ|=1dξP (ξ)=(α)√α2 − 4 ,with (α) = sgn(=(α)). Equation (7) is proved.Acknowledgement. The author is grateful to Jack Silverstein for severalinsightful discussions on the problem studied here, particularly for pointing tome the numerical method of Section 3. We also thank a referee for importantcomments on the paper.ReferencesBai, Z., Zhou, W., 2008. Large sample covariance matrices without independencestructures in columns. Statist. Sinica 18 (2), 425–442.Bai, Z. D., Yin, Y. Q., 1988. A convergence to the semicircle law. Ann. Probab.16 (2), 863–875.Grenander, U., Szego¨, G., 1958. Toeplitz forms and their applications. CaliforniaMonographs in Mathematical Sciences. University of California Press, Berkeley.Jin, B., Wang, C., Miao, B., Lo Huang, M., 2009. Limiting spectral distribu-tion of large-dimensional sample covariance matrices generated by varma. J.Multivariate Anal. 100, 2112–2125.Jonsson, D., 1982. Some limit theorems for the eigenvalues of a sample covariancematrix. J. Multivariate Anal. 12 (1), 1–38.Marcˇenko, V., Pastur, L., 1967. Distribution of eigenvalues for some sets of ran-dom matrices. Math. USSR-Sb 1, 457–483.Silverstein, J. W., 1995. Strong convergence of the empirical distribution of eigen-values of large-dimensional random matrices. J. Multivariate Anal. 55 (2), 331–339.Silverstein, J. W., Choi, S.-I., 1995. Analysis of the limiting spectral distributionof large-dimensional random matrices. J. Multivariate Anal. 54 (2), 295–309.Wachter, K. W., 1978. The strong limits of random matrix spectra for samplematrices of independent elements. Ann. Probability 6 (1), 1–18.Yin, Y. Q., 1986. Limiting spectral distribution for a class of random matrices.J. Multivariate Anal. 20 (1), 50–68.imsart-generic ver. 2008/08/29 file: lp-ss.tex date: August 16, 2011J. Yao/Marcˇenko-Pastur theorem for time series 110.0 0.5 1.0 1.5 2.0 2.5 3.00.00.10.20.30.40.50.60.70.8phi=0.4  theta=0  c=0.20.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.00.00.10.20.30.40.50.60.7phi=0.4  theta=0.2  c=0.20 1 2 3 4 5 6 70.000.050.100.150.200.250.300.350.400.45phi=0.4  theta=0.6  c=0.20 2 4 6 8 10 12 140.000.050.100.150.200.250.30phi=0.8  theta=0.2  c=0.2Figure 2. Densities of the LSD from ARMA(1,1) model. Left to right and top to bottom:(φ, θ, c) = (0.4, 0, 0.2), (0.4, 0.2, 0.2), (0.4, 0.6, 0.2), and (0.8, 0.2, 0.2).imsart-generic ver. 2008/08/29 file: lp-ss.tex date: August 16, 2011

A note on a Mar\v{c}enko-Pastur type theorem for time series

Abstract

Similar works

Full text

Available Versions

HKU Scholars Hub

Crossref