Pathwise predictability and predictors for discrete time processes are
studied in deterministic setting. It is suggested to approximate convolution
sums over future times by convolution sums over past time. It is shown that all
band-limited processes are predictable in this sense, as well as high-frequency
processes with zero energy at low frequencies. In addition, a process of mixed
type still can be predicted if an ideal low-pass filter exists for this
process.Comment: 10 pages. arXiv admin note: text overlap with arXiv:0708.034

Dokuchaev

Duren

Knab

Lyman

Marvasti

Nikolai Dokuchaev

Papoulis

Vaidyanathan

Wainstein

English

arXiv

Pathwise predictability and predictors for discrete time processes are studied in deterministic setting. It is suggested to approximate convolution sums over future times by convolution sums over past time. It is shown that all band-limited processes are predictable in this sense, as well as high-frequency processes with zero energy at low frequencies. In addition, a process of mixed type still can be predicted if an ideal low-pass filter exists for this process

Dokuchaev, Nikolai

espace@Curtin

Additional Information NOTICE: This is the author’s version of a work that was accepted for publication in Signal Processing. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Signal Processing, Vol. 92, Issue 10, (2012] DOI: http://dx.doi.org/10.1016/j.sigpro.2012.04.006 Alternative Location http://dx.doi.org/10.1016/j.sigpro.2012.04.006  On predictors for band-limited and high-frequency time series∗Nikolai DokuchaevDepartment of Mathematics and Statistics, Curtin University,GPO Box U1987, Perth, Western Australia, 6845email N.Dokuchaev@curtin.edu.au. Tel.: 61 8 92663144.1st version: December 1,2011. Revised version: March 15, 2012AbstractPathwise predictability and predictors for discrete time processes are studied in deterministicsetting. It is suggested to approximate convolution sums over future times by convolution sums overpast time. It is shown that all band-limited processes are predictable in this sense, as well as high-frequency processes with zero energy at low frequencies. In addition, a process of mixed type stillcan be predicted if an ideal low-pass filter exists for this process.Key words: prediction, spectral methods, z-transform, band-limited processes, low-pass filters, non-parametric forecast.AMS 2000 classification : 42A38, 93E10, 42B30PACS 2008 numbers: 02.30.Mv, 02.30.Nw, 02.30.Yy, 07.05.Mh, 07.05.Kf1 IntroductionWe study pathwise predictability of discrete time processes in deterministic setting. It is well known thatcertain restrictions on frequency distribution can ensure additional opportunities for prediction and in-terpolation of the processes. The classical result is Nyquist-Shannon-Kotelnikov interpolation theoremfor the continuous time band-limited processes. It is also known that optimal prediction error for sta-tionary Gaussian processes is zero for the case of degenerate spectral density. The related results can befound in Wainstein and Zubakov (1962), Knab (1981), Papoulis (1985), Marvasti (1986), Vaidyanathan(1987), Lyman et al (2000, 2001), Dokuchaev (2008,2010).The present paper extends on discrete time setting the approach suggested for continuous time pro-cesses in Dokuchaev (2008). We study a special kind of predictors such that convolution sums over∗Signal Processing 92, iss. 10, pp.2571-2575. In Fast communications section.1future are approximated by convolution sums over past times representing historical observations. Wefound some cases when this approximation can be made uniformly over a wide class of input processes,including all band-limited processes and high-frequency processes. For the processes of mixed type,we found that the similar predictability can be achieved when the model allows a low pass filter thatacts as an ideal low-pass filter for this process. These results can be a useful addition to the existingtheory of band-limited processes. The novelty is that we consider predictability of both high frequentand band-limited processes in a weak sense uniformly over classes of input processes. In addition, wesuggest a new type of predictor. Its kernel is given explicitly in the frequency domain.2 DefinitionsLet D = {z ∈ C : |z| ≤ 1}, Dc = C \D, T = {z ∈ C : |z| = 1},We denote by ℓr the set of all sequences x = {x(t)}∞t=−∞ ⊂ C such that ∥x∥ℓr =(∑∞t=−∞ |x(t)|r)1/r< +∞ for r ∈ [1,∞), ∥x∥ℓ∞ = supt |x(t)| < +∞ for r = +∞.Let ℓ+r be the set of all sequences x ∈ ℓr such that x(t) = 0 for t = −1,−2,−3, ....For complex valued sequences x ∈ ℓ1 or x ∈ ℓ2, we denote by X = Zx the Z-transformX(z) =∞∑t=−∞x(t)z−t, z ∈ C.Respectively, the inverse x = Z−1X is defined asx(t) =12π∫ π−πX(eiω)eiωtdω, t = 0,±1,±2, ....If x ∈ ℓ2, then X|T is defined as an element of L2(T).Let Hr be the Hardy space of functions that are holomorphic on Dc including the point at infinity(see, e.g., Duren (1970)). Note that Z-transform defines bijection between the sequences from ℓ+2 andthe restrictions (i.e. traces) of the functions from H2 on T.Definition 1 Let K be the class of all functions k ∈ ℓ∞ such that k(t) = 0 for t > 0 and K = Zk isK(z) =d(z)δ(z), (2.1)where d(·) and δ(·) are polynomials such that deg d < deg δ, and if δ(z) = 0 for z ∈ C then |z| > 1.The class includes all kernels k representing the anti-causal linear constant-coefficient difference equa-tions.Definition 2 Let K̂ be the class of functions k̂ : ℓ+∞ such that the function K̂(·) = Z k̂ belongs toH∞ ∩H2.2It follows from the definitions that if k̂ ∈ K̂ then k̂(t) = 0 for t < 0.We are going to study linear predictors in the form ŷ(t) =∑ts=−∞ k̂(t − s)x(s) for the processesy(t) =∑+∞s=t k(t − s)x(s), where k ∈ K and k̂ ∈ K̂. The predictors use historical values of currentlyobservable process x(·).Definition 3 Let X = {x(·)} be a class of sequences from ℓ∞, let r ∈ [1,+∞], and let K̃ ⊂ K be aclass of sequences.(i) We say that the class X is ℓr-predictable in the weak sense with respect to the class K̃ if, for anyk(·) ∈ K̃, there exists a sequence {k̂m(·)}+∞m=1 = {k̂m(·,X , k)}+∞m=1 ⊂ K̂ such that∥y − ŷm∥ℓr → 0 as m→ +∞ ∀x ∈ X ,wherey(t) =+∞∑s=tk(t− s)x(s), ŷm(t) =t∑s=−∞k̂m(t− s)x(s).(ii) Let the set Z(X ) = {X(eiω) = Zx|T, x ∈ X} be provided with a norm ∥ · ∥. We say that theclass X is ℓr-predictable in the weak sense with respect to the class K̃ uniformly with respect tothe norm ∥ · ∥, if, for any k(·) ∈ K̃ and ε > 0, there exists k̂(·) = k̂(·,X , k, ∥ · ∥, ε) ∈ K̂ such that∥y − ŷ ∥ℓr ≤ ε∥X∥ ∀x ∈ X , X = Zx.Here y(·) is the same as above, ŷ(t) =∑ts=−∞ k̂(t− s)x(s).We call functions k̂(·) in Definition 3 predictors or predicting kernels.3 The main resultLet Ω ∈ (0, π) be given, and letXL = {x(·) ∈ ℓ2 : X(eiω)= 0 if |ω| > Ω, X = Zx},XH = {x(·) ∈ ℓ2 : X(eiω)= 0 if |ω| < Ω, X = Zx}.In particular, XL is a class of band-limited processes, and XH is a class of high-frequency processes.33.1 Predictability of band-limited and high-frequency processes from L2Let K0 be the class of all functions k ∈ ℓ∞ such that k(t) = 0 for t > 0 and that K = Zk can berepresented asK(z) =z + bz + a, (3.1)for some real a ∈ (−∞,−1) ∪ (1,+∞) and b ∈ R.Theorem 1 (i) The classes XL and XH are ℓ2-predictable in the weak sense with respect to the classK0.(ii) The classes XL and XH are ℓ∞-predictable in the weak sense with respect to the class K0 uni-formly with respect to the norm ∥X(eiω)∥L2(−π,π).(iii) For any q > 2, the classes XL and XH are ℓ2-predictable in the weak sense with respect to theclass K0 uniformly with respect to the norm ∥X(eiω)∥Lq(−π,π).The question arises how to find the predicting kernels. In the proof of Theorem 1, a possible choiceof the kernels is given explicitly via Z-transforms.4 On a model with ideal low pass-pass filterCorollary 1 Assume a model with a process x(·) such that it is possible to decompose it as x(t) =xL(t) + xH(t), where xL(·) ∈ XL and xH(·) ∈ XH . Then this observer would be able to predict(approximately, in the sense of weak predictability with respect to the class K0) the values of y(t) =∑+∞s=t k(t−s)x(s) for k(·) ∈ K by predicting the processes yL(t) =∑+∞s=t k(t− s)xL(s) and yH(t) =∑+∞s=t k(t− s)xH(s) separately. More precisely, the process ŷ(t) = ŷL(t) + ŷH(t) is the prediction ofy(t), where yL(t) =∑t−∞ k̂L(t− s)xL(s) and yH(t) =∑t−∞ k̂H(t− s)xH(s), and where k̂L(·) andk̂H(·) are predicting kernels which existence for the processes xL(·) and xH(·) is established above.Let χL(eiω)= I{|ω|≤Ω} and χH(eiω)= 1 − χL(eiω)= I{|ω|>Ω}, where ω ∈ R; I denote theindicator function.The assumptions of Corollary 1 mean that there are a low-pass filter and a high-pass filter with thetransfer functions χL and χH respectively, with x(·) as the input, i.e., that the values xL(s) and xH(s)for s ≤ t are available at time t, wherexL(·) = Z−1XL, XL(eiω)= χL(eiω)X(eiω),xH(·) = Z−1XH , XH(eiω)= χH(eiω)X(eiω),4and where X = Zx. It follows that the predictability in the weak sense with respect to the class K0is possible for any process x(·) that can be decomposed without error on a band limited process anda high-frequency process, i.e., when there is a low-pass filters which behave as an ideal filter for thisprocess. (Since xH(t) = x(t)−xL(t), existence of the low pass filter implies existence of the high passfilter). On the other hand, Corollary 1 implies that the existence of ideal low-pass filters is impossiblefor general processes, since they cannot be predictable in the sense of Definition 3.Clearly, processes x(·) ∈ XL ∪ XH are automatically covered by Corollary 1, i.e., the existenceof the filters is not required for this case. For instance, we have immediately that xL(·) = x(·) andxH(·) ≡ 0 for band-limited processes.5 ProofsIt suffices to present a set of predicting kernels k̂ with the desired properties. We will use a version ofthe construction introduced in Dokuchaev (2008) for continuous time setting. This construction is verystraightforward and does not use the advanced theory of Hp-spaces.Let K1 be the class of all functions k ∈ K0 such that K = Zk can be represented asK(z) =1z + a, (5.1)for some real a ∈ (−∞,−1) ∪ (1,+∞).If k ∈ K0, then K = Zk can be represented asK(z) =z + bz + a=z + a+ b− az + a= 1 +cz + a,with a ∈ (−∞) ∪ (1,+∞), b ∈ R, and c = b − a. It follows that the process y(t) for k ∈ K0 canbe represented as y(t) = x(t) + c∑+∞s=t k1(t − s)x(s), where k1 ∈ K1. Therefore, it suffices to provetheorem for k ∈ K1 only.Let k(·) ∈ K1 and K(eiω)= Zk be defined by (5.1) for some for a ∈ (−∞,−1) ∪ (1,+∞).Let G = (−Ω,Ω), and letα = −1 + a cos(Ω)a+ cos(Ω). (5.2)Let us show that α = f(a) ∈ (−1, 1). Clearly, the functionf(a) =1 + a cos(Ω)a+ cos(Ω)is such that f ′(a) < 0 for all a such that |a| ≥ 1, f(−1) = −1, f(1) = 1, and f(±∞) = cos(Ω). Theseproperties imply that α = f(a) ∈ (−1, 1).5Further, we have that 1 + αa+ (a+ α) cos(Ω) = 0, andsign (a+ α)(1 + αa+ (a+ α) cos(ω)) > 0, ω ∈ G,sign (a+ α)(1 + αa+ (a+ α) cos(ω)) < 0, ω ∈ (−Ω,Ω)\G. (5.3)SetV (z) = 1− exp(γsign (a+ α)z + az + α), K̂(z) = V (z)K(z), γ ∈ R. (5.4)Lemma 1 (i) V (z) ∈ H∞ and K̂(z) = K(z)V (z) ∈ H∞ ∩H2.(ii) If γ < 0 and ω ∈ [−Ω,Ω], then |V (eiω)| ≤ 2. If γ > 0 and ω ∈ [−π, π]\(−Ω,Ω), then|V (eiω)| ≤ 2.(iii) If ω ∈ (−Ω,Ω), then V (eiω) → 1 as γ → −∞. If ω ∈ [−π, π]\[−Ω,Ω], then V (eiω) → 1 asγ → +∞.(iv) For any ε ∈ (0,Ω), V (eiω) → 1 as γ → −∞ uniformly in ω ∈ [−Ω + ε,Ω − ε] as γ → −∞,and V (eiω) → 1 as γ → +∞ uniformly in ω ∈ [−π, π]\(−Ω+ ε,Ω− ε).Proof of Lemma 1. Clearly, V ∈ H∞, and (z+ a)−1V (z) ∈ H2 ∩H∞, since the pole of (z+ a)−1is being compensated by multiplying with V . It follows that K(z)V (z) ∈ H2 ∩H∞. Then statement(i) follows.Further, for ω ∈ R,eiω + aeiω + α=(eiω + a)(e−iω + α)|eiω + α|2=1 + aα+ ae−iω + αeiω|eiω + α|2.HenceReeiω + aeiω + α=1 + aα+ (a+ α) cos(ω)|eiω + α|2.Then statements (ii)-(iv) follow from (5.3). This completes the proof of Lemma 1. Proof of Theorem 1. For x(·) ∈ ℓ2, let X = Zx, k = Z−1K, k̂ = Z−1K̂,y(t) =∞∑s=tk(t− s)x(s), ŷ(t) =t∑s=−∞k̂(t− s)x(s).Let Y = Zy, let V and K̂ be as defined above, and let Ŷ = K̂X .Let us consider the cases of XL and XH simultaneously. For the case of the class XL, consider γ < 0and assume that γ → −∞. Set Γ = [−Ω,Ω] for this case. For the case of the class XH , consider γ > 0and γ → +∞. Set Γ = [−π,−Ω] ∪ [Ω,+π] for this case.6Let x(·) ∈ XL or x(·) ∈ XH . In both cases, Lemma 1 gives that |V(eiω)| ≤ 2 for all ω ∈ Γ. Ifγ → −∞ or γ → +∞ respectively for XL or XH cases, then V(eiω)→ 1 for a.e. ω ∈ Γ, i.e., for a.e.ω such that X(eiω)̸= 0.Let us prove (i). Since K(eiω)∈ L∞(−π, π), K̂(eiω)∈ L∞(−π, π), and X(eiω)∈ L2(−π, π),we have that Y(eiω)= K(eiω)X(eiω)∈ L2(−π, π) and Ŷ(eiω)= K̂(eiω)X(eiω)∈ L2(−π, π).By Lemma 1, it follows thatŶ(eiω)→ Y(eiω)for a.e. ω ∈ R, (5.5)as γ → −∞ or γ → +∞ respectively for XL or XH cases. We have that|K̂(eiω)−K(eiω)| ≤ |V(eiω)− 1||K(eiω)| ≤ 2|Km(eiω)|, ω ∈ Γ, (5.6)|Ŷ(eiω)− Y(eiω)| ≤ 2|Y(eiω)| = 2|K(eiω)||X(eiω)|, ω ∈ Γ. (5.7)By (5.5),(5.7), and by Lebesque Dominance Theorem, it follows that∥Ŷ(eiω)− Y(eiω)∥L2(−π,π) → 0, i.e., ∥ŷ − y∥L2(−π,π) → 0 (5.8)as γ → −∞ or γ → +∞ respectively for XL or XH cases, where ŷ = Z−1Ŷ .Let us prove (ii)-(iii). Take d = 1 for (ii) and take d = 2 for (iii). If X(eiω)∈ Lν(−π, π) forν > d, then Hölder inequality gives∥Ŷ(eiω)− Y(eiω)∥Ld(−π,π) ≤ ∥K̂(eiω)−K(eiω)∥Lµ(Γ)∥X(eiω)∥Lν(Γ), (5.9)where µ is such that 1/µ+1/ν = 1/d. By (5.6) and by Lebesque Dominance Theorem again, it followsthat∥K̂(eiω)−K(eiω)∥Lµ(Γ) → 0 ∀µ ∈ [1,+∞), (5.10)as γ → −∞ or γ → +∞ respectively for XL or XH cases. Then, by (5.9)-(5.10), it follows that thepredicting kernels k̂(·) = k̂(·, γ) = Z−1K̂ are such as required in statements (ii)–(iii). This completesthe proof of Theorem 1. Corollary 1 follows immediately from Theorem 1.6 On the prediction error generated by a high-frequency noiseLet us estimate the prediction error for the case when predictor (5.4) designed for a band-limited processis applied to a process with a small high-frequency noise.Let Ω ∈ (0, π) and ν ∈ [0, 1) be given. Let us consider a process x(·) ∈ ℓ∞ such that |X(iω)| ≤ 1for ω ∈ G and |X(iω)| ≤ ν for ω ∈ [−π, π]\G , where X = Zx and G = (−Ω,Ω).7Assume that predictor (5.4) is constructed under the hypothesis that ν = 0 (i.e, that x(·) is a band-limited processes from XL), for some a ∈ R\[−1, 1]. For an arbitrarily small ε > 0, we can findγ = γ(ε) such that if the hypothesis that ν = 0 is correct, then∥ŷ − y∥ℓ∞ ≤ε2π, (6.1)where y(·) and ŷ(·) are such as in Definition 3.Let us estimate the prediction error for the case when ν > 0. We have that∥ŷ − y∥ℓ∞ ≤12π∥Ŷ(eiω)− Y(eiω)∥L1(−π,π),where Y = Zy and Ŷ = Z ŷ. Let Ω1 = Ω− ε/4 and G1 = (−Ω1,Ω1). By the assumptions on X , wehave that∥Ŷ(eiω)− Y(eiω)∥L1(−π,π) ≤ I1 + I2 + νI3,whereI1 = κ∫G1eγψ(ω)dω, I2 = κ∫G\G1eγψ(ω)dω, I3 = κ∫(−π,π)\Geγψ(ω)dω,and where κ = maxω ||K(eiω)|,ψ(ω) = sign (a+ α)Reeiω + aeiω + α.Note that ψ(ω) > 0 for ω ∈ G. Let ψ0 = minω∈G1 ψ(ω), and let γ = − log(2κ/ε)/ψ0. Then I1 ≤ ε/2.Further, I2 ≤ κmes (G\G1) = ε/2. Therefore, (6.1) holds if ν = 0.The value I1 + I2 represents the forecast error when ν = 0; this error can be done arbitrarily smallwith γ selected as above when ε→ 0.Let us estimate I3. Clearly, |ψ(ω)| ≤ 1 +∣∣∣ a−αeiω+α ∣∣∣ ≤ µ, where µ = 1 + |a− α|/(1− α). HenceI3 ≤ κ∫(−π,π)\Geγµdω = κ∫(−π,π)\Gelog(2κ/ε)ψ0µdω = 2κ(π − Ω)elog(2κ/ε)µψ0 .HenceνI3 ≤ 2κν(π − Ω)(2κε) µψ0.The value νI3 represents the additional error caused by the presence of unexpected high-frequency noise(when ν > 0). It can be seen that if ε → 0 than this error is increasing as a polynomial of ε−1 with therate depending on α (defined by Ω and a). If Ω → π then |α| → 1 and µ→ +∞, and, for a given ε, theerror is increasing exponentially in µ.87 Concluding remarks• By (5.2), α → ±1 as Ω → π, and the predictor suggested above loses its feasibility as Ω → π.(In particular, ∥k̂∥ℓ∞ → +∞).• If k(·) is a real valued function, then k̂ is also real valued. It follows from the fact that K (z̄) =K (z), and, therefore, K(e−iω)= K (eiω).• A similar approach can be applied to the case when X(z) vanishes on some connected set I ⊂ T.In this case, the classes K0 and K1 have to be replaced by similar classes with complex a ∈ Dc.For real valued kernels, it could be meaningful to include the functionsK represented by the sumsof two simple fractions, to ensure that the process Z−1k is real (i.e, that K (eiω) = K(e−iω)).• The predictors obtained above require the past values of x(s) for all s ∈ (−∞, t]. In practice,∑ts=−∞ k̂(t − s)x(s) can be approximated by∑ts=−M k̂(t − s)x(s) for large enough M >0. In addition, the corresponding transfer functions can be approximated by rational fractionpolynomials.• The system for the suggested predictors is stable, since the corresponding transfer functions havepoles in the domain {|z| < 1} only. However, the suggested predictors are not robust. Forinstance, if the predictor is designed for the class XL and it is applied for a process x(·) /∈ XLwith small non-zero energy at the frequencies outside [−Ω,Ω], then the error generated by thepresence of this energy is increasing if γ → ∞.• The results of this paper can be applied to discrete time stationary random Gaussian processes.In particular, assume that the spectral density of the underlying process x(t) vanishes outside theinterval [−Ω,Ω] ⊂ (−π, π). It is known that the minimal (optimal) predicting error is zero inthis case. The sequence of the predictors constructed above represents a sequence of suboptimalpredictors leading to vanishing prediction error.AcknowledgmentsThis work was supported by ARC grant of Australia DP120100928 to the author. In addition, the authorthanks Prof. Augusto Ferrante for useful discussion and advice on spectral analysis of time series.9References[1] Dokuchaev, N. (2008). The predictability of band-limited, high-frequency, and mixed processesin the presence of ideal low-pass filters. Journal of Physics A: Mathematical and Theoretical 41No 38, 382002 (7pp).[2] Dokuchaev, N. (2010). Predictability on finite horizon for processes with exponential decrease ofenergy on higher frequencies. Signal Processing 90 Iss. 2, 696–701.[3] Duren P. (1970) Theory of Hp-Spaces. Academic Press, New York.[4] Knab J.J. (1979). Interpolation of band-limited functions using the approximate prolate series.IEEE Transactions on Information Theory 25(6), 717–720.[5] Lyman R.J, Edmonson W.W., McCullough S., and Rao M. (2000). The predictability ofcontinuous-time, bandlimited processes. IEEE Transactions on Signal Processing 48, Iss. 2, 311–316.[6] Lyman R.J and Edmonson W.W. (2001). Linear prediction of bandlimited processes with flatspectral densities. IEEE Transactions on Signal Processing 49, Iss. 7, 1564–1569.[7] Marvasti F. (1986). Comments on ”A note on the predictability of band-limited processes.” Pro-ceedings of the IEEE, 74(11), 1596.[8] Papoulis A. (1985). A note on the predictability of band-limited processes. Proceedings of theIEEE, 73(8), 1332–1333.[9] Vaidyanathan P.P. (1987). On predicting a band-limited signal based on past sample values. Pro-ceedings of the IEEE, 75(8), 1125–1127.[10] Wainstein L.A. and Zubakov V.D. (1962). Extraction of Signals from Noise. Englewood Cliffs,NJ: Prentice-Hall.10

On predictors for band-limited and high-frequency time series

Crossref

Signal Processing

https://espace.curtin.edu.au/bitstream/20.500.11937/33373/2/188273_188273.pdf

On predictors for band-limited and high-frequency time series

Abstract

Similar works

Full text

Available Versions

espace@Curtin

Crossref