Witold Orzeszko

English

Research Papers in Economics

© Copyright by The Nicolaus Copernicus University Scientific Publishing House
DYNAMIC ECONOMETRIC MODELS 
Vol. 6 – Nicolaus Copernicus University – Toruń – 2004 
  
 
 
 
 
Witold Orzeszko 
Nicolaus Copernicus University in Toruń 
 
 
 
How the Prediction Accuracy of Chaotic Time Series  
Depends on Methods of Determining the Parameters of 
Delay Vectors 
 
 
1. Introduction 
 
  Unlike truly random processes, chaotic dynamics can be forecasted very 
precisely in a short run. In this paper, one of the methods applied to predicting 
chaotic dynamics – a local polynomial approximation, has been presented.   
A first step of this method is a reconstruction of system states by considering 
delay vectors. This procedure requires determining two parameters: an embed-
ding dimension and a time delay. Examples of the methods developed for this 
purpose are false nearest neighbours and mutual information. The aim of this 
paper is to examine an adequacy of these techniques in application to forecast 
methods. In addition, an alternative method of determining the parameters of 
delay vectors is proposed. 
 
 
2. The state space reconstruction 
 
  The main object of interest in the chaos theory is a dynamic system, which 
is formally defined as a pair ( ) f S, , where   is a set of system states and 
 is a map describing the dynamics of these states. Investigating   
a time series with methods from the chaos theory, one assumes, that it has been 
generated by a certain dynamic system 
d R S ⊂
S S f → :
( ) f S, . The first step of such an investi-
gation is taken by reconstructing the state space S. Its purpose is to uncover 
information about the states of the unobserved generating system and their dy-© Copyright by The Nicolaus Copernicus University Scientific Publishing House
Witold Orzeszko  232
namics, from the time series. The most widely used method of state space re-
construction is currently the technique of delay coordinates, which consist in 
constructing delay vectors (so called m-histories) in a form of 
=) , where m is called an embedding dimension. 
m
t x ˆ ..., , , ( ) 1 ( 1 lag m t lag t t x x x ⋅ − − ⋅ −
   A theoretical base for the method of delays is Takens’ theorem, which states 
that, for   a system  , where   is a set of m-histories and F is  
a map defining its dynamics, ie  , may be used to investigate prop-
erties of the unknown system 
1 2 + ≥ d m ) , ˆ ( F S S ˆ
m
t
m
t x x F 1 ˆ ) ˆ ( + =
( ) f S, . For example, an estimation of the attrac-
tor’s dimension and the Lyapunov exponents is possible. Moreover, a recon-
structed system   may be used to forecast the original motion of the sys-
tem (see Castagli et al. (1991)). 
) , ˆ ( F S
  In the delay coordinates method, the values of an embedding dimension  
m and a time delay lag must be a priori established. The proper choice of these 
parameters' values is particularly important in the case of short and noisy time 
series (see Bask (1998), Castagli et al. (1991), Zeng et al. (1991)). The false 
nearest neighbours – FNN method may be used to determine a value of the em-
bedding dimension (Kennel et al. (1992)). Its idea is to calculate an amount of, 
so called, false nearest neighbours as a function of the parameter m, in the fol-
lowing order: 
    -) for all m-histories   and   the norm 
m
t x 1 ˆ
m
t x 2 ˆ
m
t
m
t x x 2 1 ˆ ˆ −  is calculated,   (1) 
    -) the coefficient 
m
t
m
t
m
t
m
t m x x x x d 2 1
1
2
1
1 ˆ ˆ ˆ ˆ − − =
+ +  is calculated.  (2) 
If   is bigger than a certain fixed value, then   is said to be a false neigh-
bour of  . In the FNN method one should choose the value, for which the 
amount of false nearest neighbours is minimised.  
m d
m
t x 2 ˆ
m
t x 1 ˆ
  A criterion of lag determination which is often used, is a method called 
mutual information – MI, focusing on the investigated system as a producer of 
information. The aim of this method is to evaluate an amount of information 
about the state  , which may be forecasted according to information 
included in the state   (see Łażewski, Zator (2002)). To calculate this, Fraser 
and Swinney (1986) proposed a “mutual information” function and in addition 
they suggested that when its first minimum occurs, the proper lag is determined. 
) ( lag t s +
) (t s
 
 
3. A local linear approximation 
 
  Unlike truly random processes, very precise short-term predictions of cha-
otic dynamics are possible. Takens theorem implies that for   there 
exists the function  , satisfying: 
1 2 + ≥ d m
R R g
m
T → :© Copyright by The Nicolaus Copernicus University Scientific Publishing House
How the Prediction Accuracy of Chaotic Time Series...  233
  . (3)  ) , ... , , ( ) ˆ ( ) 1 ( lag m t lag t t T
m
t T T t x x x g x g x − − − + ≡ =
  It means that for time series with length of N one can determine a forecast, 
using the formula: 
  ,   (4)  ) , ... , , ( ) ˆ ( ) 1 ( lag m N lag N N T
m
N T T N x x x g x g x − − − + ≡ =
where T is a prediction horizon. Potentially   may be a complicated nonlinear 
function, but it can be approximated by 
T g
T g ~  of a standard functional form. In  
a local linear approximation the m-dimensional linear polynomial  T g ~  is consid-
ered, ie: 
  m m m T x x x x x x g α α α α + + + + = ... ) , ... , , ( ~
2 2 1 1 0 2 1 .   (5) 
  An estimation of the coefficients  i α  is proceeded by establishing a value  
k – a number of nearest (in the sense of the fixed m-dimensional norm) 
neighbours of vector  . Based on the found neighbours, the polynomial is 
fitted through the corresponding data points, by least squares. It should be 
pointed out that a priori established value k may be smaller than the amount of 
all available m-histories and that is why it is called “a local approximation”. Of 
course, the calculated forecasts depend on the values of the parameters k, m and 
lag.  
m
N x ˆ
 
 
4. Forecasting results 
 
  In this paper, four chaotic time series generated by the following models 
have been considered: 
1) Logistic map: 
  ) 1 ( 4 1 t t t x x x − = + , for the initial state  =0.7. (6)  0 x
  Logistic map is a one-dimensional discrete chaotic system, generating com-
plex dynamics widely described in literature. It has been quite often used by 
theorists of economics to construct chaotic models (see Sordi (1996)).  
2) Henon map:  
 ) , for  3 , 0 ; 4 , 1 1 ( ) , (
2
1 1 t t t t t x y x y x + − = + + ( ) 0 0, y x =( ) 9 , 0 ; 9 , 0 . (7) 
  Henon map is an example of a two-dimensional discrete chaotic system. In 
this paper the time series of its first coordinates has been investigated. 
3) Lorenz system: 
 
z xy dt
dz
y x xz dt
dy
x y dt
dx
4
92 , 45
) ( 16
− =
− + − =
− =
 (8) © Copyright by The Nicolaus Copernicus University Scientific Publishing House
Witold Orzeszko  234
  Lorenz system is a chaotic system with continuous time, proposed by a me-
teorologist E. Lorenz. The investigated time series has been generated by the 
equation  , for  ) 01 , 0 ( ⋅ = t x xt ( ) ( ) 1 , 1 , 1 ) 0 ( ), 0 ( ), 0 ( = z y x . 
4) Kaldor model. 
  The considered model is a discrete version of a continuous model of eco-
nomic growth proposed in 1940 by N. Kaldor. It consists of two difference 
equations: 
 
t t t t t t
t t t t t t t
K K Y I K K
Y S K Y I Y Y
δ
α
− = −
− = −
+
+
) , (
)) ( ) , ( (
1
1 , (9) 
 where  Y  is income, K  – capital stock, I  – gross investments,   – savings 
and 
S
δ  is the constant depreciation rate. Assuming that   and  t t t Y s Y S ⋅ = ) (
()
g
t
t
dYt
1
t K
f a Y e c I ⎟
⎠
⎞ ⎜
⎝
⎛ ⋅ + ⋅ + ⋅ =
+
−
2
2
ε , Kaldor model may generate the strange 
attractor. The following set of the parameters leading to chaotic motion were 
applied in this paper:  20 = α ,  s=0.21,  05 , 0 = δ ,  a=5,  c=20, 
d=0.01, 00001 , 0 = ε , e=0.05, f=280, g=4,5 (see Lorenz (1989)). The time series 
 generated from the initial state  () t Y 265 , 65 0 0 = = K Y  has been forecasted.  
  
  The analyzed time series with length of 1900 were divided into two parts:  
a part A – the first 1715 and a part B – the last 185 observations. For each value 
 from part B (i=1716, 1717, … ,1900) the one-step-ahead forecast has been 
calculated: 
i x
  .   (10)  ) , ... , , ( ~ ) ˆ ( ~ ~
) 1 ( 1 1 1 1 1 1 lag m i lag i i
m
i i x x x g x g x − − − − − − − ≡ =
  The part A was used to obtain parameters m and lag and to estimate poly-
nomial  1
~ g  coefficients. Values k = m+2, … ,  lag m ⋅ − − ) 1 ( 1714  have been 
considered. To evaluate the accuracy of the predictions, for each k, the root 
mean squared error:  
  ()
2 1900
1716
~
185
1 ∑
=
− ⋅ =
i
i i x x σ    (11) 
was computed. For convenience it was normalized by the standard deviation of 
the data from part A, forming the normalized error (see Farmer, Sidorowich 
(1987)):  
  % 100 ' ⋅ =
x σ
σ σ . (12) 
  Two methods of determining the parameters of delay vectors have been 
considered: 
1) “FNN–MI”: 
  False nearest neighbours and mutual information have been applied to part 
A, to calculate the embedding dimension m and the time delay lag. © Copyright by The Nicolaus Copernicus University Scientific Publishing House
How the Prediction Accuracy of Chaotic Time Series...  235
2) “PROG”: 
  The part A has been divided into two parts: A1 – the first 1650 and A2 – the 
last 65 observations. For each observation from A2 a local linear approximation 
has been used to forecast its value. The estimation of polynomial coefficients 
has been made, based on part A1. A forecast has been made for the following set 
of parameters: lag=1,2,…5,  m=1,2,…,15,  k=m+2, … ,  . 
Then the values of parameters leading to the smallest prediction error have been 
chosen.  
lag m ⋅ − − ) 1 ( 1649
  The smallest prediction errors with an adequate combination of parameters 
are summarized in Tables 1–4. For comparison, the errors obtained from 
ARMA models are also given. The illustrations of the root mean squared errors 
as a function of k are given in Figures 1–4.  
 
  Table 1. The smallest prediction errors for the logistic map  
 
Method  σ   ' σ   Parameters  Optimal k 
LA PROG  10
-7 10
-5%  m=2, lag=1 k =6 
LA FNN–MI  10
-5 0.002%  m=1, lag=7 k =4 
ARMA  0.33 96.03%  White noise   
 
  Source: Author’s calculations. 
 
  Table 2. The smallest prediction errors for the Henon map 
Method  σ   ' σ   Parameters  Optimal k 
LA PROG  10
-4 0.01%  m=4, lag=1 k =7 
LA FNN–MI  0.19
  25.50%  m=1, lag=18 k =16 
ARMA  0.63 85.59%  ARMA(2,6)  
  Source: Author’s calculations.  
 
  Table 3. The smallest prediction errors for the Lorenz system 
Method  σ   ' σ   Parameters  Optimal k 
LA PROG  0.004
  0.03%  m=14, lag=1  k=34 
LA FNN–MI  0.383
  3.00%  m=15, lag=10 k =107 
ARMA  0.008 0.06%  ARMA(5,4)  
 
  Source: Author’s calculations.  
 
  Table 4. The smallest prediction errors for the Kaldor model 
Method  σ   ' σ   Parameters  Optimal k 
LA PROG  0.11
  0.41%  m=2, lag=1 k =6 
LA FNN–MI  1.86
  7.13%  m=3, lag=3 k =13 
ARMA  17.28 66.45% ARMA(2,3)  
 
  Source: Author’s calculations.  
 
 
 © Copyright by The Nicolaus Copernicus University Scientific Publishing House
Witold Orzeszko  236
 
FNN-MI
0
0,05
0,1
0,15
0,2
0,25
0,3
0,35
0,4
31 7 1 4
k
p
r
e
d
i
c
t
i
o
n
 
e
r
r
o
r
PROG
0
0,05
0,1
0,15
0,2
0,25
0,3
0,35
0,4
41
k
p
r
e
d
i
c
t
i
o
n
 
e
r
r
o
r
7 1 3
 
Fig. 1. Prediction errors for the logistic map 
Source: Author’s calculations.  
 
 
FNN-MI
0
0,1
0,2
0,3
0,4
0,5
0,6
0,7
0,8
31 7 1 4
k
p
r
e
d
i
c
t
i
o
n
 
e
r
r
o
r
PROG
0
0,1
0,2
0,3
0,4
0,5
0,6
0,7
61
k
p
r
e
d
i
c
t
i
o
n
 
e
r
r
o
r
7 1 1
 
Fig. 2. Prediction errors for the Henon map  
Source: Author’s calculations. 
 
FNN-MI
0
1
2
3
4
5
6
17 1574
k
p
r
e
d
i
c
t
i
o
n
 
e
r
r
o
r
PROG
0
0,001
0,002
0,003
0,004
0,005
0,006
0,007
0,008
0,009
16 1701
k
p
r
e
d
i
c
t
i
o
n
 
e
r
r
o
r
 
Fig. 3. Prediction errors for the Lorenz system  
Source: Author’s calculations. 
 
 
 
 
 
 
 
 
 © Copyright by The Nicolaus Copernicus University Scientific Publishing House
How the Prediction Accuracy of Chaotic Time Series...  237
FNN-MI
0
2
4
6
8
10
12
14
16
18
20
51 7 0 8
k
p
r
e
d
i
c
t
i
o
n
 
e
r
r
o
r
PROG
0
5
10
15
20
25
41
k
p
r
e
d
i
c
t
i
o
n
 
e
r
r
o
r
7 1 3
 
Fig. 4. Prediction errors for the Kaldor model  
Source: Author’s calculations. 
 
  The differences and the quotients of prediction errors for the parameters 
obtained from the “FNN–MI” and “PROG” procedures are illustrated in Figures 
5–8. A positive difference indicates an advantage of the “PROG” procedure. 
The quotients charts illustrate how many times the prediction error obtained 
from “PROG” is smaller than the obtained one from “FNN–MI”. It has been 
shown that the most accurate forecasts of chaotic motions are obtained from  
a local linear approximation for small values of k (see Castagli (1992)). That is 
why, only   have been marked.  150 ≤ k
 
differences of prediction errors
-0,14
-0,12
-0,1
-0,08
-0,06
-0,04
-0,02
0
0,02
41 7 1 3
k
quotients of prediction errors
0
20
40
60
80
100
4 150
k
 
 
Fig. 5. The differences and the quotients of prediction errors for the logistic map  
Source: Author’s calculations. 
 
differences of prediction errors
-0,2
-0,15
-0,1
-0,05
0
0,05
0,1
0,15
0,2
0,25
61 7 1 1
k
quotients of prediction errors
0
500
1000
1500
2000
2500
3000
61
k
5 0
 
Fig. 6. The differences and the quotients of prediction errors for the Henon map 
Source: Author’s calculations.  © Copyright by The Nicolaus Copernicus University Scientific Publishing House
Witold Orzeszko  238
differences of prediction errors
0
1
2
3
4
5
6
17 1574
k
quotients of prediction errors
0
100
200
300
400
500
600
700
800
900
1000
17 150
k
 
Fig. 7. The differences and the quotients of prediction errors for the Lorenz system 
Source: Author’s calculations. 
 
differences of prediction errors
-2
0
2
4
6
8
10
51 7 0 8
k
quotients of prediction errors
0
10
20
30
40
50
60
70
80
90
51
k
5 0
 
Fig. 8. The differences and the quotients of prediction errors for the Kaldor model 
Source: Author’s calculations. 
 
  Table 5 summarizes values of k for which procedure “PROG” is superior 
and the smallest and the biggest quotients for  150 ≤ k . 
 
Table 5. A comparison of “PROG” and “FNN–MI” procedures 
 
Time series  An advantage of 
“PROG” procedure 
A range of quotients of 
prediction errors  
Logistic map  k<621  from 10.09 to 89.30 
Henon map  k<452  from 10.43 to 2590.65 
Lorenz map   every k   from 83.21 to 953.66 
Kaldor model  k<1421  from 3.36 to 79.59 
 
Source: Author’s calculations. 
 
 
5. Conclusions 
 
  The results obtained in this paper prove that a very accurate short-term pre-
diction of chaotic time series is possible. In application to analyzed time series, 
a local linear approximation has resulted in much better predictions than the 
ARMA models. The only exception was the Lorenz series, where the superior-© Copyright by The Nicolaus Copernicus University Scientific Publishing House
How the Prediction Accuracy of Chaotic Time Series...  239
ity depended on an applied procedure of determining the parameters of delay 
vectors. 
  The accuracy of forecasts obtained from a local linear approximation de-
pended highly on the applied method of establishing the parameters of delay 
vectors. For each time series, the “FNN–MI” procedure has led to much worse 
predictions than “PROG”. The results show that prediction errors also depend 
highly on a value of the third parameter: a number of nearest neighbours k, used 
in estimating the approximation polynomial coefficients. In the case of chaotic 
time series, the most accurate forecasts are obtained from a local linear ap-
proximation for small values of k. It has been shown, that for such values the 
superiority of the “PROG” procedure is clearly seen. It implies that FNN and 
MI are not adequate methods of determining the parameters of delay vectors for 
forecasting.  
 
 
References 
 
Bask, M. (1998), Esseys on Exchange Rates: Deterministic Chaos and Technical Analy-
sis, Working paper, Umea University, 1–36. 
Castagli, M. (1991), Eubank S., Farmer J. D., Gibson J., State Space Reconstruction in 
the Presence of Noise, Physica D, 51, 52–98. 
Casdagli, M. (1992), Chaos and Deterministic versus Stochastic Non-linear Modelling, 
Journal of the Royal Statistical Society B, 54, no. 2, 303–328. 
Farmer, J. D., Sidorowich, J. J. (1987), Predicting Chaotic Time Series, Physical Review 
Letters, 59, 845–848. 
Fraser, A. M., Swinney, H. L. (1986), Independent Coordinates for Strange Attractors 
from Mutual Information, Physical Review A, 33, 1134–1140. 
Kennel, M. B., Brown, R., Abarbanel, H. D. I. (1992), Determining Embedding Dimen-
sion for Phase-space Reconstruction Using a Geometrical Construction, Physical 
Review A, vol. 45, no. 6, 3403–3411. 
Lorenz, H–W. (1989), Nonlinear Dynamical Economics and Chaotic Motion, Springer- 
-Verlag, Berlin Heidelberg. 
Łażewski, M., Zator, K. (2002), Analiza chaosu deterministycznego w szeregach 
czasowych cen akcji pozbawionych trendu a prognozowanie rynków finansowych 
(Analysis of Chaos in Detrended Stock Price Series and Prediction of Financial 
Data), in: Rynek kapitałowy. Skuteczne inwestowanie. (Capital market. Efficient 
investing), ed. W. Tarczyński, Szczecin, 333–344. 
Sordi S. (1996), Chaos in Macrodynamics: an Excursion Through the Literature, pre-
sented at Perspectives in Macroeconomics 12–13 04, 1–27. 
Takens, F. (1981), Detecting Strange Attractors in Turbulence, (D. Rand and L.Young, 
Eds), in: Dynamical Systems and Turbulence, Springer-Verlag, 366–381. 
Zawadzki, H. (1996), Chaotyczne systemy dynamiczne (Chaotic Dynamic Systems), 
Wydawnictwo Akademii Ekonomicznej im. Karola Adamieckiego, Katowice. 
Zeng, X., Pielke, R.  A., Eykholt, R. (1992), Extracting Lyapunov Exponents From 
Short Time Series of Low Precision, Modern Physics Letters B, 6, 55–75. 

Analiza chaosu deterministycznego w szeregach czasowych cen akcji pozbawionych trendu a prognozowanie rynków finansowych (Analysis of Chaos

Chaos and Deterministic versus Stochastic Non-linear Modelling,

Chaos in Macrodynamics: an Excursion Through the Literature, presented at Perspectives

Chaotyczne systemy dynamiczne (Chaotic Dynamic Systems), Wydawnictwo Akademii Ekonomicznej im. Karola Adamieckiego,

Detecting Strange Attractors in Turbulence, (D. Rand and L.Young, Eds), in: Dynamical Systems and Turbulence,

Determining Embedding Dimension for Phase-space Reconstruction Using a Geometrical Construction, Physical Review A,

Esseys on Exchange Rates: Deterministic Chaos and Technical Analysis, Working paper,

Extracting Lyapunov Exponents From Short Time Series of Low Precision,

Independent Coordinates for Strange Attractors from Mutual Information,

Nonlinear Dynamical Economics and Chaotic Motion, Springer-Verlag,

Predicting Chaotic Time Series,

State Space Reconstruction in the Presence of Noise,

How the Prediction Accuracy of Chaotic Time Series Depends on Methods of Determining the Parameters of Delay Vectors

http://www.dem.umk.pl/dem/archiwa/v6/22_Orzeszko.pdf

How the Prediction Accuracy of Chaotic Time Series Depends on Methods of Determining the Parameters of Delay Vectors

Abstract

Similar works

Full text

Available Versions

Research Papers in Economics