This paper deals with the problem of the discrimination between wellpredictable and not-well-predictable time series. One criterion for the separation is given by the size of the Lyapunov exponent, which was originally defined for deterministic systems. However, the Lyapunov exponent can also be analyzed and used for stochastic time series. Experimental results illustrate the classification between well-predictable and not-well-predictable time series. --

Busse, Anja M.

English

Research Papers in Economics

econstor
www.econstor.eu
Der Open-Access-Publikationsserver der ZBW – Leibniz-Informationszentrum Wirtschaft
The Open Access Publication Server of the ZBW – Leibniz Information Centre for Economics
Nutzungsbedingungen:
Die ZBW räumt Ihnen als Nutzerin/Nutzer das unentgeltliche,
räumlich unbeschränkte und zeitlich auf die Dauer des Schutzrechts
beschränkte einfache Recht ein, das ausgewählte Werk im Rahmen
der unter
→  http://www.econstor.eu/dspace/Nutzungsbedingungen
nachzulesenden vollständigen Nutzungsbedingungen zu
vervielfältigen, mit denen die Nutzerin/der Nutzer sich durch die
erste Nutzung einverstanden erklärt.
Terms of use:
The ZBW grants you, the user, the non-exclusive right to use
the selected work free of charge, territorially unrestricted and
within the time limit of the term of the property rights according
to the terms specified at
→  http://www.econstor.eu/dspace/Nutzungsbedingungen
By the first use of the selected work the user agrees and
declares to comply with these terms of use.
zbw
Leibniz-Informationszentrum Wirtschaft
Leibniz Information Centre for Economics
Busse, Anja M.
Working Paper
Classification of Processes by the
Lyapunov exponent
Technical Report / Universität Dortmund, SFB 475 Komplexitätsreduktion in Multivariaten
Datenstrukturen, No. 2004,70
Provided in cooperation with:
Technische Universität Dortmund
Suggested citation: Busse, Anja M. (2004) : Classification of Processes by the Lyapunov
exponent, Technical Report / Universität Dortmund, SFB 475 Komplexitätsreduktion in
Multivariaten Datenstrukturen, No. 2004,70, http://hdl.handle.net/10419/22583Classiﬁcation of Processes by the
Lyapunov exponent
Anja M. Busse
University of Dortmund
⋆
Department of Statistics
44221 Dortmund, Germany
Abstract. This paper deals with the problem of the discrimination between well-
predictable and not-well-predictable time series. One criterion for the separation
is given by the size of the Lyapunov exponent, which was originally deﬁned for
deterministic systems. However, the Lyapunov exponent can also be analyzed and
used for stochastic time series. Experimental results illustrate the classiﬁcation
between well-predictable and not-well-predictable time series.
1 Introduction
For the description and the analysis of time series it is useful to initially intro-
duce a coarse classiﬁcation in order to be able to choose the most appropriate
tools for the more detailed analysis.
One important classiﬁcation is to discriminate between well-predictable
and not-well-predictable processes. Information about the predictability of
a process facilitates e.g. a sensible choice of the forecasting window. In the
case of chaotic time series the prediction accuracy can decrease considerably
already after only a few time-steps in contrast to a stationary stochastic
process (Abarbanel (1996), Casdagli (1991)).
In addition, in the analysis of stochastic processes there often is the prob-
lem that only one time series is available and no previous knowledge about
the temporal-functional relationship is given.
Despite these restrictions a formal identiﬁcation of predictable time series
can be achieved by analyzing the Lyapunov spectrum or the largest Lyapunov
exponent of the time series (this is often just referred to as the Lyapunov
exponent). Originally, the Lyapunov exponent was deﬁned for non-stochastic,
deterministic systems. Anyhow, the concept behind the Lyapunov exponent
can be embedded into a statistical framework.
The remainder of this paper is organized as follows. After an introduction
of the Lyapunov exponent (Sec 2) we will show that it can be used as a
criterion to discriminate between well-predictable and not-well predictable
time series (Sec 3). Experimental results of a BTA-deep-hole drilling process
⋆ This work has been supported by the Deutsche Forschungsgemeinschaft, Sonder-
forschungsbereich 475.2 Busse
x0
x* 0
∆
∆0
x*
x* N
x N
N
Fig.1. Two trajectories are regarded over time in order to observe the convergence
or divergence of a process.
illustrate the method of separation by the Lyapunov exponent (Sec 4). A
conclusion is drawn in Sec. 5.
2 Lyapunov exponent
One possibility to distinguish between well-predictable and not-well-predict-
able time series is given by the computation of the largest Lyapunov exponent
(often brieﬂy called the Lyapunov exponent). This was originally deﬁned for
non-stochastic, deterministic processes. However, the Lyapunov exponent can
also be analyzed and used for the stochastic case.
Firstly, it will be introduced for deterministic processes. The dynamics of
deterministic processes is deﬁned by
xt+1 = ft(x0) = f(xt) , (1)
with initial point or initial state x0 ∈ IR
k, xt describes the state at time
t. The functional relationship is described by f and it is assumed that f is
diﬀerentiable everywhere. Hence, the dynamics is entirely deterministic.
The Lyapunov exponent describes the divergence of two diﬀerent trajec-
tories. This can be motivated as follows:
In Fig. 1 the behavior of two nearby trajectories is shown. The starting
point x∗
0 is “nearby” but displaced from x0. Furthermore, the trajectories
follow the same functional relationship. The distance between x0 and x∗
0 is
given by
 0 = |x∗
0 − x0|. (2)
Hence, the distance after one iteration can be approximated by applying the
ﬁrst order Taylor expansion as follows:
 1 = |x∗
1 − x1| = |f(x∗
0) − f(x0)| ≈ |f′(x0)| · |x∗
0 − x0|. (3)
After N iterations the distance between the trajectories arises from using the
chain rule:
 N = |x∗
N − xN| ≈
N−1 Y
i=0
|f′(xi)| ·  0 (4)Classiﬁcation by the Lyapunov exponent 3
Thus, we are interested in diverging or converging of the trajectories after
N iterations in comparison to the beginning. This is estimated by an expan-
sion rate. Obviously, the expansion rate of the trajectories can be expressed
by
 N
 0
≈
N−1 Y
i=0
|f′(xi)| = eN λN(x0) , (5)
where λN is the characteristic value dependent on time N and x0. This ex-
pansion rate illustrates the behavior of the trajectories after N iterations in
dependence of  0 and x0.
The consideration of the asymptotic behavior for N → ∞ yields the
deﬁnition for the Lyapunov exponent of deterministic processes:
λ(x0) := lim
N→∞
1
N
N−1 X
i=0
ln|f
′(xi)|. (6)
It is the long time consideration of the average logarithmic derivation after
N equals inﬁnity many iterations. The Lyapunov exponent measures the
asymptotic average logarithmic expansion rate along two trajectories.
The derivative f′ of the function f is often unknown. It has to be evaluated
from the given observation series. Various approaches for the calculation of λ
have been suggested in the literature (for more details see for example Sano
and Sawada (1985), Kantz and Schreiber (1997)).
If stochastic processes are considered, two cases have to be distinguished
separately: The random eﬀect is additive in the functional equation and the
random eﬀect is not necessarily additive.
First the case with an additive noise is considered. The dynamics of
stochastic processes with an additive random eﬀect is deﬁned by
Xt+1 = f(Xt) + ǫt. (7)
By transforming
Xt+1 = g(Xt,ǫt), with g(Xt,ǫt) = f(Xt) + ǫt (8)
we obtain the same derivatives of g and f so that the deﬁnition of the Lya-
punov exponent for stochastic processes with an additive noise is directly
derived from the deterministic case. The function g is inserted in the deﬁni-
tion of the Lyapunov exponent for deterministic processes and the deﬁnition
for stochastic processes with an additive noise is obtained:
λ(X0) := lim
N→∞
1
N
N−1 X
i=0
ln||
d
dXt
g(Xt(xi),ǫt)|. (9)
However, an additive noise can not always be justiﬁed because this as-
sumption is too restrictive with regard to possible model classes. Thus, the4 Busse
t I
0 I
xt
x0
Fig.2. Information loss of the information area I0 in comparison to the information
area It
general case is considered. The dynamics of stochastic processes with a non-
necessarily additive noise is deﬁned by
Xt+1 = h(Xt,ǫt). (10)
The Lyapunov exponent can be naturally generalized as:
˜ λ(x0) := lim
N→∞
1
N
X
t = 0N − 1[ln|
d
dXt
h(Xt(x0),ǫt)|] (11)
In any case, as an estimator of the Lyapunov exponent
ˆ λ =
1
N
N−1 X
i=0
ln|
d
dXt
h(Xt,ǫt)|. (12)
is taken. For details about the Lyapunov exponent for deterministic and
stochastic processes see Busse et al. (2001), Busse (2003), and Busse and
Weihs (2004).
3 Well-predictable and not-well-predictable processes
The knowlegde about the quality of prediction of processes is an important
property for the interpretation of the predicted results. The greater the infor-
mation loss in a multi-step-forcasting the greater the decrease in the quality
of prediction. Thus, it is interesting to know a measure of information loss
for avoiding possible misinterpretations. The Lyapunov exponent can be in-
terpreted as an expansion rate with a direct context to the information loss
over time.
If we assume that the true starting point x0 of a time series is possibly
displaced by an ǫ, we know only the information area about the starting
point we do not know the proper position of x0. After t-time steps the time
series is in the information area at time t, It and after t+1-time steps in the
information area It+1 (Fig. 2). If the information area is small, we have more
information about the true position of the data point in contrast to a greaterClassiﬁcation by the Lyapunov exponent 5
information area (Beck (1993)). As an adequate measure of information the
information content bn of a true position of a data point in an information
area In of the volume  n is given by:
bn := ln
1
 n
= −ln( n). (13)
The connection to the volume of an information area is given by
 n = exp(−bn). (14)
It can be characterized by the distance between two trajectories of a process
at time n. For the evaluation of the quality of prediction we are interested
in the information loss from one time to the next. For this the diﬀerence of
two information contents before and after an iteration step are determined.
Thus, the information loss IV about the true position of a data point in one
iteration step is given by
IV = bn − bn+1 = ln n+1 − ln n ≈ ln|f′(xn)|, (15)
with  n+1 ≈ |f′(xn)| ·  n. If the diﬀerence is positive, IV describes an
information increase, whereas an information loss is given, if In is less than
In+1.
The information loss is the logarithmic ﬁrst derivative of the functional
relationship of a process, so that the Lyapunov exponent can be used for the
description of the average information loss:
λ(X0) = lim
N→∞
1
N
N−1 X
i=0
ln|f′(Xi)| ≈ lim
N→∞
1
N
N−1 X
i=0
(bi − bi+1). (16)
In contrast to the traditional classiﬁcation of time series we do not use
the given data points but the possible position areas like k-means clustering
(Hastie (2001)).
The classiﬁcation of both deterministic and stochastic processes by the
Lyapunov exponent is given by:
• λ(x0) < 0 ⇔  N <  0 ⇒ good predictability
The information about the true position of the data increases due to
the reduction of the information area. Consequently, we get a good pre-
dictability.
• λ(x0) ≈ 0 ⇔  N ≈  0 ⇒ predictability like a random walk
Here, the information content levels oﬀ. We have neither information loss
nor information increase.
• λ(x0) > 0 ⇔  N >  0 ⇒ bad predictability
The information loss about the true position of the data increases over
time due to the information area increases. Consequently, we get a bad
predictability.6 Busse
a
c
c
e
l
e
r
a
t
i
o
n
 
(
i
n
 
m
/
s
)
time (in s/15000)
2
0
50
550100 550300
−4000
−2000
0
2000
4000
560000 550000
570000 570200
0
2000
4000
−4000
540000
−50
580000 570000
−2000
Fig.3. Acceleration of the drill head in the non-chatter (left) and the chatter (right)
area
4 Experimental results
The Lyapunov exponent achieved a distinction between well-predictable and
not-well-predictable time series. We applied this classiﬁcation to a real-world-
problem. The aim was to analyze a BTA-deep-hole drilling process and to
control at best working conditions (VDI (1974)). BTA-deep-hole drilling is
used to produce holes with a high length-to-diameter-ratio. But the slen-
derness of the tool can yield unwanted states, like chatter. This should be
avoided, because chatter generates surface discontinuity at the workpiece,
noise exposure and increases the wear of cutting edges substantially.
For the analysis we were given a time series of acceleration data with dif-
ferent types of the process. First the non-chatter area with a weakly periodi-
cal part and the chatter area with a strongly periodical part. The transition
between these areas appears funnel shaped.
The aim was to identify the chatter early to avoid the possible conse-
quences. For this we characterize the transition in time windows of length
1024 data points. We chose the Lyapunov exponent because its ability to dis-
tinguish between good and bad predictability makes it possible to estimate
the starting point of the transition. For every time window in the transition
area the Lyapunov exponent of the given time series was evaluated. For this,
equation (6) and equation (12) respectively is estimated by the approach of
Kantz and Schreiber (1997). In order to identify the ”true” transition it is
important whether the Lyapunov exponent is less than 0, that is to classify
a good forecast-property or whether λ is greater than 0.
For the interpretation of the resultsnote that amplitude increase is in
the time window 559313–560335. The identiﬁcation of the change between
the non-chatter and the chatter area occured one and a half drill rotations
earlier than the real amplitude increase (see Table 1, 556240–558287).Classiﬁcation by the Lyapunov exponent 7
Table 1. Lyapunov exponent in time windows in the transition area.
time windows (data points) Lyapunov exponent classiﬁcation decision
550096–551119 0.004 > 0
551120–552143 0.009 > 0
552144–553167 0.008 > 0
553168–554191 0.019 > 0
554192–555215 0.016 > 0
555216–556239 0.012 > 0
556240–557263 0.003 > 0
557264–558287 -0.004 < 0
558288–559312 -0.003 < 0
559313–560335 -0.004 < 0
560336–561359 -0.004 < 0
561360–562383 -0.013 < 0
The distinction between well predictable and not-well predictable pro-
cesses by the Lyapunov exponent was applied with good results to various
time series. For more details about other applications see for example Busse
(2003).
5 Conclusion
We analyzed the Lyapunov exponent in the context of the separation be-
tween well-predictable and not-well-predictable processes. Such a classiﬁca-
tion seems useful since it would facilitate a more detailed analysis of the
underlying process with respect to the choice of the appropriate tools. In
this work the Lyapunov exponent was suggested for separation. This crite-
rion describes the asymptotical average logarithmic expansion of the model
derivative.
It was shown that the Lyapunov exponent can be used for the evaluation
of predictability. The Lyapunov exponent as a classiﬁcation criterion can be
used without knowledge stochastics of the process and without knowledge
about the temporal-functional relationship, only using the given time series.
In addition, diﬀerent areas of a BTA-deep-hole drilling process were clas-
siﬁed by the Lyapunov exponent. Detection of the transition to chatter was
possible substantially earlier than the rise in acceleration was visible.
References
Abarbanel, H. D. I. (1996). Analysis of Observed Chaotic Data. Institute for Non-
linear Science. Springer Verlag, New York.
Arnold, V. I. and A. Avez (1968). Ergodic problems of classical mechanics.
W. A. Benjamin, New York.8 Busse
Beck, C. and F. Schl¨ ogl (1993). Thermodynamics of chaotic systems. Cambridge
University Press, Cambridge.
Busse, A. M., D. Steuer and C. Weihs (2001). An Approach for the Determina-
tion of Predictable Time Series. Technischer Bericht 12, SFB 475, Fachbereich
Statistik, Universit¨ at Dortmund, 44221 Dortmund, Germany.
Busse, A. M. (2003). Klassiﬁkation von Datenreihen mit Hilfe des Lyapunov–
Exponenten. Dissertation Fachbereich Statistik. Universit¨ at Dortmund.
eldorado.uni-dortmund.de:8080/FB5/ls7/forschung/2003/Busse.
Busse, A. M. and C. Weihs (2004). Lyapunov exponent for stochastic time series.
Technischer Bericht 37, SFB 475, Fachbereich Statistik, Universit¨ at Dortmund,
44221 Dortmund, Germany.
Casdagli, M. (1991). Chaos and deterministic versus stochastic non-linear modeling.
J. R. Statist. Soc. B 54, 303–328.
Hastie, T., R. Tibshirani and J. Friedman (2001). The Elements of Statistical Learn-
ing. Data Mining, Inference, and Prediction. Springer, New York.
Kantz, H. and T. Schreiber (1997). Nonlinear time series analysis. Cambridge Uni-
versity Press, Cambridge.
Sano, M. and Y. Sawada (1985). Measurements of the lyapunov spectrum from a
chaotic time series. Physical Review Letters 55, 1082–1085.
Stout, W. F. (1974). Almost sure convergence. Academic Press, New York.
VDI-Richtlinie 3210 (1974, Juni). Tiefbohrverfahren. VDI, D¨ usseldorf.

Classification of Processes by the Lyapunov exponent

Abstract

Similar works

Full text

Available Versions

Research Papers in Economics