Filipot, Jean-François

Naveau, Philippe

Platzer, Paul

Tandeo, Pierre

Yiou, Pascal

HAL-Université de Bretagne Occidentale

Predicting Analog Forecasting Errors using Dynamical Systems

HAL UVSQ

HAL-CEA

HAL-INSU

Archive Ouverte en Sciences de l'Information et de la Communication

HAL Id: hal-02276974
https://hal-imt-atlantique.archives-ouvertes.fr/hal-02276974
Submitted on 3 Sep 2019
HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d’enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.
Predicting Analog Forecasting Errors using Dynamical
Systems
Paul Platzer, Pascal Yiou, Pierre Tandeo, Philippe Naveau, Jean-François
Filipot
To cite this version:
Paul Platzer, Pascal Yiou, Pierre Tandeo, Philippe Naveau, Jean-François Filipot. Predicting Analog
Forecasting Errors using Dynamical Systems. CI 2019 : 9th International Workshop on Climate
Informatics, Oct 2019, Paris, France. ￿hal-02276974￿
PREDICTING ANALOG FORECASTING ERRORS
PREDICTING ANALOG FORECASTING ERRORS
USING DYNAMICAL SYSTEMS
Paul Platzer1,2,3, Pascal Yiou1, Pierre Tandeo2, Philippe Naveau1, Jean-Franc¸ois Filipot3
Abstract—Nearest neighbor algorithms called Analog
Forecasting have been used to produce short-range to
long-range forecasts in many atmospheric and oceanic ap-
plications. Analog Forecasting is often treated as a purely
empirical method, independent from physical equations.
In this paper, we investigate Analog Forecasting error
from a dynamical systems point of view. Assuming that
analogs follow the same dynamics as the system of
concern, we evaluate statistical properties of Analog
Forecasting errors. We further design dynamics-based
systematic error correction methods for standard Analog
Forecasting techniques. These procedures are tested on
the 3-dimensional Lorenz-63 system.
I. MOTIVATION
Two states of a system are called ”analogs” when
they meet a similarity criterion such as a low Euclidean
distance. Analog Forecasting (AF) is based on the as-
sumption that similar states will have similar evolution,
and produces forecasts based on the ”successors” in
time of the analogs of the current state. AF has been
used in a wide variety of applications in atmospheric
prediction, see for instance [1], [2] or [3]. Although
AF is less precise than forecasting methods based on
the numerical resolution of physical equations, AF can
provide statistical forecasts at a low computational cost,
outperforming persistence and climatological forecasts.
Also, increasing observational data and computer mem-
ory make analogs a promising forecast tool.
Although the concept of analogs was originally in-
troduced by [4] to gain information on the dynamics of
the atmosphere, today AF is mostly used as a purely
empirical method and the link between AF and the
underlying dynamics of the system is rarely mentioned.
However [5] could improve the skills of AF by using
dynamics-based weights. [6] used a dynamical systems
framework to study analogs, but focusing on recurrence
time statistics rather than AF performance.
Corresponding author: P Platzer, paul.platzer@imt-atlantique.fr
1Laboratoire des Sciences du Climat et de l’Environnement,
Saclay, France 2IMT-Atlantique, Plouzane´, France 3France E´nergies
Marines, Plouzane´, France
Here we propose to improve our understanding of
forecasting errors associated with AF by expressing
them as a function of the dynamics of the system of
concern. We assume, as a first step, that the analogs
and the system follow exactly the same dynamics.
With calculations similar to [7], we give a simple
approximation for the error of standard AF techniques
for small lead times. This enables us to evaluate AF
root mean square error (RMSE). We further propose
two systematic error correction methods for standard
AF techniques. We test our methods on the famous
Lorenz-63 system [8] in numerical experiments.
This study aims at bridging the gap between purely
data-driven AF and purely model-driven methods. We
show that understanding the role of the system’s dynam-
ics can not only provide statistical information about AF
error, but also help improving AF performances.
II. METHOD
Let x0 represent the system state at time t = 0. We
are interested in estimating xt, for time t > 0. AF starts
with finding a finite number of analogs of x0 inside a
large database called the catalog. We note ak0 the k-
th analog of x0. Then AF considers their successors
at time t, where the k-th is noted akt . Then, a common
AF technique is the Locally Constant (LC) forecast [9],
which takes a weighted average of all successors (with
weights ωk) as a forecast. LC forecast is written:
LCt =
∑
k
ωka
k
t . (1)
The Locally Incremental (LI) forecast [9] makes use of
the differences between successors and analogs called
”increments”, i.e. (akt − ak0), rather than successors akt .
LI takes the sum of the initial state x0, and the weighted
average of increments
∑
k ωk(a
k
t − ak0). It can also be
written:
LIt = LCt − ε0, (2)
where ε0 = LC0 − x0. We suppose that x follows the
dynamical equation dxdt = f(x), as well as all a
k. We
PLATZER, YIOU, TANDEO, NAVEAU, FILIPOT
write Jt = ∇f(xt)T (T-superscript is the transpose and
∇ is the gradient operator) the Jacobian matrix of f
along the trajectory x at time t. If we further assume
that ωk‖ak0−x0‖ is small enough (up to a given norm)
for all k, the error for the LC forecast is:
LCt − xt ≈
[
I+ tJ0 +
t2
2
(J20 + J˙0)
]
ε0, (3)
and the error for the LI forecast is:
LIt − xt ≈
[
tJ0 +
t2
2
(J20 + J˙0)
]
ε0. (4)
where I is the identity matrix and J˙0 = dJ0dt . The deriva-
tion of these formulae is similar to the calculations of
[7]. From those formulae we can:
1) infer statistical properties of AF errors associated
with LC-type and LI-type AF techniques,
2) apply a systematic error correction to improve the
skills of those techniques.
From Eqs. (3–4), it follows that LF and LI are
unbiased as long as ε0 is of zero mean. Squared error
of LC can be evaluated up to the order t2 by taking the
square of Eq. (3):
‖LCt−xt‖2 ≈ εT0
(
I+ 2tJ0 + t
2(J20 + J˙0 + J
T
0 J0)
)
ε0,
(5)
and taking the square of Eq. (4) gives the square error
of LI up to the order t3:
‖LIt−xt‖2 ≈ εT0
(
t2JT0 J0 + t
3JT0 (J
2
0 + J˙0)
)
ε0. (6)
Eqs. (5–6) involve products of J0-dependent terms
and ε0-dependent terms. We assume that J0 and ε0
are statistically independent. Thus, when we estimate
the RMSE of LC and LI, we can calculate separately
averages of J0-dependent terms, and averages of ε0-
dependent terms, and multiply them. The first averages
are taken over the attractor’s invariant distribution,
which is equivalent to taking an average over a very
long trajectory. The second averages are taken over the
attractor’s invariant distribution (because ε0 depends
on x0) and over possible realizations of the catalog
(because ε0 depends on LC0). Both could be estimated
using the analog database called the ”catalog”.
Finally, we define the following error-corrected AF
techniques:
GDCt = LIt −
[
t < J0 > +
t2
2
< J20 >
]
ε0 , (7)
where GDC stands for ”global dynamics correction”,
and
LDCt = LIt −
[
tJ0 +
t2
2
(J20 + J˙0)
]
ε0 , (8)
where LDC stands for ”local dynamics correction”.
The means (symbol <>) are taken over the invariant
distribution of the attractor. GDC only needs average
information from the Jacobian matrix, which can be
inferred offline. LDC needs local information on the
dynamics, which needs to be inferred online. Both
global and local information about the Jacobian could
be estimated using the catalog.
The quality of the estimation of J0-dependent quan-
tities from the catalog depends on the dimension of
the attractor and the size and quality of the catalog.
However, this problematic is not studied further here
and in our numerical experiments we use directly
analytical expressions of the Jacobian at x0.
III. EVALUATION
We use the 3-dimensional Lorenz Model [8] in its
standard non-dimensional form, with parameters σ =
10, ρ = 28, β = 8/3. Time integration is done through
a 4th-order Runge-Kutta finite difference numerical
scheme with a time step of 0.01 (non-dimensional
units). For this model the Jacobian at any point x0 is:
J0 =
 −σ σ 0ρ− z0 −1 −x0
y0 x0 −β
 , (9)
where x0 = (x0, y0, z0). We build 50 independent cat-
alogs of 40000 analogs each. For each catalog we draw
400 initial vectors x0 from the invariant distribution of
the attractor. For each of these points we apply AF with
our four different methods (LC, LI, GDC and LDC) at
20 different lead times from t = 0.01 to t = 0.2.
Each analog forecast goes through the following
initial steps:
1) select the 40 analogs of x0 with the lowest
Euclidian distance to x0,
2) if two or more selected analog follow each other
in time by one integration time step, make a group
of all these analogs and keep only the one with
the lowest Euclidian distance to x0,
3) use Gaussian kernels for the weights ωk ∝
exp
(−0.5‖x0 − ak0‖2/λ2) with λ set to the me-
dian of ‖x0−ak0‖ inside the small set of analogs
used for the forecast.
For GDC, the averages of J0-dependent terms are
estimated offline, using Eq. (9) and within a random
trajectory of 100000 points on the attractor. For LDC,
J0-dependent terms are computed online, using Eq. (9).
As mentioned earlier, all those terms could be estimated
without knowledge of the model equations, provided
that the catalog is large and precise enough.
PREDICTING ANALOG FORECASTING ERRORS
Fig. 1. RMSE associated with locally constant and locally incre-
mental analog forecasting. Black, full: LC, empirical. Black, dashed:
LC, predicted using Eq. (5). Red, full: LI, empirical. Red, dashed:
LI, predicted using Eq. (6).
Empirical RMSE associated with LC and LI are
shown in the full lines (black and red) of figure 1,
with estimations from Eqs. (5–6) in dashed lines. The
averages of J0-dependent terms are estimated on a
trajectory of 100000 points on the attractor. The av-
erages of ε0-dependent terms are estimated over the
50*40000 different analog forecasts, which span 50
different catalogs and 40000 different points x0 inside
the attractor. Estimations based on Eqs. (5-6) are perfect
for t → 0, and as t grows several neglected terms
influence the validity of our estimations. In particular,
we neglected higher-order terms in the t-expansion of
Eqs. (5-6) and non-linear terms in ε0. However, our
estimations capture correctly the behavior and order of
magnitude of RMSE for all lead times considered here.
Root median square errors associated with our four
different forecasting methods are shown in figure 2. The
median values of LC (black) and LI (red) square errors
are smaller than their mean values, this is due to rare but
large values of analog forecasting errors. At small lead
times, LDC (green) outperforms GDC (blue), which
outperforms LI (red), which outperforms LC (black).
However, our different types of error corrections have
a negative influence on the performances for large lead
times, as the hypotheses that were used to build these
error corrections are not fulfilled for large lead times.
As x0 spans the attractor, the ranges of variations
of the x0, y0 and z0 are -19 to 18, -26 to 24, and 3 to
47, respectively. Thus all forecasting methods presented
here show good performances with RMSE and root
median square errors of only a few percent of the state
variables’ ranges of variations.
Fig. 2. Empirical root median squared errors associated with dif-
ferent analog forecasting techniques. Black: locally constant (LC).
Red: locally incremental (LI). Blue: global dynamics correction
(GDC). Green: local dynamics correction (LDC).
IV. CONCLUSION
AF is an empirical method, but its performances can
be interpreted in the framework of dynamical systems,
allowing for systematic error correction. We have ex-
pressed the leading terms of the errors of common AF
techniques for short lead times. We then used those
expressions on the Lorenz system as a means of esti-
mating the RMSE and for systematic error correction,
yielding positive results for short lead times.
Our work is limited to AF error due to imprecision
of the analogs, i.e. non-vanishing ε0, but we wish
to extend this formalism to the case the analogs and
the system state follow different dynamics, creating
additional error with potential bias. We should then
compare the way AF variance is estimated empirically
in AF applications (as in [10] and [9]) with our estima-
tions based on the dynamics. Also, we could use more
general formulae than Eq. (4) to find bounds for the AF
error, possibly independent from local dynamics, to use
it as error bound for AF. Finally, this formalism should
be extended to the case of partial observations of a
high-dimensional dynamical system, which is a more
realistic situation. Indeed, in most atmospheric and
oceanic applications only a few physical variables (such
as temperature, pressure, humidity...) are observed, and
observations are limited in time and space due to
operational constraints. In such situations, AF is usually
combined with Takens’ time-lagged embeddings, using
analogs on partial observations at current and past times
[11].
ACKNOWLEDGMENTS
Funding for the authors was provided by ANR No.
10-IEED-0006-26 and ERC grant No. 338965-A2C2.
PLATZER, YIOU, TANDEO, NAVEAU, FILIPOT
REFERENCES
[1] L. Delle Monache, F. A. Eckel, D. L. Rife, B. Nagarajan, and
K. Searight, “Probabilistic Weather Prediction with an Analog
Ensemble,” Monthly Weather Review, vol. 141, no. 1974,
pp. 3498–3516, 2013.
[2] A. Ayet and P. Tandeo, “Nowcasting solar irradiance using
an analog method and geostationary satellite images,” Solar
Energy, vol. 164, no. July 2017, pp. 301–315, 2018.
[3] P. Yiou and C. De´andre´is, “Stochastic ensemble climate fore-
cast with an analogue model,” Geoscientific Model Develop-
ment, vol. 12, no. 2, pp. 723–734, 2019.
[4] E. N. Lorenz, “Atmospheric Predictability as Revealed by
Naturally Occurring Analogues,” Journal of the Atmospheric
Sciences, vol. 26, no. 4, pp. 636–646, 1969.
[5] Z. Zhao and D. Giannakis, “Analog forecasting with
dynamics-adapted kernels,” Nonlinearity, vol. 29, no. 9,
pp. 2888–2939, 2016.
[6] C. Nicolis, “Atmospheric Analogs and Recurrence Time Statis-
tics: Toward a Dynamical Formulation,” Journal of the Atmo-
spheric Sciences, vol. 55, pp. 465–475, 1998.
[7] C. Nicolis, P. Perdigao, and S. Vannitsem, “Dynamics of
Prediction Errors under the Combined Effect of Initial Condi-
tion and Model Errors,” Journal of the Atmospheric Sciences,
vol. 66, pp. 766–778, 2009.
[8] E. N. Lorenz, “Deterministic nonperiodic flow,” Journal of the
atmospheric sciences, vol. 20, no. 2, pp. 130–141, 1963.
[9] R. Lguensat, P. Tandeo, P. Ailliot, M. Pulido, and R. Fablet,
“The analog data assimilation,” Monthly Weather Review,
vol. 145, no. 10, pp. 4093–4107, 2017.
[10] P. Yiou, “AnaWEGE: a weather generator based on analogues
of atmospheric circulation,” Geoscientific Model Development,
vol. 7, pp. 531–543, apr 2014.
[11] F. Takens, “Detecting strange attractors in turbulence,” in
Dynamical Systems and Turbulence, Warwick 1980, pp. 366—
-381, Springer, 1981.


English

https://hal-imt-atlantique.archives-ouvertes.fr/hal-02276974/document

Predicting Analog Forecasting Errors using Dynamical Systems

Abstract

Similar works

Full text

Available Versions

HAL-Université de Bretagne Occidentale

HAL UVSQ

HAL-CEA

HAL-INSU

Archive Ouverte en Sciences de l'Information et de la Communication