ABSTRACT In this work we demonstrate the utility of stochastic empirical models. Linear stochastic models have been successful at reproducing responses to forcing of large-scale atmospheric flow. However, the linear forms of the models reach their limit on highly nonlinear problems. It is shown here that for simple low dimensional problems, a quadratically nonlinear empirical model is successful at reproducing a chaotic attractor. In addition, empirical models can be used to diagnose the important dynamics of a flow system and identify the essential terms to include in a forward dynamical model. Once the empirical model has been built, it can then be used to diagnose the underlying dynamics. INTRODUCTION Numerical modeling of unsteady flows has traditionally involved using some type of time stepping combined with known dynamics discretized from a partial differential equation. However, sometimes the details of the dynamics are not sufficiently known or we wish to develop a model that reproduces dynamic behavior without involving the details of the full equations of motion. In such cases, we can substitute an empirical model based on observed data. These stochastic empirical models are built from measured or simulated data and are based on a Markhov model. Given a sufficient amount of data to develop the model, the discretized dynamics of the traditional model can be replaced by a matrix of computed values that serve as a propagator matrix. Empirical models have gained popularity in recent years as an alternative to the more traditional dynamical models. They have proven powerful in many scientific and engineering areas to simulate and predict the behavior of dynamical systems. In contrast to the traditional dynamical models that are based on first principles, empirical models are based on data. Thus, they are stochastic in nature and hav

Sue Ellen Haupt

CiteSeerX

 
 
 
 
 
 
 
 
Downloaded From: https://proceedings.asmedigitaProceedings of FEDSM’03
4TH ASME_JSME Joint Fluids Engineering Conference
Honolulu, Hawaii, USA, July 6–11, 2003
FEDSM2003-45761
EMPIRICAL MODELING OF UNSTEADY FLOWS 
Sue Ellen Haupt 
Department of Mechanical and Aerospace Engineering 
Utah State University 
Logan, UT 84322-4130 
Proceedings of ASME FEDSM’03 
4th ASME_JSME Joint Fluids Engi ing Confer nce 
Honolulu, Hawaii, U , ly 6-10, 2003 
 
 
F E DSM200 3-4 5761 
 
 
 
ABSTRACT 
 In this work we demonstrate the utility of stochastic 
empirical models. Linear stochastic models have been 
successful at reproducing responses to forcing of large-scale 
atmospheric flow.  However, the linear forms of the models 
reach their limit on highly nonlinear problems.  It is shown here 
that for simple low dimensional problems, a quadratically 
nonlinear empirical model is successful at reproducing a 
chaotic attractor.  In addition, empirical models can be used to 
diagnose the important dynamics of a flow system and identify 
the essential terms to include in a forward dynamical model.    
Once the empirical model has been built, it can then be used to 
diagnose the underlying dynamics. 
 
 
INTRODUCTION 
Numerical modeling of unsteady flows has 
traditionally involved using some type of time stepping 
combined with known dynamics discretized from a partial 
differential equation.  However, sometimes the details of the 
dynamics are not sufficiently known or we wish to develop a 
model that reproduces dynamic behavior without involving the 
details of the full equations of motion.  In such cases, we can 
substitute an empirical model based on observed data.  These 
stochastic empirical models are built from measured or 
simulated data and are based on a Markhov model.  Given a 
sufficient amount of data to develop the model, the discretized 
dynamics of the traditional model can be replaced by a matrix 
of computed values that serve as a propagator matrix.   
Empirical models have gained popularity in recent 
years as an alternative to the more traditional dynamical 
models. They have proven powerful in many scientific and 
engineering areas to simulate and predict the behavior of 
dynamical systems.  In contrast to the traditional dynamical 
models that are based on first principles, empirical models are 
based on data.  Thus, they are stochastic in nature and have lcollection.asme.org on 06/29/2019 Terms of Uproven quite good at predicting such difficult phenomena as 
flow characteristics and weather and climate behavior [1-4]. It 
is interesting to note that the empirical model of Penland [5,6] 
predicted the El Nino of 1997-98 before many of the well-
crafted dynamical models based on the most recent physical 
parameters. They have also been shown to reproduce a 
response to imposed forcing better than the traditional 
linearized dynamical models [3]. It is important that the 
analysis of these models can identify specific terms in the 
dynamical equations and whether they are over- or under-
specified for the application [3]. Linear empirical models are 
easy to produce from data using standard least squares 
inversion techniques demonstrated in the following section. 
However, for highly nonlinear fluid dynamics 
problems, linear models are no longer adequate [2]. In these 
cases, a nonlinear term is often necessary to capture the 
dynamics. Since in fluid dynamics nonlinearity often enters as 
the quadratic advective term, the nonlinearity of the empirical 
model is also likely to be of quadratic form. Unfortunately, 
nonlinear models are more difficult to devise due to the 
introduction of higher order tensors to the problems.  We 
resolve this issue through redefining the problem in terms of 
optimization and directly searching for the propagator matrix 
given the data using an artificial intelligence technique such as 
a genetic algorithm (GA). 
 
 
SOLUTION METHODS 
 
Linear Empirical Models 
 We consider a linear, time-varying model that can be 
written in the form: 
 
1 Copyright © 2003 by ASME 
se: http://www.asme.org/about-asme/terms-of-use
Downξ
vv
r
+= sB
dt
sd
                  (1) 
 
where: sv  is the N-dimensional state anomalies (the time  
mean has been subtracted) and can represent  
states such as velocities at various locations  
or the spectral coefficients of the velocity 
 
dt
sdv
 is the time rate of change of the state 
 B  is a linear NN × matrix that relates the  
above two 
 ξ
v
 is a vector of white noise (denoting the error in the  
fit) 
 
The dynamics are contained in the matrix, B.  Nonlinearities 
are parameterized by corrections to B as well as within the 
noise portion of ξ. This simple linear form is easily fit using 
standard analytical techniques to minimize the least square 
error between the model and time series data. These techniques 
involve minimizing the square of the error between the model 
and the time averaged data tendencies. Specifically, we wish to 
minimize the noise vector (equivalently, the error). We assume 
that sv behaves as a Markov process. We can minimize the error 
in a least square sense by requiring that: 
 













 −=
2
sB
dt
sdE v
r
 is minimized                  (2) 
The parentheses represent a spatial averaging while the angle 
brackets represent an ensemble time average.The solution to 
this problem involves finding where the derivative vanishes.  In 
doing this, one computes the covariance and lagged covariance 
matrices as: 
)()( tsts T=Λ  is the covariance matrix, averaged  
over time.  The superscript (T) indicates a  
matrix transpose. 
)()( tsts Tττ +=Λ  is the lagged covariance  
matrix.   
 τ  is the chosen lag time. 
    
Upon solving the least squares problem, we find: 
ττ /ln 





Λ
Λ
=B                                                (3) 
or in terms of an propagator equation, we can write 
 
 )()()( tsGts ττ =+                                          (4) 
where: 
 )exp()( ττ BG =                         (5) 
 
Note that the model depends on the value of the lag, τ , the 
amount of data used to build it,  and the resolution of the data. 
Such linear empirical models often compete well with 
linearized dynamical models in reproducing the statistics of the  
loaded From: https://proceedings.asmedigitalcollection.asme.org on 06/29/2019 Terms of Usmodeled field.  In addition, much can be learned about the flow 
dynamics. 
 
 
Nonlinear Empirical Models 
When the dynamics are highly nonlinear, stochastic 
empirical models can also be formulated using nonlinear 
dynamics.  For instance, a quadratic empirical model could 
have the form:  
ξ++= BssCss Tt               (6) 
 
The nonlinear interactions now occur explicitly through the 
nonlinear third order tensor operator, C.  Thus B is an N x N 
matrix that serves as the linear propagator, C is an N x N x N 
third order tensor that gives the coefficients of the quadratic 
interactions, and ξ  is the constant noise vector.  We wish to 
compute the matrices B and C so that the least square error of 
(12), 
 
( )2sCsBssE Tt −−=  is minimized           (7) 
is minimized.  The angle brackets denote a time average.  
Minimizing E with respect to B and C gives the system of 
equations: 
 
1
43 0
−−=
=+
sssssCssB
CTT
t
           (8) 
 
where: 
 
ssssssssssssT
ssssssssssT tt
−=
−=
−
−
1
4
1
3
                       (9) 
 
Although this is a closed form solution, to compute the third 
order tensor C requires inverting the fourth order tensor 4T .  
Such an inversion is not trivial.  Therefore, we choose to 
instead compute C in equation (8) by doing a best fit with a 
genetic algorithm. 
 
Genetic Algorithms 
The flow chart in Figure 1 provides a “big picture'' 
overview of a continuous genetic algorithm, that is, one in 
which the parameters are real numbers.  The parameters are the 
genes which are strung together in a one-dimensional array 
known as a chromosome.  The GA begins with a population of 
chromosomes which are fed to the cost function for evaluation.  
The fittest chromosomes survive while the highest cost ones die 
off. This process mimics natural selection in the natural world.  
The lowest cost survivors mate.  The mating process combines 
information from the two parents to produce two offspring. 
Some of the population experiences mutations. The mating and 
mutation operators introduce new chromosomes which may 
have a lower cost than the prior generation. The process iterates 
until an acceptable solution is found.  More specific description 
of the process follows. 2 Copyright © 2003 by ASME 
e: http://www.asme.org/about-asme/terms-of-use
Down 
Random Initial Population
Natural Selection
Mating
Mutation
Convergence Check
of Continuous Parameters
done  
Figure 1. Flow chart of a continuous genetic algorithm. 
 
 
  As seen in the figure, the first step of a continuous 
parameter genetic algorithm is creating the population of 
chromosomes.  First, the real parameters are concatenated 
together into a chromosome as:  
 [ ]
parN
ppppchromosome LL α21=             (10) 
where the ip  are the parameters and there are a total of 
parN parameters. The parameters are simply floating point 
numbers.  The encoding function needs only keep track of 
which digits represent which parameters and to make sure they 
are within given bounds.  A population of such chromosomes is 
created using a random number generator so that the 
chromosome arrays are gathered together in a two dimensional 
matrix. 
     Once the chromosomes have been created, their cost or 
fitness must be evaluated.  This is done by the cost or objective 
function, which is very problem specific.  The lowest cost 
chromosomes ( keepN ) remain in the population while the 
higher ones are deemed less fit and die off.  The reduced 
population is then the portion of the population available for 
mating. 
     There are a variety of methods to pair the chromosomes for 
mating.  Some popular methods are reviewed by Haupt and 
Haupt [7].  Here, we choose to pair the chromosomes according 
to numerical rank.  After the cost function evaluation, the 
chromosomes are sorted in order from lowest cost to highest.  
That is, the nth chromosome will have a probability of mating 
of: 
 
 
loaded From: https://proceedings.asmedigitalcollection.asme.org on 06/29/2019 Terms of Us∑
=
+−
=
keepN
k
keep
n
k
nN
P
1
1
                                          (11) 
Then the cumulative probabilities are used for selecting which 
chromosomes mate. 
       Once two parents are chosen, some method must be 
devised to produce offspring which are some combination of 
these parents. Many different approaches have been tried for 
crossing over in continuous parameter genetic algorithms [7,8].. 
 The method used here is a combination of an 
extrapolation method with a crossover method. We wanted to 
find a way to closely mimic the advantages of the binary 
genetic algorithm mating scheme. It begins by randomly 
selecting a parameter in the first pair of parents to be the 
crossover point.  
{ }parNrandomroundup ×=α                       (12) 
 
We'll let  
 [ ]
[ ]
par
par
dNddd
mNmmm
ppppparent
ppppparent
LL
LL
α
α
212
211
=
=
                    (13) 
 
where the m and d subscripts discriminate between the mom 
and the dad parent. Then the selected parameters are combined 
to form new parameters that will appear in the children: 
 
[ ]
[ ]ααα
ααα
β
β
dmdnew
dmmnew
pppp
pppp
−−=
−−=
2
1                                 (14) 
 
where β  is also a random value between 0 and 1. The final 
step is to complete the crossover with the rest of the 
chromosome as before: 
 [ ]
[ ]
par
par
mNnewdd
dNnewmm
ppppoffspring
ppppoffspring
LL
LL
2212
1211
=
=
              (15) 
 
If the first parameter of the chromosomes is selected, then only 
the parameters to right of the selected parameter are swapped. 
If the last parameter of the chromosomes is selected, then only 
the parameters to the left of the selected parameter are 
swapped. This method does not allow offspring parameters 
outside the bounds set by the parent unless β  is greater than 
one.  In this way, information from the two parent 
chromosomes is combined a way that mimics the crossover 
process during meiosis. 
     If care is not taken, the genetic algorithm converges too 
quickly into one region of the cost surface. If this area is in the 
region of the global minimum, that is good. However, some 
functions have many local minima and the algorithm could get 
stuck in a local well. If we do nothing to solve this tendency to 
converge quickly, we could end up in a local rather than a 
global minimum. To avoid this problem of overly fast 
convergence, we force the routine to explore other areas of the 
cost surface by randomly introducing changes, or mutations, in 3 Copyright © 2003 by ASME 
e: http://www.asme.org/about-asme/terms-of-use
Downsome of the parameters. A mutated parameter is replaced by a 
new random parameter. 
 
 
 
EXAMPLE  – LORENZ EQUATIONS 
     Empirical modeling with nonlinear terms is demonstrated 
for matching low order nonlinear systems, such as the Lorenz 
equations [9].  These equations model the nonlinear dynamics 
of the coefficients of the three lowest order modes when 
atmospheric flow equations are expanded in a Fourier series. 
These equations can be written: 
          
xybzz
xzyxy
yxx
+−=
−−=
+−=
&
&
&
ρ
σσ
            (16) 
where x, y, z are the lowest order coefficients of a truncated 
series of atmospheric flow and we use parameters: 
28,3
8,10 === ρσ b  These parameters produce a 
chaotic regime that results in a strange attractor.  The equations 
(16) were integrated using a fourth order Runge-Kutta method 
to produce the data in Figure 2. 
 
Figure 2.  A Lorenz attractor computed by integrating equations 
(16) in time for 2000 steps. 
 
 
     The parameters of the GA are an initial population of 
500, working population of 100, mutation rate of 0.3 for a total 
of 200 generations.  Taking into account symmetries for this 
problem results in 18 unique parameters to find.  For this highly 
nonlinear regime, we were able to find a solution, which when 
propagated via equation (6), produces the time evolution 
depicted in Figure 3.  Although the match is not perfect, we 
have replicated the general shape of the strange attractor and 
the size of the domain is approximately correct. 
For comparison, the solution is compared to a linear 
model fit (1), which is merely the linear part of (6).  For this 
portion, there is a simple closed form solution that is easily 
computed.  The linear match is shown in Figure 4.  We see that 
the linear model is not able to capture the shape of the attractor, 
but instead shows a decaying spiral behavior.  Note that decay 
is an essential feature of the linear inverse model since, by 
construction, all eigenvalues of matrix B must be negative.  
This is not to say that the linear model is not useful.  For  
loaded From: https://proceedings.asmedigitalcollection.asme.org on 06/29/2019 Terms of Usufficiently long e-folding times, the basic dynamics can be 
captured for linear systems. 
 
  
 
 
Figure 3.  Nonlinear model of Lorenz attractor (equation 6) as 
computed with a GA. 
 
 
      
 
Figure 4.  Linear model of the Lorenz equations. 
 
 
DISCUSSION 
     Empirical models show promise as a new technieque to 
build models of data.  Here we have given an example of a 
problem with known behavior where they have performed 
better than alternative techniques.   
           The example problem demonstrates that for nonlinear 
systems that cannot be modeled adequately by the linear form 
of the empirical model, even chaotic attractors can be captured 
through application of a nonlinear empirical model when 
coupled with the artificial intelligence methodology of genetic 
algorithms. The nonlinear model (6) when combined with a GA 
reproduced the shape of the attractor to the Lorenz equations 
reasonably well compared to the essential decary of the 
linearized method. Thus, although the linear model did not 
adequately represent the data, the nonlinear model was able to 
capture the general form of the attractor.  For a chaotic system, 
we do not expect to reproduce the exact time behavior and are 
rather pleased to see the form of the attractor near to that of the 
actual data used to create the model. 
     Empirical models are not perfect. However, they show 
great ability at not only reproducing the behavior of complex 
models [1-3], but also serve to predict future behavior better 4 Copyright © 2003 by ASME 
se: http://www.asme.org/about-asme/terms-of-use
 
 
Downlothan more simplified models and reproduce a response to 
forcing better than linearizxed forward dynamical models [3]. 
Such models are only beginning to be explored. They show 
promise in helping to understand the dynamics of flows 
difficult to diagnose, such as through machinery and 
microchannel flow. 
 
ACKNOWLEDGMENTS 
Elements of this work were inspired through discussions 
with colleagues and students, including Grant Branstator, 
Jeffrey Weiss, and Christina Perez. 
 
REFERENCES 
 
[1] Hasselman, K., 1988:  PIPs and POPs: The reduction of 
complex dynamical systems using principal interaction and 
oscillation patterns. J. Geophys. Res., 93, 11015-11021. 
 
[2] Penland, C., 1989: Random forcing and forecasting using 
principal oscillation pattern analysis. Mon. Weather Rev., 117, 
2165-2185. 
 
[3] Branstator, G. and S.E. Haupt, 1998: An empirical model of 
forced barotropic atmospheric anomalies, J. Climate, 11, 2645-
2667.  
 
aded From: https://proceedings.asmedigitalcollection.asme.org on 06/29/2019 Terms of Use 
[4] Penland, C. and M. Ghil, 1993:  Forecasting northern 
hemisphere 700mb geopotential height anomalies using 
empirical normal modes. Mon. Weather Rev., pp. 
[5] Penland, C. and T. Magorian, 1993:  Prediction of NINO3 
sea-surface temperatures using linear inverse modeling., J. 
Climate, pp. 
[6] Penland, C. and L. Matrosova, 1994: A balance condition 
for stochastic numerical models with application to the El 
Nino-Southern Oscillation, J. Climate, 7, pp. 1352-1372. 
[7] Haupt, R.L. and S.E. Haupt, 1998:  Practical Genetic 
Algorithms, New York: John Wiley & Sons, 177 pp. 
 
[8] Adewuya, A.A., 1996. New Methods in Genetic Search 
with Real-Valued Chromosomes, Master’s Thesis, Cambridge, 
Massachusetts Institute of Technology. 
 
[9] Lorenz, E.N., 1963: Deterministic Nonperiodic Flow, J. 
Atmos. Sciences, 20, 130-141. 
 
5 Copyright © 2003 by ASME 
: http://www.asme.org/about-asme/terms-of-use


FEDSM2003-45761 EMPIRICAL MODELING OF UNSTEADY FLOWS

http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=5132F6F7CC2AF50EAC3216DA03B81FC8?doi=10.1.1.1074.674&rep=rep1&type=pdf

FEDSM2003-45761 EMPIRICAL MODELING OF UNSTEADY FLOWS

Abstract

Similar works

Full text

Available Versions

CiteSeerX