To interpolate data which is sampled in finite, discrete time steps into a continuous signal e.g. for resampling, normallya model has to be introduced for this purpose, like linear interpolation, splines, etc. In this paper we attemptto derive a natural method of interpolation, where the correct model is derived from the data itself, using somegeneral assumptions about the underlying process. Applying the formalism of generalized iteration, iteration semigroupsand iterative roots we attempt to characterize a method to find out if such a natural interpolation for agiven time series exists and give a method for its calculation, an exact one for linear autoregressive time seriesand a neural network approximation for the general nonlinear case

Kindermann, Lars

Lewandowski, A.

English

Electronic Publication Information Center

Natural Interpolation of Time SeriesLars Kindermann Achim LewandowskiRIKEN Brain Science Institute, Japan ÖFAILab for Mathematical Neuroscience Austrian Research Institute for Artificial Intelligence   lars.kindermann@reglos.de   achim@oefai.at          www.reglos.de/kindermann         www.oefai.atAbstractTo interpolate data which is sampled in finite, discrete time steps into a continuous signal e.g. for resampling, nor-mally a model has to be introduced for this purpose, like linear interpolation, splines, etc. In this paper we attemptto derive a natural method of interpolation, where the correct model is derived from the data itself, using somegeneral assumptions about the underlying process. Applying the formalism of generalized iteration, iteration sem-igroups and iterative roots we attempt to characterize a method to find out if such a natural interpolation for agiven time series exists and give a method for it’s calculation, an exact one for linear autoregressive time seriesand a neural network approximation for the general nonlinear case.1 IntroductionLet  be an auto regressive time series: . We will not deal here with finding ,i.e. predicting the time series, instead we assume is already known or already approximately derived from thegiven data. We will attempt to embed the discrete series of ,  into a continuous function ,. To clarify the idea we present the method at first for the case that the timeseries is generated totally deter-ministically ( ) by an underlying autonomous dynamical system. Later we will consider the influences ofadditional external inputs and noise. The time evolution of any autonomous dynamical systems is represented by a solution of the translation equation[1], (1) where  is an state vector representing an initial condition and  are arbitrary time intervals. For continuoustime dynamical systems this equation holds for every positive . If we assume the given time series is a discretetime sampling of an underlying continuously evolving signal, we have to solve (1) under the conditions and , where  is the discrete time mapping represented by the data. (Without loss ofgenerality we can assume the sampling rate of the discrete time data to be one, which will result in a nice and veryintuitive formalism.) To double the sampling rate for example, (1) becomes . Substituting  weget , the functional equation for the iterative root of the mapping  [3]. By writing formally  the connection to iteration theory becomes visible: Time evolution of dis-crete time systems can be regarded as the iteration of a time step mapping function (iterative map) and this con-cept extends to continuous time dynamical systems by means of generalized or continuous iteration, allowing fornon-integer iteration counts. The following mathematical problems appear [3,4]:• For a given function , does there exist the iteration semigroup ?, • Is the solution unique?• How to calculate it explicitely or numerically.To apply this theory usually,  has to be a complete state vector of the dynamical system. This means that  hasto be a function of the last state only: . When  also depends on earlier values of the time series, in order to obtain a self-mapping we introduce the function :  which maps the vector xt xt f xt 1– xt 2– … xt n–, , ,( ) εt+=  f  f xt t 0 1 2 …, , ,= x t( )t R+∈εt 0=Φ x0 t1 t2+,( ) Φ Φ x0 t1,( ) t2,( )=x0 t1 t2,tΦ x 0,( ) x= Φ x 1,( ) f x( )=  f f x( ) Φ Φ x 12--,   12--,  = ϕ x( ) Φ x12--,  ≡ϕ ϕ x( )( ) f x( )= fΦ x t,( ) f t x( )≡ f  f tx  fxt f xt 1–( )=  fxt 2– … xt n–, , F Rn Rn→RIKEN BRAIN SCIENCE INSTITUTE BSIS TECHNICAL REPORTSNo. 03-3, May 2003 - Rev 2, Apr 2004http://www.brain.riken.jp/labs/bsis/TR/03-3.pdf towith . Except for the first element this is a trivial time shift operation, each elementof  is just replaced by its successor. But because  is a self mapping within a -dimensional space now, timedevelopment can be calculated by iterating  and we can try to find the generalized iteration with non-integeriteration counts to find a time continuos embedding , the continuous iteration semigroup of  and extract afunction  from this [2]. 2 Linear CaseThe idea is best demonstrated for the linear case, where it’s application simplifies and unifies several problems.For a linear autoregessive time series AR(n) model with , the mapping  can be written as asquare matrix with the coefficients  in the first row and the lower subdiagonal filled with ones. Then we can compute and the discrete time evolution of the the system can now be calculated using the matrix powers. This autoregressive system is called linear embeddable if the matrix power  exists also for all real . Thisis the case if   can be decomposed into  with  being a diagonal matrix consisting of the eigen-values  of  and  being an invertible square matrix which columns are the eigenvectors of . Additionallyall  must be non-negative to have a linear and real embedding, otherwise we will get a complex embedding. Then we can obtain  with . Now we have a continuous function  and the interpolation of the original time series  consistsof the first element of .In case there is also a constant term, i.e. the mean is not zero, , we just have to append a con-stant one to all the vectors  and take.A special case is  , the square root of a matrix, which solves the matrix equation . It resem-bles the iterative root of linear functions and corresponds to a doubling of the sampling rate.A few lines of Maple code can automate this procedure both for symbolic and numeric expressions. A sampleworksheet is available at the authors webpage.xt 1– xt 1– … xt n–, ,=xt xt xt 1– … xt n 1–( )–, , ,=xt f xt 1– xt 2– … xt n–, , ,( )=x F nFFt Fx t( )xt akxt k–k 1=n∑= FFa1 a2 … an  1 0 0 0  0 1 0 0  0 0 1 0  =akxt F xt 1–⋅=xt n+ Fnxt 1–⋅=Ft t R+∈F F S A S 1–⋅ ⋅= Aλi F S FλiFt S At S 1–⋅ ⋅= Atλ1t 0 00 … 00 0 λnt=x t( ) Ft x0⋅= x t( )xxt akxt k–k 1=n∑ b+=xt xt xt 1– … xt n 1–( )– 1, , , ,=Fa1 a2 … an b1 0 0 0 00 1 0 0 00 0 1 0 00 0 0 0 1=F1 2⁄ F1 2⁄ F1 2⁄⋅ F=3 ExamplesWe will now provide some simple examples to demonstrate this formalism.3.1 One dimensional linear caseThe time series given by some  and  simply doubles every time step. The natural interpolation weimmediately get by applying the former formalism in the trivial one dimensional case is , which is ofcourse exactly what we expect: exponential growth. But a little change makes the problem much more difficult: If we expect a mixture of constant and exponential growth, but what is the exact continuous law? Take  then the series is generated by . We get immediately by eigenvalue decompositionand for the first component  (which equals  for ).We don’t have to consider about stability or stationarity of the AR(1) model here but note that to obtain a com-pletely real valued function ,  has to be positive. Later we will discuss about the meaning of such cases withcomplex embeddings, but for short it means that there is no one-dimensional continuous time dynamical systemthat can generate such timeseries. In the linear case this should be clear because a negative  implies oscillatorybehavior of . This means, some initial condition  won’t be enough to determine the continuation of the tra-jectory, it could be on the rising or falling slope. The underlying dynamical system needs to have one more hiddendimension to allow embedding. The other dimension can be represented by the imaginary part of  which willvanish at all integer times . But taking only the real part will still result in a valid interpolation of the givenseries, the observable of the system.This is such an embedding of the AR(2) process  with . Circles mark thetime series , the left graph shows the real part of , our natural interpolation, the imaginary part is on theright.3.2 Two dimensional linear caseThe well known Fibonacci series , ,  can be generated in this manner by and . By eigenvalue decomposition of  we getx0 xt 2xt 1–=x t( ) 2tx0=xt axt 1– b+=xt xt 1,[ ]= F a b0 1=x t( ) Ftx0 SAtS 1– x0 = =1 11 a–b----------- 01 00 at0 b1 a–-----------1 ba 1–-----------x01=x t( ) atx0 bat 1–a 1–------------+= ax0 b+ t 1→x t( ) aax t( ) x0x t( )txt12--xt 1––13--xt 2–12--+ += x0 x1 1= =xt x t( )x0 0= x1 1= xt xt 1– xt 2–+=F 1 11 0= x1 1 0,[ ]= Fxt 1+ Ftx1 SAtS 1– x1 = =1 5+2---------------1 5–2---------------1 11 5+2---------------  t00 1 5–2---------------   t15------12--12 5---------–15------–12--12 5---------+10=which turns out to evaluate exactly to Binet’s famous formula for the Fibonacci series in the first component [5]. Because the second eigenvalue is negative, a real linear continuous time embedding does not exist and  takescomplex values on non-integer . Figure 2 shows real and complex part of . 4 Nonlinear CaseSo far we considered only linear dependencies of the past  which could easily be mapped to matrix expressions.The problem becomes much more complicated if we allow for arbitrary . Even for one dimension this cannot besolved analytically in the general case, so we use neural networks to compute approximations for fractional iter-ates of arbitrary functions [7].4.1 One dimensional nonlinear systems The general solution of the real valued translation equation  with  being continuous and strictly monotonic in  and  is given by  . If the discretetime mapping  is given, this is Abel’s functional equation . If  has a fixpoint this can further be transformed to Schröder’s functional equation , the eigenvalueproblem for nonlinear functions. In those cases when either of these equations can be solved for the unknownfunction , it is immediately possible to obtain the embedding by  [2].This can be easily solved for example in the linear case: If we take  and initial value  the time stepmapping is , we get the Abel type functional equation  or ,which is solved by . So we get for the continuous embedding the expected result again, exponentialgrowth.However, this analytical method is limited to a small selection of functions and it can be shown that there existembeddings for a much wider range of mappings which cannot be calculated analytically yet. Furthermore thetheory so far is developed mainly for real or complex valued functions, solving Abel or Schröder type functionalequations in higher dimensions is in general beyond reach.But simple neural networks can be used to find precise approximations for those embeddings as well [7,8,9].Take , the iterated logistic map, which is a favorite textbook example for the emergence ofchaotic behavior within a simple dynamical system. However, this is a discrete time system, so the questionshould arise naturally if it is possible to embed the  into continuous trajectories  which now obey the func-tional equation  for any non-integer . Or even more general, is there any continuoustime system that takes the same values at integer times? Iteration theory proofs that the answer is no if  [6], but this could be expected also by the theorem of Poincare-Bendixon, which implies that chaotic behavior isimpossible in continuous time systems of less than three dimensions. To obtain a continuous embedding of thisseries, we again have to introduce some hidden dimension, like allowing complex values for x, in iteration theorythese generalized solutions are called phantom roots of functions [1].4.2 Multidimensional nonlinear systemWe took a time series of yearly snapshots from a discrete non linear Lotka-Volterra type predator - prey system (x= hare, y = lynx) as training data. xt15------1 5+2---------------  t 1 5–2---------------  t–=x t( )x x t( ) f fΦ x0 t1 t2+,( ) Φ Φ x0 t1,( ) t2,( )= Φ: R R R→×Φ x t Φ x t,( ) ϕ 1– ϕ x( ) t+( )=Φ x 1,( ) f x( )=  f x( ) ϕ 1– ϕ x( ) 1+( )= f f x( ) ϕ 1– cϕ x( )( )=ϕ  f t x( ) ϕ 1– ϕ x( ) t+( )=xt axt 1–= x0 f x( ) ax= ax ϕ 1– ϕ x( ) 1+( )= ϕ ax( ) ϕ x( ) 1+=ϕ x( ) xalog= f t x( ) a xa 0log t+ x0at= =xt 1+ 4λxt 1 x– t( )=xt x t( )x t 1+( ) 4λx t( ) 1 x t( )–( )= tλ 3 2⁄>From these samples only we calculated the monthly population by use of a neural network based method to com-pute iterative roots and fractional iterates with a pseudo newton algorithm [9].This figure shows the yearly training data as circles and the interpolated monthly values. Additionally the fore-casted values for the next 12 months are shown, together with the true value after 1 year which was excluded frommodel fitting.The given method provides a natural way to estimate not only the values over a year, but also to extrapolate arbi-trarily smooth into the future.5 Discussion and OutlookThe method demonstrates that there is a close relation between prediction and interpolation. A necessary condi-tion for the existence of a natural interpolation of a time series is predictability. If there are random influences andwe require that the values  coincide with  for integer , we can still use the embedding function to get anear fit and add an additional interpolation method for the residuals . This has again to be selected freelyof course, but it minimizes the amount of arbitrariness involved in interpolating. References1. G. Targonski: Topics in Iteration Theory. Vandenhoeck und Ruprecht, Göttingen (1981) 2. M.C. Zdun: Continuous iteration semigroups. Boll. Un. Mat. Ital. 14 A (1977) 65-703. M. Kuczma, B. Choczewski & R. Ger: Iterative Functional Equations. Cambridge University Press, Cambridge(1990)4. K. Baron & W. Jarczyk: Recent results on functional equations in a single variable, perspectives and open prob-lems. Aequationes Math. 61. (2001), 1-485. R.L. Graham, D.E. Knuth & O. Patashnik: Concrete Mathematics. Addison-Wesley, Massachusetts (1994) 6. R.E. Rice, B. Schweizer & A. Sklar: When is f(f(z)) = az2+bz+c for all complex z? Amer. Math. Monthly 87(1980) 252-2637. L. Kindermann: Computing Iterative Roots with Neural Networks. Proc. Fifth Conf. Neural Information Pro-cessing, ICONIP (1998) Vol. 2:713-7158. E. Castillo, A. Cobo, J.M Gutiérrez & R.E Pruneda: Functional Networks with Applications. A Neural-BasedParadigm. Kluwer Academic Publishers, Boston/Dordrecht/London (1999)9. L. Kindermann & A. Lewandowski: A Comparison of Different Neural Methods for Solving Iterative Roots.Proc. Seventh Int’l Conf. on Neural Information Processing, ICONIP, Taejon (2000) 565-569xt 1+ 1 a b yt–+( ) xt=yt 1+ 1 c– d xt+( ) yt=0 2 4 6 8 10 1201234567PreyPredatorx t( ) xt txt x t( )–

Natural Interpolation of Time Series

http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.15.8663

Natural Interpolation of Time Series

Abstract

Similar works

Full text

Available Versions

Electronic Publication Information Center