Efficient likelihood evaluation of state-space representations

We develop a numerical procedure that facilitates efficient likelihood evaluation in applications involving non-linear and non-Gaussian state-space models. The procedure approximates necessary integrals using continuous approximations of target densities. Construction is achieved via efficient importance sampling, and approximating densities are adapted to fully incorporate current information. We illustrate our procedure in applications to dynamic stochastic general equilibrium models. --particle filter,adaption,efficient importance sampling,kernel density approximation,dynamic stochastic general equilibrium model

Efficient Likelihood Evaluation of State-Space Representations

We develop a numerical procedure that facilitates efficient likelihood evaluation in applications involving non-linear and non-Gaussian state-space models. The procedure employs continuous approximations of filtering densities, and delivers unconditionally optimal global approximations of targeted integrands to achieve likelihood approximation. Optimized approximations of targeted integrands are constructed via efficient importance sampling. Resulting likelihood approximations are continuous functions of model parameters, greatly enhancing parameter estimation. We illustrate our procedure in applications to dynamic stochastic general equilibrium models.Adaption, dynamic stochastic general equilibrium model, efficient importance sampling, kernel density approximation, particle filter.

Quasi maximum likelihood estimation for strongly mixing state space models and multivariate L\'evy-driven CARMA processes

We consider quasi maximum likelihood (QML) estimation for general non-Gaussian discrete-ime linear state space models and equidistantly observed multivariate L\'evy-driven continuoustime autoregressive moving average (MCARMA) processes. In the discrete-time setting, we prove strong consistency and asymptotic normality of the QML estimator under standard moment assumptions and a strong-mixing condition on the output process of the state space model. In the second part of the paper, we investigate probabilistic and analytical properties of equidistantly sampled continuous-time state space models and apply our results from the discrete-time setting to derive the asymptotic properties of the QML estimator of discretely recorded MCARMA processes. Under natural identifiability conditions, the estimators are again consistent and asymptotically normally distributed for any sampling frequency. We also demonstrate the practical applicability of our method through a simulation study and a data example from econometrics

Filtering and Smoothing with Score-Driven Models

We propose a methodology for filtering, smoothing and assessing parameter and filtering uncertainty in misspecified score-driven models. Our technique is based on a general representation of the well-known Kalman filter and smoother recursions for linear Gaussian models in terms of the score of the conditional log-likelihood. We prove that, when data are generated by a nonlinear non-Gaussian state-space model, the proposed methodology results from a first-order expansion of the true observation density around the optimal filter. The error made by such approximation is assessed analytically. As shown in extensive Monte Carlo analyses, our methodology performs very similarly to exact simulation-based methods, while remaining computationally extremely simple. We illustrate empirically the advantages in employing score-driven models as misspecified filters rather than purely predictive processes.Comment: 33 pages, 5 figures, 6 table

Smooth particle filters for likelihood evaluation and maximisation

In this paper,a method is introduced for approximating the likelihood for the unknown parameters of a state space model.The approximation converges to the true likelihood as the simulation size goes to infinity. In addition,the approximating likelihood is continuous as a function of the unknown parameters under rather general conditions.The approach advocated is fast, robust and avoids many of the pitfalls associated with current techniques based upon importance sampling.We assess the performance of the method by considering a linear state space model, comparing the results with the Kalman filter, which delivers the true likelihood. We also apply the method to a non-Gaussian state space model, the Stochastic Volatility model, finding that the approach is efficient and effective. Applications to continuous time finance models are also considered. A result is established which allows the likelihood to be estimated quickly and efficiently using the output from the general auxiliary particle filter

Smooth Particle Filters for Likelihood Evaluation and Maximisation

In this paper, a method is introduced for approximating the likelihood for the unknown parameters of a state space model. The approximation converges to the true likelihood as the simulation size goes to infinity. In addition, the approximating likelihood is continuous as a function of the unknown parameters under rather general conditions. The approach advocated is fast, robust and avoids many of the pitfalls associated with current techniques based upon importance sampling. We assess the performance of the method by considering a linear state space model, comparing the results with the Kalman filter, which delivers the true likelihood. We also apply the method to a non-Gaussian state space model, the Stochastic Volatility model, finding that the approach is efficient and effective. Applications to continuous time finance models are also considered. A result is established which allows the likelihood to be estimated quickly and efficiently using the output from the general auxilary particle filter. keywords: Importance Sampling ; Filtering ; Particle filter ; Simulation ; SIR ; State space

Sequential Monte Carlo methods for data assimilation in strongly nonlinear dynamics.

Data assimilation is the process of estimating the state of dynamic systems (linear or nonlinear, Gaussian or non-Gaussian) as accurately as possible from noisy observational data. Although the Three Dimensional Variational (3D-VAR) methods, Four Dimensional Variational (4D-VAR) methods and Ensemble Kalman filter (EnKF) methods are widely used and effective for linear and Gaussian dynamics, new methods of data assimilation are required for the general situation, that is, nonlinear non-Gaussian dynamics. General Bayesian recursive estimation theory is reviewed in this thesis. The Bayesian estimation approach provides a rather general and powerful framework for handling nonlinear, non-Gaussian, as well as linear, Gaussian estimation problems. Despite a general solution to the nonlinear estimation problem, there is no closed-form solution in the general case. Therefore, approximate techniques have to be employed. In this thesis, the sequential Monte Carlo (SMC) methods, commonly referred to as the particle filter, is presented to tackle non-linear, non-Gaussian estimation problems. In this thesis, we use the SMC methods only for the nonlinear state estimation problem, however, it can also be used for the nonlinear parameter estimation problem. In order to demonstrate the new methods in the general nonlinear non-Gaussian case, we compare Sequential Monte Carlo (SMC) methods with the Ensemble Kalman Filter (EnKF) by performing data assimilation in nonlinear and non-Gaussian dynamic systems. The models used in this study are referred to as state-space models. The Lorenz 1963 and 1966 models serve as test beds for examining the properties of these assimilation methods when used in highly nonlinear dynamics. The application of Sequential Monte Carlo methods to different fixed parameters in dynamic models is considered. Four different scenarios in the Lorenz 1063 [sic] model and three different scenarios in the Lorenz 1996 model are designed in this study for both the SMC methods and EnKF method with different filter sizThe original print copy of this thesis may be available here: http://wizard.unbc.ca/record=b160051

Additive, Dynamic and Multiplicative Regression

We survey and compare model-based approaches to regression for cross-sectional and longitudinal data which extend the classical parametric linear model for Gaussian responses in several aspects and for a variety of settings. Additive models replace the sum of linear functions of regressors by a sum of smooth functions. In dynamic or state space models, still linear in the regressors, coefficients are allowed to vary smoothly with time according to a Bayesian smoothness prior. We show that this is equivalent to imposing a roughness penalty on time-varying coefficients. Admitting the coefficients to vary with the values of other covariates, one obtains a class of varying-coefficient models (Hastie and Tibshirani, 1993), or in another interpretation, multiplicative models. The roughness penalty approach to non- and semiparametric modelling, together with Bayesian justifications, is used as a unifying and general framework for estimation. The methodological discussion is illustrated by some real data applications

State And Parameter Estimation With A Sequential Monte Carlo Method In A Three Dimensional Transport Model

Due to the inherent randomness and heterogeneity of the transport process, macrodispersion, non-fickian motion, and ergodicity, general assumptions of linearity and Gaussian distribution do not hold for the real field. Therefore, a state-space transport model for the non-linear and non-Gaussian system is proposed in this study. In this study, the state variable (concentration vector) and parameter (first-order decay) are updated with the available measurements. The probabilistic state-space formulation and updating of information on receipt of new measurements is formulated in the Bayesian framework. particle filter, a sequential Monte Carlo method, provides a rigorous general framework for dynamic state estimation problems in the Bayesian scheme. Here the reactive contaminant transport in subsurface is treated as a dynamic state and parameter estimation problem. A type of particle filter, commonly called Sequential Importance Resampling (SIR) is used for this subsurface transport problem. The model estimation is compared with a reference true random field. A promising improvement of the estimation accuracy is attained with the SIR particle filter while compared with a traditional deterministic approach. The standard deviations of the residuals were calculated for the comparison purpose. The particle filter data assimilation scheme reduces the prediction error by 48% in estimation accuracy. In case of having fixed parameters in the model, a standard technique to perform parameter estimation consists of extending the state with the parameter to transform the problem into optimal filtering problem. This approach requires the use of special particle filtering techniques which suffer from several drawbacks. An alternative statistical approach was adopted here to combine parameter estimation with the particle filter scheme. The concept of Euclidian norm was introduced in order to address the sequential weight assignment to the parameter estimation. The SIR particle filter scheme successfully estimated the parameter (first-order decay). With the use of the updated parameter in the state prediction, prediction error of the SIR particle filter data assimilation scheme became 78% smaller than the error from the deterministic model