Essays on state-space and regime-switching models in a high-dimensional setting

This dissertation consists of three chapters on high-dimensional Markov regime-switching and linear Gaussian state-space models. The first chapter (coauthored with Zhongjun Qu) presents a modeling framework and estimation methods for detecting regime switching in high-dimensional data. Given that the maximum likelihood estimator fails to converge due to the high dimensionality of the parameter space, we develop an effective Gibbs sampling algorithm for estimation and inference. As an empirical application, we consider a dataset consisting of four groups of US aggregate macroeconomic variables and their disaggregate counterparts: industrial production, capacity utilization, nonfarm payroll employees, and hours worked. Our model produces smaller mean-squared forecasting errors than the corresponding aggregate model in most cases. It also provides forecasts for sub-series, which the aggregate model can not offer. Moreover, we find that our model's inference on probabilities of recessions is very close to the NBER's business cycle dating. The second chapter examines several econometric issues related to data aggregation for linear Gaussian state-space models. It generalizes the aggregation results from autoregressive moving average (ARMA) models to linear Gaussian state-space models. Also, it provides new theoretical results for inference on state variables, highlighting the channels through which the disaggregate model achieves better estimates than the aggregate model. Monte Carlo simulations confirm the theoretical results. An empirical application to aggregate and disaggregate unemployment data reveals the extent of the information loss caused by aggregation. The third chapter generalizes the first chapter's framework to allow mixed sampling frequencies. We propose a data-driven model and estimate it using the Gibbs sampler. From simulations, we demonstrate that the mixed-frequency model outperforms the model with only high-frequency processes when estimating the state variable. As an empirical application, we consider a dataset with monthly employment and quarterly industrial production, quarterly capacity utilization, and quarterly hours worked. We find that our mixed-frequency Markov regime-switching model outperforms the classic single-frequency Markov regime-switching model to estimate the US monthly business cycles at the aggregate level

Generalized structured additive regression based on Bayesian P-splines

Generalized additive models (GAM) for modelling nonlinear effects of continuous covariates are now well established tools for the applied statistician. In this paper we develop Bayesian GAM's and extensions to generalized structured additive regression based on one or two dimensional P-splines as the main building block. The approach extends previous work by Lang und Brezger (2003) for Gaussian responses. Inference relies on Markov chain Monte Carlo (MCMC) simulation techniques, and is either based on iteratively weighted least squares (IWLS) proposals or on latent utility representations of (multi)categorical regression models. Our approach covers the most common univariate response distributions, e.g. the Binomial, Poisson or Gamma distribution, as well as multicategorical responses. For the first time, we present Bayesian semiparametric inference for the widely used multinomial logit models. As we will demonstrate through two applications on the forest health status of trees and a space-time analysis of health insurance data, the approach allows realistic modelling of complex problems. We consider the enormous flexibility and extendability of our approach as a main advantage of Bayesian inference based on MCMC techniques compared to more traditional approaches. Software for the methodology presented in the paper is provided within the public domain package BayesX

Bayesian semiparametric multi-state models

Multi-state models provide a unified framework for the description of the evolution of discrete phenomena in continuous time. One particular example is Markov processes which can be characterised by a set of time-constant transition intensities between the states. In this paper, we will extend such parametric approaches to semiparametric models with flexible transition intensities based on Bayesian versions of penalised splines. The transition intensities will be modelled as smooth functions of time and can further be related to parametric as well as nonparametric covariate effects. Covariates with time-varying effects and frailty terms can be included in addition. Inference will be conducted either fully Bayesian (using Markov chain Monte Carlo simulation techniques) or empirically Bayesian (based on a mixed model representation). A counting process representation of semiparametric multi-state models provides the likelihood formula and also forms the basis for model validation via martingale residual processes. As an application, we will consider human sleep data with a discrete set of sleep states such as REM and non-REM phases. In this case, simple parametric approaches are inappropriate since the dynamics underlying human sleep are strongly varying throughout the night and individual specific variation has to be accounted for using covariate information and frailty terms

Bayesian Semiparametric Multi-State Models

Multi-state models provide a unified framework for the description of the evolution of discrete phenomena in continuous time. One particular example are Markov processes which can be characterised by a set of time-constant transition intensities between the states. In this paper, we will extend such parametric approaches to semiparametric models with flexible transition intensities based on Bayesian versions of penalised splines. The transition intensities will be modelled as smooth functions of time and can further be related to parametric as well as nonparametric covariate effects. Covariates with time-varying effects and frailty terms can be included in addition. Inference will be conducted either fully Bayesian using Markov chain Monte Carlo simulation techniques or empirically Bayesian based on a mixed model representation. A counting process representation of semiparametric multi-state models provides the likelihood formula and also forms the basis for model validation via martingale residual processes. As an application, we will consider human sleep data with a discrete set of sleep states such as REM and Non-REM phases. In this case, simple parametric approaches are inappropriate since the dynamics underlying human sleep are strongly varying throughout the night and individual-specific variation has to be accounted for using covariate information and frailty terms

Bayesian Semiparametric Multi-State Models

Multi-state models provide a unified framework for the description of the evolution of discrete phenomena in continuous time. One particular example are Markov processes which can be characterised by a set of time-constant transition intensities between the states. In this paper, we will extend such parametric approaches to semiparametric models with flexible transition intensities based on Bayesian versions of penalised splines. The transition intensities will be modelled as smooth functions of time and can further be related to parametric as well as nonparametric covariate effects. Covariates with time-varying effects and frailty terms can be included in addition. Inference will be conducted either fully Bayesian using Markov chain Monte Carlo simulation techniques or empirically Bayesian based on a mixed model representation. A counting process representation of semiparametric multi-state models provides the likelihood formula and also forms the basis for model validation via martingale residual processes. As an application, we will consider human sleep data with a discrete set of sleep states such as REM and Non-REM phases. In this case, simple parametric approaches are inappropriate since the dynamics underlying human sleep are strongly varying throughout the night and individual-specific variation has to be accounted for using covariate information and frailty terms

Penalized additive regression for space-time data: a Bayesian perspective

We propose extensions of penalized spline generalized additive models for analysing space-time regression data and study them from a Bayesian perspective. Non-linear effects of continuous covariates and time trends are modelled through Bayesian versions of penalized splines, while correlated spatial effects follow a Markov random field prior. This allows to treat all functions and effects within a unified general framework by assigning appropriate priors with different forms and degrees of smoothness. Inference can be performed either with full (FB) or empirical Bayes (EB) posterior analysis. FB inference using MCMC techniques is a slight extension of own previous work. For EB inference, a computationally efficient solution is developed on the basis of a generalized linear mixed model representation. The second approach can be viewed as posterior mode estimation and is closely related to penalized likelihood estimation in a frequentist setting. Variance components, corresponding to smoothing parameters, are then estimated by using marginal likelihood. We carefully compare both inferential procedures in simulation studies and illustrate them through real data applications. The methodology is available in the open domain statistical package BayesX and as an S-plus/R function