Recursive estimation of a drifted autoregressive parameter

Suppose the X0,...., Xn are observations of a one-dimensional stochastic dynamic process described by autoregression equations when the autoregressive parameter is drifted with time, i.e. it is some function of time: ¿0,...., ¿n, with ¿k = ¿ (k/n). The function ¿(t) is assumed to belong a priori to a predetermined nonparametric class of functions satisfying the Lipschitz smoothness condition. At each time point t those observations are accessible which have been obtained during the preceding time interval. A recursive algorithm is proposed to estimate ¿??t??. Under some conditions on the model, we derive the rate of convergence of the proposed estimator when the frequencyof observations n tends to infinity

Adaptive Bayesian inference on the mean of an infinite-dimensional normal distribution

We consider the problem of estimating the mean of an infinite-break dimensional normal distribution from the Bayesian perspective. Under the assumption that the unknown true mean satisfies a smoothness condition, we first derive the convergence rate of the posterior distribution for a prior that is the infinite product of certain normal distributions and compare with the minimax rate of convergence for point estimators. Although the posterior distribution can achieve the optimal rate of convergence, the required prior depends on a smoothness parameter q. When this parameter q is unknown, besides the estimation of the mean, we encounter the problem of selecting a model. In a Bayesian approach, this uncertainty in the model selection can be handled simply by further putting a prior on the index of the model. We show that if q takes values only in a discrete set, the resulting hierarchical prior leads to the same convergence rate of the posterior as if we had a single model. A slightly weaker result is presented when q is unrestricted. An adaptive point estimator based on the posterior distribution is also constructed. Primary Subjects: 62G20. Secondary Subjects: 62C10, 62G05. Keywords: Adaptive Bayes procedure; convergence rate; minimax risk; posterior distribution; model selection

Online tracking of a drifting parameter of a time series

We propose an online algorithm for tracking a multivariate time-varying parameter of a time series. The algorithm is driven by a gain function. Under assumptions on the gain function, we derive uniform error bounds on the tracking algorithm in terms of chosen step size for the algorithm and on the variation of the parameter of interest. We give examples of a number of different variational setups for the parameter where our result can be applied, and we also outline how appropriate gain functions can be constructed. We treat in some detail the tracking of time varying parameters of an AR($d$) model as a particular application of our method

Online tracking of a drifting parameter of a time series

We propose an online algorithm for tracking a multivariate time-varying parameter of a time series. The algorithm is driven by a gain function. Under assumptions on the gain function, we derive uniform error bounds on the tracking algorithm in terms of chosen step size for the algorithm and on the variation of the parameter of interest. We give examples of a number of different variational setups for the parameter where our result can be applied, and we also outline how appropriate gain functions can be constructed. We treat in some detail the tracking of time varying parameters of an AR($d$) model as a particular application of our method

Optimal two-stage procedures for estimating location and size of maximum of multivariate regression function

We propose a two-stage procedure for estimating the location \bolds{\mu} and size M of the maximum of a smooth d-variate regression function f. In the first stage, a preliminary estimator of \bolds{\mu} obtained from a standard nonparametric smoothing method is used. At the second stage, we "zoom-in" near the vicinity of the preliminary estimator and make further observations at some design points in that vicinity. We fit an appropriate polynomial regression model to estimate the location and size of the maximum. We establish that, under suitable smoothness conditions and appropriate choice of the zooming, the second stage estimators have better convergence rates than the corresponding first stage estimators of \bolds{\mu} and M. More specifically, for $\alpha$-smooth regression functions, the optimal nonparametric rates $n^{-(\alpha-1)/(2\alpha+d)}$ and $n^{-\alpha/(2\alpha+d)}$ at the first stage can be improved to $n^{-(\alpha-1)/(2\alpha)}$ and $n^{-1/2}$, respectively, for $\alpha>1+\sqrt{1+d/2}$. These rates are optimal in the class of all possible sequential estimators. Interestingly, the two-stage procedure resolves "the curse of the dimensionality" problem to some extent, as the dimension d does not control the second stage convergence rates, provided that the function class is sufficiently smooth. We consider a multi-stage generalization of our procedure that attains the optimal rate for any smoothness level $\alpha>2$ starting with a preliminary estimator with any power-law rate at the first stage.Comment: Published in at http://dx.doi.org/10.1214/12-AOS1053 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

Rate-optimal Bayesian intensity smoothing for inhomogeneous Poisson processes

We apply nonparametric Bayesian methods to study the problem of estimating the intensity function of an inhomogeneous Poisson process. We exhibit a prior on intensities which both leads to a computationally feasible method and enjoys desirable theoretical optimality properties. The prior we use is based on B-spline expansions with free knots, adapted from well-established methods used in regression, for instance. We illustrate its practical use in the Poisson process setting by analyzing count data coming from a call centre. Theoretically we derive a new general theorem on contraction rates for posteriors in the setting of intensity function estimation. Practical choices that have to be made in the construction of our concrete prior, such as choosing the priors on the number and the locations of the spline knots, are based on these theoretical findings. The results assert that when properly constructed, our approach yields a rate-optimal procedure that automatically adapts to the regularity of the unknown intensity function

Empirical Bayesian test of the smoothness.

In the context of adaptive nonparametric curve estimation problem, a common assumption is that a function (signal) to estimate belongs to a nested family of functional classes, parameterized by a quantity which often has a meaning of smoothness amount. It has already been realized by many that the problem of estimating the smoothness is not sensible. What then can be inferred about the smoothness? The paper attempts to answer this question. We consider the implications of our results to hypothesis testing. We also relate them to the problem of adaptive estimation. The test statistic is based on the marginalized maximum likelihood estimator of the smoothness for an appropriate prior distribution on the unknown signal