Nonparametric sequential prediction for stationary processes

We study the problem of finding an universal estimation scheme $h_n:\mathbb{R}^n\to \mathbb{R}$, $n=1,2,...$ which will satisfy \lim_{t\rightarrow\infty}{\frac{1}{t}}\sum_{i=1}^t|h_ i(X_0,X_1,...,X_{i-1})-E(X_i|X_0,X_1,...,X_{i-1})|^p=0 a.s. for all real valued stationary and ergodic processes that are in $L^p$. We will construct a single such scheme for all $1<p\le\infty$, and show that for $p=1$ mere integrability does not suffice but $L\log^+L$ does.Comment: Published in at http://dx.doi.org/10.1214/10-AOP576 the Annals of Probability (http://www.imstat.org/aop/) by the Institute of Mathematical Statistics (http://www.imstat.org

On universal estimates for binary renewal processes

A binary renewal process is a stochastic process $\{X_n\}$ taking values in $\{0,1\}$ where the lengths of the runs of 1's between successive zeros are independent. After observing ${X_0,X_1,...,X_n}$ one would like to predict the future behavior, and the problem of universal estimators is to do so without any prior knowledge of the distribution. We prove a variety of results of this type, including universal estimates for the expected time to renewal as well as estimates for the conditional distribution of the time to renewal. Some of our results require a moment condition on the time to renewal and we show by an explicit construction how some moment condition is necessary.Comment: Published in at http://dx.doi.org/10.1214/07-AAP512 the Annals of Applied Probability (http://www.imstat.org/aap/) by the Institute of Mathematical Statistics (http://www.imstat.org