Lee E. Ohanian

English

Research Papers in Economics

Notes on Bootstrapping in Linear Models with i.i.d. Disturbances
The bootstrap is a method of re-sampling. It is very easy to use, and can be a very
useful way of conducting small sample inference. (That is, when the sample is not
sufficiently large to rely on the asymptotic distribution theory).
We will discuss how to use the bootstap to construct empirical distribution functions for
estimated parameters.
First, consider the classical linear model:
yt  xt  t
Recall the standard assumptions here: the x’s are fixed, and are of dimension T x N,
and the disturbance term is an i.i.d. process.
Suppose we estimate this model using OLS. We get:
   xt
xt1xt
yt
The empirical counterpart to the disturbance term is:
 t  yt  xt 
Now, let’s use the bootstrap to construct the empirical distribution of the OLS
estimator,  .
We first use the parameteric bootstrap.
To do this, we make some parametric assumptions about the process that  is drawn
from. What type of assumptions should we make? First, we know that the process is
mean zero by assumption. For the second moment, we can use the sample variance
estimate. What about the type of distribution? Since the normality assumption is so
common, one can first test whether the empirical residuals are normal. There are a
number of tests available for testing for normality, and many econometric routines
1provide these tests. Typically, these tests evaluate whether there is excess skewness
and kurtosis in the series. Let’s assume for the time being that the empirical residuals
pass the normality test. This suggests we can use the normal distribution function.
(1) Generate a sequence of length ”T” random, normal, variables that are mean zero,
and have the same variance as that of the empirical residuals. Call this series e1.(In
matlab, this is done using the ”randn” function).
(2) Construct y1 as:
yt
1  xt   et
1
(3) Re-estimate  , and call this   1 :
  1  xt
xt1xt
yt
1
(4) Repeat steps 1-3 ”N-1” times, and index the estimated  by i
Now, we have ”N” estimates of  :   ii1
n . We can use the distribution of the  to
construct confidence intervals, form empirical densities, calculate variances and
covariances, etc.
Now, let’s use the non-parametric bootstrap. The only difference is how we generate
new series of residuals that we use to re-estimate the parameters.
(1) Take the empirical residuals  tt1
T . Now, let’s re-order these residuals randomly.
To do this, use a uniform random number generator (in matlab, this is the ”rand”
function). Generate a length ”T” series of uniform random numbers between 0 and 1.
Suppose T is 5, and the random numbers generated are .6, .4, .2, .5, .1. Since .6 is
the largest number, the first empirical residual would become the last one. Since .4 is
the third largest number, the second empirical residual would become the third one,
etc.
(2) Call this re-shuffled set of residuals  t
1t1
T . Now, Construct y1 as:
yt
1  xt   t
1
(3) Re-estimate  , and call this   1 :
  1  xt
xt1xt
yt
1
2(4) Repeat steps 1-3 ”N-1” times, and index the estimated  by i
Now, we have ”N” estimates of  :   ii1
n . We can use the distribution of the  to
construct confidence intervals, form empirical densities, calculate variances and
covariances, etc.
3

Notes on Bootstrapping in Linear Models with i.i.d. Disturbances

http://www.econ.ucla.edu/people/papers/Ohanian/Ohanian282.pdf

Notes on Bootstrapping in Linear Models with i.i.d. Disturbances

Abstract

Similar works

Full text

Available Versions

Research Papers in Economics