The paper considers the K-statistic, Kleibergen’s (2000) adaptation of the Anderson-Rubin (AR) statistic in instrumental variables regression. Compared to the AR-statistic this K-statistic shows improved asymptotic efficiency in terms of degrees of freedom in overidenti?ed models and yet it shares, asymptotically, the pivotal property of the AR statistic. That is, asymptotically it has a chi-square distribution whether or not the model is identi?ed. This pivotal property is very relevant for size distortions in ?nite-sample tests. Whereas Kleibergen (2000) focuses especially on the asymptotic behavior of the statistic, the present paper concentrates on finite-sample properties in a Gaussian framework. In that case the AR statistic has an F-distribution. However, the K-statistic is not exactly pivotal. Its finite-sample distribution is affected by nuisance parameters. Here we consider the two extreme cases, which provide tight bounds for the exact distribution. The first case amounts to perfect identification —which is similar to the asymptotic case—where the statistic has an F-distribution. In the other extreme case there is total underidentification. For the latter case we show how to compute the exact distribution. Thus we provide tight bounds for exact con?dence sets based on the efficient K-statistic. Asymptotically the two bounds converge, except when there is a large number of redundant instruments.

Bekker, Paul A.

Kleibergen, Frank

English

Research Papers in Economics

Finite-Sample Instrumental Variables
Inference using an Asymptotically
Pivotal Statistic
Paul A. Bekker∗ and Frank Kleibergen†
University of Groningen University of Amsterdam
June, 2001
Abstract
The paper considers the K-statistic, Kleibergen’s (2000) adaptation of
the Anderson-Rubin (AR) statistic in instrumental variables regres-
sion. Compared to the AR-statistic this K-statistic shows improved
asymptotic eﬃciency in terms of degrees of freedom in overidentiﬁed
models and yet it shares, asymptotically, the pivotal property of the
AR statistic. That is, asymptotically it has a chi-square distribution
whether or not the model is identiﬁed. This pivotal property is very
relevant for size distortions in ﬁnite-sample tests. Whereas Kleibergen
(2000) focuses especially on the asymptotic behavior of the statistic,
the present paper concentrates on ﬁnite-sample properties in a Gaus-
sian framework. In that case the AR statistic has an F-distribution.
However, the K-statistic is not exactly pivotal. Its ﬁnite-sample dis-
tribution is aﬀected by nuisance parameters. Here we consider the two
extreme cases, which provide tight bounds for the exact distribution.
The ﬁrst case amounts to perfect identiﬁcation—which is similar to
the asymptotic case—where the statistic has an F-distribution. In
the other extreme case there is total underidentiﬁcation. For the lat-
ter case we show how to compute the exact distribution. Thus we
provide tight bounds for exact conﬁdence sets based on the eﬃcient
K-statistic. Asymptotically the two bounds converge, except when
there is a large number of redundant instruments.
∗Department of Economics, University of Groningen, P.O. Box 800, 9700 AV Gronin-
gen, The Netherlands, E-mail: P.A.Bekker@eco.rug.nl
†Department of Quantitative Economics, University of Amsterdam, Roetersstraat 11,
1018 WB Amsterdam, The Netherlands, E-mail: kleiberg@fee.uva.nl.1I n t r o d u c t i o n
Instrumental variables inference can be aﬀected rather strongly by the quality
of the instruments. Especially in the presence of weak instruments usual
inferential procedures such as likelihood based Wald, Likelihood Ratio and
Lagrange Multiplier tests may show considerable size distortions and the
performance of conﬁdence sets may be abysmally poor, see e.g. Nelson and
Startz (1990), Staiger and Stock (1997), Zivot et. al. (1998), and Hausman
and Hahn (1999). However, an exact test in a Gaussian context has been
provided by Anderson and Rubin (1949). The resulting Anderson-Rubin
(AR) statistic is pivotal and has an F-distribution, which does not depend on
nuisance parameters and is not aﬀected by the degree of underidentiﬁcation.
However, the AR statistic has a limiting chi square distribution with a
number of degrees of freedom that equals the number of instruments. This
number exceeds, or equals, the number of structural parameters, which af-
fects the power of the test statistic. As has been shown by Kleibergen (2000)
it is possible to construct a statistic, the K-statistic, with similar asymp-
totic pivotal properties but with a limiting chi-square distribution that has a
number of degrees of freedom equal to the number of structural parameters.
Thus, the K-statistic has an asymptotic distribution with a minimal number
of degrees of freedom without being hampered by a poor performance close to
1points of underidentiﬁcation, as is the case for the more traditional inference
procedures. This conclusion can be drawn with respect to the asymptotic be-
havior of the test statistic. Kleibergen (2000) does not provide ﬁnite-sample
evidence. Here we do consider ﬁnite sample properties.
Under Gaussianity the AR statistic has an F-distribution and thus pro-
vides an exact ﬁnite-sample test. The K-statistic has a ﬁnite-sample distribu-
tion that depends on nuisance parameters. It does thus not provide an exact
test. Based on the asymptotic pivotal behavior, we, however, expect the
asymptotic distribution to provide an accurate approximation of the exact
ﬁnite-sample distribution but we do not know the degree of accuracy. There-
fore, we compute bounds for the exact distribution of the K-test statistic by
assuming Gaussian distributions for the disturbances.
We ﬁnd on one extreme—provided by the case of perfect identiﬁcation—
an F-distribution, similar to the distribution of the AR statistic, albeit with
fewer degrees of freedom. On the other extreme—provided by the case of
total underidentiﬁcation—we ﬁnd a distribution that is complicated analyt-
ically, but can be simulated easily. That is, both extreme distributions can
be computed in practice. We show that these extreme cases provide tight
bounds. In many practical cases the two bounds are very close indeed, as
we might expect based on the asymptotic pivotal property of the statistic.
However, in case of many redundant instruments, the bounds may diﬀer even
2asymptotically. In fact, we also compute the limiting distribution in this case
by means of the many-instruments asymptotics as described in Bekker (1994).
We ﬁnd it to be diﬀerent from the large-sample chi-square distribution.
The paper is organized as follows. In Section 2 we describe the model
and discuss brieﬂy the AR statistic. The K-statistic is discussed in Section
3. The computation of its distribution, based on simulations, is described in
Section 4. The proof of the tightness of the bounds is given in an Appendix.
Section 5 provides some Monte Carlo computations.
2 The Model and the Anderson-Rubin Statis-
tic
Consider a classical instrumental variables regression model in a cross-section
context. That is, let
y = Xβ+ ε, (1)
X = ZΠ+V, (2)
where Z is an n×k matrix of full column rank that consists of the nonstochas-
tic instrumental variables and X is n × m, which may contain endogenous
as well as exogenous explanatory variables. The latter are assumed to be
3columns of Z as well and m ≤ k. If interest is restricted to those elements of
the parameter vector β that relate to the endogenous explanatory variables,
the instruments in Z might be separated into two parts. For ease of exposi-
tion we will not do this and consider the m-vector β as a whole. We assume
the rows of (ε,V ) to be independently normally distributed with zero mean;
we denote the covariance matrix by Σ.
Let, in general for a matrix H of full column rank the projection matrix
be denoted by PH = H(H H)−1H . Consider the quantity
AR(β)=
(y − Xβ) PZ(y − Xβ)
(y − Xβ) (In − PZ)(y − Xβ)
. (3)
Minimizing AR(β)o v e rβ provides the LIML estimator. However, we are
not concerned with estimation but with testing. If the argument in (3)
equals the true β,t h e ny − Xβ = ε and the numerator and denominator of
(3) have independent chi-square distributions with k and n − k degrees of
freedom, respectively. So, when multiplied by (n−k)/k, AR(β) has an Fk,n−k-
distribution. An exact test of H0: β = β
  is found by verifying whether or not
AR(β
 ) has a small enough p-value in the Fk,n−k-distribution. The resulting
test is exact and known from Anderson and Rubin (1949). Notice that the
AR-statistic has an F-distribution whether or not the model is identiﬁed.
That is, even if Π has a deﬁcient column rank AR(β) will be distributed as
4an Fk,n−k random variable. This shows that it is a pivotal statistic.
Although the AR-test is exact under Gaussianity, it may have poor power
if the number of instruments k exceeds the number of explanatory variables
m. That is, in the numerator of (3), y − Xβ is projected on the full space
spanned by Z. We know, however, that, if β deviates from the true value, the
mean of y − Xβ is located in the subspace spanned by ZΠ. The problem is
then that we do not know Π. Still, the dimension of the space onto which we
project y − Xβ can be reduced in a way that preserves the pivotal property
of the statistic asymptotically.
3 Removing redundant degrees of freedom
from the AR Statistic
Consider a regression of the columns of V on ε.T h a ti s ,l e t
V = ελ
  + W, (4)
λ =
σV,ε
σ2
ε
, (5)
where W is independent of ε;a n dσV,ε and σ2
ε denote the covariance of V
and ε and the variance of the latter, respectively. Consequently, X − ελ
  =
ZΠ+W, is like Z independent of ε but has a smaller column dimension. If
5λ were known, we could replace the projection matrix PZ in the numerator
of (3) by P ˜ X,w i t h
˜ X(λ)=PZ(X − (y − Xβ)λ
 ). (6)
For the true value of β, ˜ X(λ) is stochastic independent of ε. Thus, this adap-
tation of the AR statistic would still be pivotal, but now distributed, when
multiplied by (n − k)/m,a sFm,n−k.O f c o u r s e , λ is unknown. However,
Kleibergen (2000) uses simply a consistent estimator of λ that is stochas-
tic independent of Z y and Z X under the null where β is speciﬁed. This
estimator, ˆ λ, is speciﬁed as in (4) with σV,ε and σ2
ε replaced by
ˆ σV,ε =
X (In − PZ)(y − Xβ)
n − k
,
ˆ σ
2
ε =
(y − Xβ) (In − PZ)(y − Xβ)
n − k
. (7)
The resulting K-statistic is given by
K(β)=
(y − Xβ) P ˜ X(ˆ λ)(y − Xβ)
(y − Xβ) (In − PZ)(y − Xβ)/(n − k)
. (8)
Asymptotically, it is distributed as a chi square with m degrees of freedom.
Further discussions are given by Kleibergen (2000).
64 Bounding the Finite-Sample Distribution
Although the asymptotic distribution of (8) is simple, the ﬁnite-sample dis-
tribution is more complicated. In fact, it depends on nuisance parameters. In
this section we will analyse this dependency and show that the distribution
can be bounded by distribution functions that do not depend on unknown
parameters.
As the denominator of (8) is given by ˆ σ
2
ε, we ﬁnd that the statistic (8)
results from projecting the ﬁrst column of the following matrix onto the space
that is spanned by its last m columns:
PZ (y,X)

 

10
−βI m

 


 

  σ
−1
ε −
  σ
V,ε
  σ2
ε
0 Im

 
 =( 0 ,ZΠ) + PZ (ε,V )

 

  σ
−1
ε −
  σ
V,ε
  σ2
ε
0 Im

 
.
(9)
In order to be able to simulate the distribution of K(β), we deﬁne a matrix
(  ε,  W) as follows
(  ε,   W)=
 
ε
σε
,WΣ
−1/2
W
 
,
where W has been deﬁned in (4) and ΣW denotes the covariance matrix of
its rows. We note that the matrix (  ε,  W) has independent standard normally
7distributed elements. We deﬁne their estimated variance and covariance by
  σ  W,  ε =
  W  (In − PZ)  ε
n − k
,
  σ
2
  ε =
  ε
 (In − PZ)  ε
n − k
.
The second term in (9) can now be expressed as follows
PZ (ε,V )




  σ
−1
ε −
  σ
V,ε
  σ2
ε
0 Im



 = PZ
 
  ε,  W
 




σε
σ
V,ε
σε
0Σ
1/2
W








  σ
−1
ε −
  σ
V,ε
  σ2
ε
0 Im




= PZ
 
  ε,  W
 

 

 
  σε
σε
 −1 σ
V,ε
σε −
σε  σ
V,ε
  σ2
ε
0Σ
1/2
W

 

= PZ
 
  ε,  W
 




  σ
−1
  ε −
  σ
  W,  ε
  σ2
  ε
0 Im








10
0Σ
1/2
W



,
(10)
where the last equality follows from
  σ  W,  ε =
Σ
−1/2
W
σε
  σW,ε
=
Σ
−1/2
W
σε
 
ˆ σV,ε −
ˆ σ
2
εσV,ε
σ2
ε
 
,
ˆ σ
2
  ε =
ˆ σ
2
ε
σ2
ε
.
8Consequently, the K-statistic K(β)i sg i v e nb y
K(β)=
  ε
 P   X  ε
ˆ σ
2
  ε
(11)
  X = PZ
 
  W −  ε
  σ
 
  W,  ε
  σ
2
  ε
 
+ ZΠΣ
−1/2
W . (12)
If the matrix ΠΣ−1/2 were known, the distribution of K(β)c o u l db es i m u -
lated, since (  ε,  W) can be simulated, as they have a standard normal distri-
bution. However, the matrix ΠΣ
−1/2
W is not known. Still we can distinguish
between two extreme cases.
One extreme case is given by total underidentiﬁcation: Π = 0. In this
case the distribution can be simulated easily. The distribution depends only
on the dimension parameters k, m and n.
Another extreme case is when Π = θΠ
 ,w h e r eΠ
  is ﬁxed and the scalar
θ grows to inﬁnity, θ →∞ . In the latter case P   X converges in probability
to PZΠ and so the distribution of K(β)/m converges to Fm,n−k.T h e s et w o
extreme cases in which we can construct the exact distribution of K(β)/m
can be shown to give upper and lower bounds on the exact ﬁnite sample dis-
tribution of K(β)/m. This results since as |θ| increases, then K(β) decreases
stochastically, i.e. the distribution function moves to the left. Intuitively,
this can be understood by noticing that   ε is correlated with the variables in
  X. A more formal justiﬁcation is given in the Appendix.
9The above implies that we can compute bounds for the ﬁnite-sample dis-
tribution of the K-statistic using simulation. Conservative tests or conﬁdence
sets can be based on the critical value found for the distribution simulated
at Π = 0. However, if n is large then   σ  W,  ε is small, in probability, and the
correlation between   ε and   X becomes negligible. In that case the bounds are
very close. Indeed, if n →∞ ,t h e nK(β)
A ∼ χ2
m, see Kleibergen (2000).
Would this limit distibution also provide an accurate approximation if
there are many redundant instruments? In order to answer this question we
consider the limit distribution of K(β) for an asymptotic parameter sequence
where the number of instruments increases with the number of observations:
k/n → α>0.
For the lower bound we found a ﬁnite-sample distribution given by Fm,n−k.
Clearly, the lower bound converges to a χ2
m-distribution whether or not α>0.
For the upper bound, where Π = 0, we can use the following limits, which
can be easily veriﬁed.
  σ
2
  ε
p
→ 1,
  σ  W,  ε
p
→ 0,
  W
 PZ  W/k
p
→ Im,
10  ε
 PZ  ε/k
p
→ 1,
  W
 PZ  ε/k
p
→ 0,
so
  X
    X/k
p
→ Im. (13)
Furthermore,
  W  PZ  ε/(k
1/2)
A ∼N(0,I m),
(n − k)
1/2  σ  W,  ε
A ∼N(0,I m),
so
  X
   ε/(k
1/2)
A ∼N(0,(1 − α)
−1Im). (14)
Consequently, we ﬁnd for the upper bound, where Π = 0,
(1 − k/n)K(β)
A ∼ χ
2
m, (15)
which holds whether or not α>0. In particular if α>0 we ﬁnd a simple
diﬀerence between the asymptotic upper and lower bounds. The diﬀerence is
of order k/n. These practical results are conﬁrmed numerically in the next
section.
115 Monte Carlo Results
To determine the applicability of the bounds on the exact distribution of
K(β), we construct the bounds for m = 1 and a few values of k and n.
Figures 1 to 7 show the lower and upper bounds on the exact distribution
function of
K(β)
m (8). The ﬁgures show a considerable diﬀerence between the
lower and upper bound, when k/n is large. When k/n gets smaller, the diﬀer-
ence decreases and becomes negligible, as expected. The ﬁgures also contain
a limiting approximation of the upper bound. Instead of the asymptotic
approximation
χ2
m
m , based on (15), we used the Fm,n−k distribution, which is
more accurate in small samples and more convenient as well, since it is also
used for the lower bound. The ﬁgures show this limiting approximation is
accurate even for small values of n. Hence, it suﬃces to use critical values
from the Fm,n−k distribution both as lower bound and, by multiplying them
by 1/(1 − k/n), as upper bound of the exact critical values of
K(β)
m .
120 2 4 6 8 10 12 14 16 18
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Figure 1: Lower bound (—) and upper bound (- -) and limiting approxima-
tion of upper bound (...) of the distribution function of K(β)/m for n = 10,
k =5a n dm =1 .
0 1 2 3 4 5 6 7 8 9 10
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Figure 2: Lower bound (—) and upper bound (- -) and limiting approxima-
tion of upper bound (...) of the distribution function of K(β)/m for n = 25,
k =5a n dm =1 .
130 2 4 6 8 10 12
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Figure 3: Lower bound (—) and upper bound (- -) and limiting approxima-
tion of upper bound (...) of the distribution function of K(β)/m for n = 25,
k =1 0a n dm =1 .
0 1 2 3 4 5 6 7 8 9 10
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Figure 4: Lower bound (—) and upper bound (- -) and limiting approxima-
tion of upper bound (...) of the distribution function of K(β)/m for n = 50,
k =1 0a n dm =1 .
140 1 2 3 4 5 6 7 8 9 10
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Figure 5: Lower bound (—) and upper bound (- -) and limiting approxima-
tion of upper bound (...) of the distribution function of K(β)/m for n = 100,
k =1 0a n dm =1 .
0 2 4 6 8 10 12
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Figure 6: Lower bound (—) and upper bound (- -) and limiting approxima-
tion of upper bound (...) of the distribution function of K(β)/m for n = 100,
k =2 5a n dm =1 .
150 2 4 6 8 10 12
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Figure 7: Lower bound (—) and upper bound (- -) and limiting approxima-
tion of upper bound (...) of the distribution function of K(β)/m for n = 100,
k =5 0a n dm =1 .
16Appendix
In order to prove the assertion in Section 4 that K(β) is stochastically de-
creasing in |θ|,w h e r eΠ=θΠ
 , we ﬁrst condition on (In − PZ)(  ε,  W). This
does not aﬀect the distribution of PZ(  ε,  W). Let
H =( Z
 Z)
1/2Π
 
Σ
−1/2
W ,
u =( Z
 Z)
−1/2Z
   ε,
U =( Z
 Z)
−1/2Z
 (  W −  ε  λ
 
),
  λ =
  σ  W,  ε
  σ
2
  ε
.
We may regress u on U so that u = Uγ + ν,w h e r eν is independent of U.
Next we also condition on U and consider the numerator of K(β)g i v e nb y
  εP   X  ε =( Uγ+ ν)
 PL(Uγ+ ν),
L = Hθ+ U.
Let Q be an orthogonal m × m dimensional matrix such that Uγ is orthog-
onal to the last m − 1 columns of L(L L)−1/2Q, such that it has a posi-
tive inner product with the ﬁrst column: (γ U PLUγ)1/2. Furthermore let
ξ = Q (L L)−1/2L ν, whose elements are independently normally distributed,
17then
  εP   X  ε =( Uγ+ ν)
 L(L
 L)
− 1
2QQ
 (L
 L)
− 1
2L
 (Uγ+ ν)
=
 
Q
 (L
 L)
− 1
2L
 (Uγ+ ν)
    
Q
 (L
 L)
− 1
2L
 (Uγ+ ν)
 
=( ξ1 +( γ
 U
 PLUγ)
1/2)
2 +
m  
i=2
ξ
2
i.
Consequently, we only have to show that, conditional onU,( ξ1+(γ U PLUγ)1/2)2
stochastically decreases as |θ| increases. Here we ﬁnd an analogy with the
non-central χ2 distribution that decreases as the non-centrality parameter
decreases. So we only have to prove that the non-random function
γ
 U
 (Hθ+ U){(Hθ+ U)
 (Hθ+ U)}
−1 (Hθ+ U)
 Uγ
decreases as |θ| increases. This can be shown to hold true by considering the
derivative with respect to θ, which completes the proof.
18References
[1] Anderson, T.W. and H. Rubin. Estimators of the Parameters of a Single
Equation in a Complete Set of Stochastic Equations. The Annals of
Mathematical Statistics, 21:570–582, (1949).
[2] Bekker, P.A. Alternative Approximations to the Distributions of Instru-
mental Variable Estimators. Econometrica, 62:657–681, 1994.
[3] Hausman, J. and J. Hahn. A New Speciﬁcation Test for the Validity of
Instrumental Variables. Econometrica, forthcoming.
[4] Kleibergen, F. Pivotal Statistics for testing Structural Parameters in
Instrumental Variables Regression. Tinbergen Institute Discussion Paper
TI2000-055/4, 2000.
[5] Nelson, C.R. and R. Startz. Some Further Results on the Exact Small
Sample Properties of the Instrumental Variables Estimator. Economet-
rica, 58:967–976, 1990.
[6] Staiger, D. and J.H. Stock. Instrumental variables regression with weak
instruments. Econometrica, 65:557–586, 1997.
19[7] Zivot, E., R. Startz, and C. R. Nelson. Valid Conﬁdence Intervals and
Inference in the Presence of Weak Instruments. International Economic
Review, 39:1119–1144, 1998.
20

Finite-sample instrumental variables inference using an asymptotically pivotal statistic

Abstract

Similar works

Full text

Available Versions

Research Papers in Economics