In this paper, we warn on the overoptimistic conclusions led by weak instruments testing when good leverage points are present in the first stage of an IV estimation. Some simulations and an empirical application are provided to illustrate the point raised.Instrumental variables, Weak instruments, Outliers, Robust statistics

Darwin Ugarte Ontiveros

Vincenzo Verardi

English

Research Papers in Economics

 
SUPPOSEDLY STRONG INSTRUMENTS AND 
GOOD LEVERAGE POINTS 
DARWIN UGARTE ONTIVEROS & VINCENZO VERARDI 
WP 1203 
DEPARTMENT OF ECONOMICS  
WORKING PAPERS SERIES Supposedly strong instruments and good leverage
points
Darwin Ugarte Ontiveros∗† Vincenzo Verardi‡
January 31, 2012
Abstract
In this paper, we warn on the overoptimistic conclusions led by weak instruments
testing when good leverage points are present in the ﬁrst stage of an IV estimation.
Some simulations and an empirical application are provided to illustrate the point
raised.
Keywords: Instrumental variables, Weak instruments, Outliers, Robust Statistics, In-
formal Sectors. JEL classiﬁcation: C3, C12, O1
1 Introduction
Recent econometric literature on instrumental variable (IV) estimations has paid a con-
siderable attention to the relevance of the instruments used. Indeed, it is well-known that
when the instruments are weak, the hypothesis tests can exhibit large size distortions and
coefﬁcients and the standard errors can be severely biased (see Andrews and Stock, 2005).
A standard approach to assess the relevance of the instruments is to compare the ﬁrst-
stage F-statistic associated to the Wald test checking for the signiﬁcance of the excluded
∗Corresponding author. Tel.:+32 81724837; fax +32 81724840.
†Economics School of Louvain, University of Namur (CRED). Rempart de la Vierge, 8. B-5000 Namur.
E-mail: dugarteo@fundp.ac.be.
‡University of Namur (CRED) and Université libre de Bruxelles (ECARES and CKE). Rempart de la
Vierge, 8. B-5000 Namur. E-mail: vverardi@fundp.ac.be.
1instruments to some tabulated values (see Stock and Yogo, 2005). When several explana-
tory variables are endogenous and need to be instrumented, a Cragg and Donald (1993)
or a Kleibergen and Paap (2006) statistic is called-on but the logic remains the same. We
consider here only the case of a single troublesome variable. The generalization to more
is straightforward.
What we investigate is the behavior of weak instruments testing when some observa-
tions, while following the same "endogenous regressor-instruments" relation as the others,
lie isolated on the regression hyperplane (i.e. good leverage). We argue that ﬁrst stage
good leverage points lead to overoptimistic conclusions on the strength of the instruments
and consequently on the quality of the instrumenting strategy. We present Monte Carlo
simulations to support this conclusion. The paper of McKenzie and Sakho (2010) study-
ing the effect of formality on micro-ﬁrm proﬁts in Bolivia is revisited to illustrate how a
limited number of apparently well behaving observations may substantially inﬂuence the
conclusions on weak instruments.
The structure of this note is the following: in section 2 the effect of good leverage
points on weak instruments inference is discussed and in section 3, some Monte Carlo
simulations are run to assess the size of the problem. Section 4 highlights the relevance
of the result using an economic example. Section 5 concludes.
2 Good leverage points and weak instruments
The problem induced by good leverage points on weak instrument testing can easily be
explained calling on the well-known Wald test. The intuition is the following: assume a
linear model of the type
y = Xβ + u (1)
where y is the n × 1 dependent variable vector, X is a n × p matrix of explanatory
variables and u is the n × 1 vector of disturbances. One column of X is assumed corre-
lated with the error term. The least squares estimated parameters are therefore biased and
2inconsistent. To tackle this problem, instrumental variable estimations are generally used
relying on a set of k excluded instruments (Z). The relevance of Z is usually tested by
comparing the ﬁrst stage F-statistic (of the Wald test checking if all parameters associated
to excluded instruments are equal to zero) to tabulated critical values available from Stock
and Yogo (2005). The F−statistic is computed as
F =
(Rˆ θ − q)￿ (R(Z￿Z)−1R￿)
−1 (Rˆ θ − q)
ks2 (2)
where R is the matrix of parameters restrictions, q is a k-dimensional vector of zeros,
ˆ θ is the vector of estimated parameters and s2 is the estimated variance of the ﬁrst stage
residuals.
The effect of good leverage points on F is unambiguous. Good leverage points have a
rather limited effect on the denominator of equation (2) since, by lying close to the regres-
sion hyperplane, they do not affect substantially the estimated variance of the residuals.
On the other hand, their inﬂuence on the numerator is large (via Z￿Z). As a consequence,
a very limited number of individuals outlying in the space of the instruments Z but follow-
ing the model relating X to Z may lead to overoptimistic conclusions about the relevance
of instruments. What we suggest to do to cope with this is to identify all type of outliers
in the ﬁrst stage and downweight their importance (in this note we use the most drastic
weighting scheme that consists in awarding a weight zero to any outlying observation
but others can be envisaged). The outliers are identiﬁed by running a multivariate out-
lier detection tool on the matrix (X,Z). The multivariate estimator used here is Stahel
(1981) and Donoho (1982), hereafter called SD: it consists in calculating the outlyingness
of each point by projecting the data cloud unidimensionally in all the possible directions
and estimating the distance from each observation to the centre of each projection. Then,
the degree of outlyingness is deﬁned as the maximal distance that is obtained when con-
sidering all possible projections (see Maronna and Yohai, 2006). Since this outlyingness
distance is distributed as
￿
χ2
p+k, we can choose a quantile above which we consider an
observation as being outlying (we consider here the 95th percentile). Once the importance
awarded to outliers is downweighted, the problem raised above disappears.
3In the next section we illustrate the effect of a limited number of ﬁrst stage good
leverage points on the F-statistic and investigate the effectiveness of the procedure we
propose by running some Monte Carlo simulations.
3 Monte Carlo results
In the simulations, we use a data generating process y = X1 + X2 + u and X1 = θZ +
X2+v where X2 and Z are generated from two independent N(0,1) and disturbances are
generate from a bivariate normal,

 u
v

 ∼ N

(0,0),

 1 ρ
ρ 1




The degree of "endogeneity" ρ is set to 0.75 and θ to
√
2. In this way the ﬁrst stage
F-statistic is equal to 3. The sample size n considered is 1000. The simulation setup is
the following: we contaminate the dataset by replacing 1% of the X (and Z is modiﬁed
accordingly to keep the relation unchanged) by adding an integer that ranges from 1 to
10. For each contamination we run 2000 simulations and calculate the bias of F and the
percentage of rejection. The resulting curbs are presented in Figure 1.
[INSERT FIGURE I HERE]
The plain line in the left panel shows that the bias of the F-statistic increases quickly
when good leverage points are present, leading to overoptimistic conclusions. The plain
line in the right panel of Figure 1 shows that the percentage of (incorrect) rejections of the
null of weak instrument increases quickly to 100%. These results highlight the extreme
sensitivity of the ﬁrst stage F-statistic to good leverage points with relatively low con-
tamination levels. The dashed line in Figure 1 also shows that the weighed IV estimator
we propose (SDIV from now on) behaves well since the F-statistic and the percentage of
rejection are not sensible to good leverage points. We simulated alternative scenarios in-
creasing the number of instruments and/or the percentage of contamination and the results
are similar.
44 Example: The effect of formality on micro-ﬁrm proﬁts
In McKenzie and Sakho (2010) the effect of formality on micro-ﬁrms proﬁts in Bolivia is
investigated. The authors use the logarithm of the distance to a tax ofﬁce as an instrument
for ﬁrms’ formality level, their main explanatory variable. Furthermore they use a set
of control variables to remove confounding effects. For the sake of brevity we do not
discuss them here and just refer to them as included instruments. Their main ﬁnding is that
increasing formality in Bolivia raises ﬁrm’s proﬁts. Their instrumenting strategy seems to
perform well since their ﬁrst stage F-statistic is approximately 12.68 which leads to the
conclusion that the bias of least squares has been substantially reduced (about 88%) when
instrumenting.
To identify inﬂuential outliers in the ﬁrst stage estimation, we run the projection-based
SD estimator on all the variables entering the ﬁrst stage equation (i.e. the explanatory
troublesome variable, the included and the excluded instruments). If we run a simple
(trimmed) linear regression excluding the identiﬁed outliers (23 observations), the esti-
mated coefﬁcients do not vary substantially, which seems to indicate that outliers were
not an issue in their dataset. Therefore, all evidence point towards a powerful instrument-
ing strategy.
However, when we have a closer look to the ﬁrst stage equation, some concerns
emerge. To start with, we plot the standardized residuals of the trimmed regression against
the outlyingness distance obtained by running the SD-estimator only on the (included and
excluded) instruments. Since residuals are assumed normally distributed in a linear re-
gression (for Gaussian data), we consider a point as potentially outlying in the vertical
dimension if the (absolute) Standardized Residuals (SR from now on) are larger than the
97.5th percentile of the normal distribution (i.e. 1.96). As far as the instruments outly-
ingness distance is concerned, it is well-known to be distributed as a
￿
χ2 with degrees
of freedom equal to the total number of (included and excluded) continuous instruments
plus the number of endogenous regressors. We decided to consider a point as potentially
outlying in the space of the instruments if its Outlyingness Distance (OD) is larger than
5the 95th percentile of a
￿
χ2
10 distribution (i.e. 4.28). These limits are plotted respec-
tively as horizontal and vertical lines. The resulting graphical tool, described thoroughly
in Rousseeuw and Van Zomeren (1990), allows to recognize the type of outliers present.
Indeed, standard observations should be associated with absolute SR < 1.96 and instru-
ments OD < 4.28. On the other hand, vertical outliers should have instruments OD com-
parable to the bulk of data but should be associated to absolute SR > 1.96. Bad leverage
points should be associated to instruments OD > 4.28 and absolute SR > 1.96. Finally,
good leverage points (the one we are interested in here) should be associated to absolute
SR < 1.96 and OD > 4.28. Good leverage points, by lying close to the regression hy-
perplane, do not distort the slope estimations and are therefore generally considered as
desirable in the literature.
[INSERT FIGURE II HERE]
From Figure 2 it is clear that, as explained above, no bad leverage points and only very
mild vertical outliers are present. The slope coefﬁcients should therefore be trustworthy.
In the ﬁrst column of Table 1 here below we reproduce the original results of McKenzie
and Sakho (2010) obtained using all the data since apparently no inﬂuential outlier was
present. However, from the 23 points identiﬁed as potentially outlying, 14 of them are
good leverage points. As explained in the previous section, these could false the weak
instrument test even if their effect on the slope coefﬁcient estimation is negligible. We
therefore re-estimate the key speciﬁcation of McKenzie and Sakho (2010) removing all
potential outliers and present the results in column two of Table 1. In column three we
apply a similar trimmed IV estimator but where only the 14 good leverage points iden-
tiﬁed in Figure 2 are removed. The results obtained by both outlier-free estimations are
coherent and lead to the conclusion that, contrarily to what stated in the original paper,
the instruments are weak since the F-statistic is more than halved and falls well below the
acceptable critical levels. Even if the instrumenting strategy seemed appealing, the appar-
ent strength of the instruments was due to a limited number of ﬁrms and once controlling
for this, the instrument turned-out to be weak.
6[INSERT TABLE I HERE]
5 Conclusions
In the econometric literature, it is well established that bad outliers affect IV estimations
(see Desbordes and Verardi, Forthcoming). On the contrary the effect of "good" outliers
is not well documented. In this paper, we investigate the latter focusing on their effect on
the quality of testing for weak instruments in a IV setup. We present some simulations
and show that the ﬁrst stage F-statistic increases substantially when even a limited number
of good leverage points are present. We propose a simple method to correct for this using
Stahel (1981) and Donoho (1982) multivariate location and scatter estimator.
References
Andrews, D. W., Stock, J. H., 2005. Inference with Weak Instruments. Cambridge Uni-
versity Press, Ch. 6.
Cragg, J. G., Donald, S. G., April 1993. Testing identiﬁability and speciﬁcation in instru-
mental variable models. Econometric Theory 9 (2), 222–240.
Desbordes, R., Verardi, V., Forthcoming. A robust instrumental variables estimator. Stata
Journal.
Donoho, D. L., 1982. Breakdown properties of multivariate location estimators. Ph.D.
thesis, Harvard University, Cambridge, MA.
Kleibergen, F., Paap, R., July 2006. Generalized reduced rank tests using the singular
value decomposition. Journal of Econometrics 133 (1), 97–126.
Maronna, Ricardo A., M. D. R., Yohai, V. J., 2006. Robust Statistics: Theory and Meth-
ods. Wiley Series in Probability and Statistics.
McKenzie, D., Sakho, S., January 2010. Does it pay ﬁrms to register for taxes? the impact
of formality on ﬁrm proﬁtability. Journal of Development Economics 91 (1), 15–24.
Rousseeuw, P.J., VanZomeren, B.C., 1990.Unmaskingmultivariateoutliersandleverage
points. Journal of the American Statistical Association 85 (411), 633–639.
Stahel, W. A., 1981. Robuste schatzungen inﬁnitesimale optimalitat und schatzungen von
kovarianzmatrizen. Ph.D. thesis, ETH Zurich.
7Stock, J. H., Yogo, M., 2005. Testing for weak instruments in linear IV regression. Cam-
bridge University Press, Ch. 5, pp. 80–108.
8Acknowledgements
The authors would like to thank Marjorie Gassner and Giulia Camilotti for insightful
comments, and David Mckenzie for sharing the data. Vincenzo Verardi is Associated
Researcher of the FNRS and gratefully acknowledges their ﬁnancial support.
9Table 1: Robust against outliers IV estimations
(1) (2) (3)
Dependent: lnProﬁts Original No outlier No good leverage
First stage
Instrument: lnDistance -0.13*** -0.11** -0.10**
(0.04) (0.05) (0.04)
F-statistic 12.68 5.8 5.99
Second stage
Endogenous: Formality 1.45** 0.81 0.89
(0.68) (0.89) (0.88)
Observations 369 346 355
Robust standard errors in parentheses
** p<0.01, ** p<0.05, * p<0.1
Figure 1: Bias and rate of rejection of the F-statistic
10Figure 2: Type of outliers
11DEPARTMENT OF ECONOMICS WORKING PAPERS SERIES 
UNIVERSITY OF NAMUR 
!
 
Faculty of economics, social sciences and management 
8 Rempart de la Vierge — 5000 Namur — Belgium 
 
marie-helene.mathieu@fundp.ac.be 
www.fundp.ac.be/eco/economie/recherche 
 
  RECENT ISSUES 
 
 
WP 1202  Anders Fredriksson 
Bureaucracy intermediaries, corruption and red tape. 
 
WP 1201  François Libois and Vincenzo Verardi 
Semiparametric fixed-effect estimator. 
 
 
   
  2011 ISSUES 
 
 
WP 1119  Paolo Casini and Lore Vandewalle 
Public good provision in Indian rural areas: the returns to 
collective action by microfinance groups. 
 
WP 1118  Lore Vandewalle 
The role of accountants in Indian microfinance groups: a trade-off 
between financial and non-financial benefits. 
 
WP 1117  Jean-Marie Baland, Rohini Somanathan and Lore Vandewalle 
Socially disadvantaged groups and microfinance in India. 
 
WP 1116  Jean-Marie Baland, Sanghamitra Das and Dilip Mookherjee 
Forest degradation in the Himalayas: determinants and policy 
options. 
 
WP 1115  Vincenzo Bove and Petros G. Sekeris 
Economic determinants of third party intervention in civil war. 
 DEPARTMENT OF ECONOMICS WORKING PAPERS SERIES 
UNIVERSITY OF NAMUR 
!
 
Faculty of economics, social sciences and management 
8 Rempart de la Vierge — 5000 Namur — Belgium 
 
marie-helene.mathieu@fundp.ac.be 
www.fundp.ac.be/eco/economie/recherche 
 
WP 1114  Martin Carree and Marcus Dejardin 
Firm entry and exit in local markets: market pull and 
unemployment push. 
 
WP 1113  Maoliang Ye, Sam Asher, Lorenzo Caraburi, Plamen Nikolov 
One step at a time: does gradualism build coordination? 
 
WP 1112  Marcus Dejardin 
Entrepreneurship and rent-seeking behaviour. 
 
WP 1111  Gani Aldashev and Catherine Guirkinger 
Deadly anchor: gender biais under Russian colonization of 
Kazakhstan 1898-1908. 
 
WP 1110  Vincenzo Verardi and Nicolas Debarsy 
Robison's !n-consistent semiparametric regression estimator in 
Stata. 
 
WP 1109   Catherine Guirkinger and Jean-Philippe Platteau 
The effect of land scarcity on farm structure: empirical evidence 
from Mali. 
 
WP 1108  Catherine Guirkinger and Jean-Philippe Platteau 
Transformation of the family under rising land pressure: a 
theoretical essay. 
 
WP 1107  Stéphanie Weynants 
Informal insurance with endogenous group size. 
 
WP 1106  Vincent Somville 
Daily collectors, public good provision and private consumption: 
theory and evidence from urban Benin. 
 DEPARTMENT OF ECONOMICS WORKING PAPERS SERIES 
UNIVERSITY OF NAMUR 
!
 
Faculty of economics, social sciences and management 
8 Rempart de la Vierge — 5000 Namur — Belgium 
 
marie-helene.mathieu@fundp.ac.be 
www.fundp.ac.be/eco/economie/recherche 
 
WP 1105  Philippe Lemay-Boucher, Joel Noret and Vincent Somville 
Double, double toil and trouble: an investigation on occult forces 
expenditure in Southern Benin. 
 
 
WP 1104  Jean-Philippe Platteau and Vincent Somville 
Elite capture through information distortion: uniformly distributed 
signal.  
 
WP 1103   Jean-Philippe Platteau, Vincent Somville and Zaki Wahhaj 
Elite capture through information distortion: a theoretical essay. 
 
WP 1102  Giacomo De Luca and Petros G. Sekeris 
Beyond divide-and-rule: sparking civil war to hold on power. 
 
WP 1101  Romain Houssa 
Uncertainty about welfare effects of consumption fluctuations. 

Breakdown properties of multivariate location estimators.

Does it pay ﬁrms to register for taxes? the impact of formality on ﬁrm proﬁtability.

Generalized reduced rank tests using the singular value decomposition.

Inference with Weak Instruments.

points.

Robust Statistics: Theory and Methods.

Robuste schatzungen inﬁnitesimale optimalitat und schatzungen von kovarianzmatrizen.

Testing for weak instruments in linear IV regression.

Testing identiﬁability and speciﬁcation in instrumental variable models.

Supposedly Strong Instruments and Good Leverage Points

http://www.fundp.ac.be/eco/economie/recherche/wpseries/wp/1203.pdf

Supposedly Strong Instruments and Good Leverage Points

Abstract

Similar works

Full text

Available Versions

Research Papers in Economics