Eryuruk, Gunce

Hall, Alastair R.

Jana, Kalidas

English

We assess relative performance of three recently proposed instrument selection methods via a Monte Carlo study that investigates the finite sample behavior of the post-selection estimator of a simple linear IV model. Our results suggest that no one method dominates. © 2009 Elsevier B.V. All rights reserved

The University of Manchester - Institutional Repository

A Comparative Study of Three Data-Based Methods ofInstrument SelectionGunce EryurukITAM1Alastair R. HallUniversity of Manchester2andKalidas JanaUniversity of Texas at Brownsville3August 14, 20091CIE, Instituto Tecnológico Autónomo de México, Avenida Camino a Santa Teresa 930, 10700México D.F., Mexico. E-mail: gunce.eryuruk@itam.mx2Corresponding author: Economics, School of Social Sciences, University of Manchester, Manch-ester M13 9PL, UK. E-mail: alastair.hall@manchester.ac.uk3Department of Business Administration, University of Texas at Brownsville, 80 Fort Brown,Brownsville, TX 78520, USA. E-mail: kalidas.jana@utb.eduAbstractWe assess relative performance of three recently proposed instrument selection methodsvia a Monte Carlo study that investigates the finite sample behavior of the post-selectionestimator of a simple linear IV model. Our results suggest that no one method dominates.Key Words: Instrument Selection; Canonical Correlations Information Criterion; RelevantMoments Selection Criterion; Approximate Mean Square Error Criterion; Two-Stage LeastSquares.JEL Classification: C13, C30.1 IntroductionThis paper compares the properties of three recently proposed methods for instrument se-lection,namely the approximate Mean Square Error Criterion (AMSE) of Donald & Newey(2001), the Canonical Correlations Information Criterion (CCIC) of Hall & Peixe (2003),and the Relevant Moments Selection Criterion (RMSC) of Hall, Inoue, Jana & Shin (2007).While the three methods under study are tied by the common goal of instrument selection,they are different in terms of their underlying objectives. Donald and Newey’s (2001) objec-tive is to achieve an improved finite sample risk property of the estimators. They attain thisgoal by minimizing the approximate Mean Square Error Criterion based on higher-orderasymptotics. The objective of Hall & Peixe (2003) and Hall, Inoue, Jana & Shin (2007),on the other hand, is to achieve an improved quality of asymptotic approximation to thefinite sample behavior of the estimators. They gain this objective by eliminating the redun-dant moment conditions based on certain canonical correlations: CCIC exploits explicitlythe canonical correlations (CCs) between the regressors and instruments; RMSC exploitsimplicitly the long run canonical correlations (LRCCs) between the unknown true scorevector and the product of the instrument vector and error. While the properties of eachof the methods have been explored by their proponents, there have been no comparativestudies of these methods to date. Our paper intends to fill that gap.2 Three methods of instrument selectionIn this section, we describe the three methods of instrument selection mentioned in theIntroduction in the context of the model used in our simulation study. Accordingly, considerthe model1yt(1×1)= xt(1×1)θ0(1×1)+ ut(1×1),xt = z′t(1×q)π0(q×1)+ et(1×1), t = 1, 2, . . . , T(1)where  utet∣∣∣∣ zt iid∼ N 00 , σ2u σueσue σ2e . (2)We assume that the candidate set of moment conditions is given byE[ztut(θ0)] = 0 (3)where ut(θ) = yt − xtθ and zt is the q × 1 vector of instruments in (1). In this case, theonly difference between various choices of moments lies in the chosen instrument vector andso we refer to zt as the candidate set of instruments. We use a q × 1 selection vector c inthe notation of Andrews (1999) to denote which elements of the instrument vector zt areincluded in a particular moment condition: if cj = 1 then the jth element of zt is included;if cj = 0 then the jth element of zt is excluded. The case in which all instruments are usedis denoted by c = ιq where ιq is q × 1 vector of ones. The moments associated with c arewritten asE[zt(c)ut(θ0)] = 0 (4)where zt(c) = S(c)zt and S(c) is a selection matrix that picks out the elements of ztindicated by c. Note that |c| = c′c equals the number of elements in zt(c). The set of allpossible selection vectors is denoted by C, that isC ={c ∈ <q; cj = 0, 1, for j = 1, 2, . . . q, and c = (c1, . . . cq)′, |c| ≥ 1}.For brevity, statistics of interest are now indexed by c and so, for example, θ̂T (c) denotesthe 2SLS estimator based on (4).Specializing the definitions to the model of this section, the three instrument selectionmethods are as follows.2CCIC: The selected instrument vector is ĉCCIC where1ĉCCIC = argminc∈C{T ln[1−R2T (c)] + (|c| − 1) ln[T ]}(5)where RT (c) is the sample multiple correlation coefficient between xt and zt(c), and T isthe sample size.RMSC: The selected instrument vector is ĉRMSC where2ĉRMSC = argminc∈C RMSC(c) (6)andRMSC(c) = ln[σ̂2u(c){T−1T∑t=1xtzt(c)′{T−1T∑t=1zt(c)zt(c)′}−1× T−1T∑t=1zt(c)xt}−1 + (|c| − 1)√Tln√T (7)with σ̂2u(c) = T−1 ∑Tt=1{ut(θ̂T (c)}2.Approximate MSE criterion: The selected instrument vector is ĉAMSE whereĉAMSE = argminc∈C AMSE(c) (8)andAMSE(c) = {σ̂au}2|c|2T+ σ̃2u(R̂(c)− σ̂2 |c|T)(9)1Note that we use the penalty term associated with BIC. Other choices are possible but Hall & Peixe(2003) found this choice to work best of those they considered; see Hall & Peixe (2003) for further discussion.2Note that we again use the penalty term associated with BIC (Schwarz, (1978)). As with CCIC, otherchoices are possible but this choice appears to work best; see Hall, Inoue, Jana & Shin (2007) for furtherdiscussion.3where σ̂au = T−1ã′ũ, ã = ẽ[π̃′Z(ιq)′Z(ιq)π̃/T ]−1, ẽ = [I−P (ιq)]x, P (c) = Z(c)[Z(c)′Z(c)]−1Z(c)′,Z(c) is a T ×|c| matrix with tth row zt(c)′, π̃ is a preliminary estimator of π0, ũ = y−xθ̃ is aT×1 residual vector with tth element ut(θ̃), θ̃ is a preliminary estimator of θ0, σ̃2u = T−1ũ′ũ,σ̂2 = T−1ã′ã, and R̂(c) is a measure of the goodness of fit of the first stage reduced formmodel, xt = zt(c)′π(c) + et(c). Donald & Newey (2001) consider two choices for R̂(c); forbrevity, we focus on just one of these, the cross-validation criterion,R̂cv(c) =1TT∑i=1[âi(c)]2[1− Pii(c)]2where âi(c) is the ith element of â(c) = ê(c)[π̃′Z(ιq)′Z(ιq)π̃/T ]−1 and ê(c) = [IT − P (c)]x.3 Simulation design and resultsWe now describe the simulation design. Data are generated from (1)-(3) with p = 1,θ0 = 0.1, different specifications of π0 described below, [ut, et, z′t]′ ∼ N(0,Σ), Σ is a matrixwhose diagonal elements are all one and whose only non-zero off diagonal elements arecov(ut, et) = σue ∈ {0.1, 0.5, 0.9}, and sample size T ∈ {100, 500}. We consider threespecifications for π0 in the equation for xt:• Model 1: Declining coefficientsπ(1)0,i = k(q)(1− iq + 1)4for i = 1, . . . , q, (10)where the constant k(q) is chosen so that the squared population multiple correlationbetween xt and zt, R21 say, is 0.5.• Model 2: Equal coefficientsπ(2)0,i = π(2)0 =√0.52q(1− 0.52)for i = 1, . . . , q, (11)which implies R21 = 0.5.4• Model 3: Two non-zero coefficients and the rest zeroπ0,i = k(q)(1− iq + 1)4, i = 1, 2; (12)= 0 for i = 3, . . . , q,where the constant k(q) is chosen so that (12) holds for R21 = 0.5. Note that(zt,3, zt,4, . . . , zt,q) are redundant given (zt,1, zt,2) and so (zt,1, zt,2) are referred to asthe “relevant” instruments in this model.There are two natural approaches to instrument selection: a sequential strategy in whichonly q possible choices are considered in a sequence such as (z1), (z1, z2), (z1, z2, z3) etc.,and a strategy in which all possible combinations are considered.In their investigation of AMSE(c), Donald & Newey (2001) use Models 1 and 2 with q ∈{20, 30} and employ a sequential strategy in which the sequence of instruments consideredis given by c1, c2, . . . , cq where ci = [ı′i, 0′q−i]′ and ιi is a i × 1 vector of ones and 0q−i isa (q − i) × 1 vector of zeros, i = 1, 2, . . . , q. In our simulations, we generate data fromboth Models 1 and 2, and select instruments using this sequential strategy based on allthree instrument selection criteria. However, for brevity we only report results for q = 20;the results for q = 30 are qualitatively the same as for q = 20 and are available from theauthors upon request. In their evaluation of CCIC(c), Hall & Peixe (2003) consider Model3 with q = 8 and selection is based on a comparison of all possible combinations of allinstruments, that is {c ∈ C}; note that there are 255 possible combinations within Model3.3 We compare the performance of all three criteria within this design.To assess the relative performance of the CCIC, the RMSC and the AMSE, we focus onthe finite sample performance of the post-selection 2SLS estimator. To assess the adequacyof standard asymptotic theory, we report the median bias and the coverage probability of90% confidence intervals based on conventional asymptotic theory. However, due to poten-tial concerns about non-existence of moments, we also report the median absolute deviation3Hall, Inoue, Jana & Shin (2007) also use this set up in their evaluation of RMSC(c).5and the interdecile range. To compute AMSE, in Models 1 and 2, we obtain the prelim-inary estimators π̃ and θ̃ using the number of instruments that minimized the first stagecross-validation criteria. In Model 3 we obtain π̃ and θ̃ using all eight instruments. Theresults are presented in Tables 1 through 3. All results correspond to 10,000 replications.We begin with Model 1. Following Donald & Newey (2001), we consider the sequentialstrategy for instrument selection described above for this model. Notice that the selectionstrategy involves including the instruments in the order of their explanatory power for xt,and so the order of selection is from “best to worst”. This scenario corresponds to thecase in which a practitioner has a priori information about the relative importance of theinstruments. The results for this case are shown in Table 1. It can be seen from the tablethat for both sample sizes, T = 100 and T = 500, and for all three values of endogeneityconsidered, the median bias and coverage probabilities under AMSE, CCIC and RMSC aresimilar and thus no unique ranking among the three criteria emerges. In terms of medianabsolute deviation and interdecile range, AMSE and CCIC are similar but both dominateRMSC. It can also be seen from Table 1 that the median bias increases and the coverageprobability distortions worsen with the increase in endogeneity under all three criteria.Table 2 contains the results for Model 2. Here again, we use the sequential selectionstrategy. However, this time, all the instruments have equal explanatory power for xtand so there are no “best” or “worst” instruments per se. It can be seen that for allconfigurations considered the median biases under AMSE and CCIC are similar and smallerthan the median bias under RMSC. Interestingly, the biases under each criterion tend tobe larger under Model 2 than under Model 1. The coverage probabilities also exhibitgreater distortions in this model than under Model 1. No one criterion yields coverageprobabilities that are closer to the nominal level than the others over all configurations. Asin Model 1, the performance of the post-selection estimator deteriorates as the degree ofendogeneity increases. In terms of both median absolute deviation and interdecile range,AMSE dominates for σue ∈ {0.1, 0.5} but no criterion dominates for σue = 0.9.6Table 3 gives the results for Model 3. Recall that in this case only two of the eightinstruments can explain xt and selection is over all possible combinations. Following Halland Peixe (2003) and Hall, Inoue, Jana & Shin (2007), we consider selection over all possiblecombinations of instruments for this case. The results reveal that under each parameterconfiguration the median biases and coverage probabilities are similar for each criterion,and thus no unique ranking emerges. In fact, in all configurations, all three methodsperform very well and the empirical coverage rate under all three methods is almost equalto the nominal rate of 90%. All three methods also lead to post-selection estimators withcomparable median absolute deviations and interdecile ranges.Taking all our simulation results together, it seems to us that no one instrument selectioncriterion clearly dominates the others in terms of median bias, the closeness of the coverageprobability to the nominal level, and dispersion. In some designs, we find that all threecriteria lead to post-selection estimators with small bias and coverage probabilities closeto the nominal level; but in other designs, we find that all three criteria lead to post-selection estimators whose finite sample behaviour is poorly approximated by conventionalasymptotic theory.ReferencesAndrews, D. W. K. (1999). Consistent moment selection procedures for generalized methodof moments estimation. Econometrica, 67, 543–564.Donald, S. G. & Newey, W. K. (2001). Choosing the number of instruments. Econometrica,69, 1161–1192.Hall, A. R., Inoue, A., Jana, K., & Shin, C. (2007). Information in generalized method ofmoments estimation and entropy based moment selection. Journal of Econometrics, 138,488–512.7Hall, A. R. & Peixe, F. P. M. (2003). A consistent method for the selection of relevantinstruments. Econometric Reviews, 22 (3), 269–287.Schwarz, G. (1978). Estimating the dimension of a model. The Annals of Statistics, 6,461–464.8Table 1: Properties of AMSE(c), CCIC(c) & RMSC(c)Model 1: sequential selection strategyqmax = 20 T=100 T=500R21 = 0.5; σue = 0.1.AMSE CCIC RMSC AMSE CCIC RMSCMed Bias 0.010 0.006 0.008 0.003 0.002 0.002Cov Rate 0.886 0.906 0.914 0.898 0.904 0.905Interdecile Range 0.251 0.257 0.291 0.114 0.114 0.127Med AD 0.066 0.068 0.077 0.030 0.030 0.033R21 = 0.5; σue = 0.5.AMSE CCIC RMSC AMSE CCIC RMSCMed Bias 0.036 0.035 0.037 0.009 0.008 0.010Cov Rate 0.850 0.861 0.875 0.892 0.894 0.894Interdecile Range 0.261 0.256 0.288 0.113 0.114 0.124Med AD 0.068 0.067 0.075 0.030 0.031 0.033R21 = 0.5; σue = 0.9.AMSE CCIC RMSC AMSE CCIC RMSCMed Bias 0.053 0.060 0.059 0.014 0.016 0.016Cov Rate 0.818 0.800 0.832 0.873 0.871 0.873Interdecile Range 0.252 0.244 0.273 0.114 0.112 0.125Med AD 0.066 0.063 0.072 0.030 0.029 0.032Notes: Med Bias = median bias, Cov Rate = the empirical coverage of anominal 90% confidence interval, Med AD = median absolute deviation.9Table 2: Properties of AMSE(c), CCIC(c) & RMSC(c)Model 2: sequential selection strategyqmax = 20 T=100 T=500R21 = 0.5; σue = 0.1.AMSE CCIC RMSC AMSE CCIC RMSCMed Bias 0.013 0.022 0.021 0.004 0.003 0.008Cov Rate 0.891 0.918 0.939 0.897 0.895 0.909Interdecile Range 0.240 0.418 0.403 0.113 0.113 0.178Med AD 0.062 0.088 0.102 0.029 0.030 0.046R21 = 0.5; σue = 0.5.AMSE CCIC RMSC AMSE CCIC RMSCMed Bias 0.094 0.096 0.108 0.018 0.018 0.038Cov Rate 0.728 0.771 0.841 0.865 0.863 0.858Interdecile Range 0.253 0.409 0.392 0.112 0.111 0.172Med AD 0.063 0.082 0.100 0.029 0.029 0.044R21 = 0.5; σue = 0.9.AMSE CCIC RMSC AMSE CCIC RMSCMed Bias 0.166 0.172 0.185 0.046 0.033 0.064Cov Rate 0.634 0.479 0.637 0.726 0.782 0.749Interdecile Range 0.417 0.376 0.358 0.110 0.108 0.170Med AD 0.094 0.069 0.092 0.028 0.029 0.044Notes: See Table 1.10Table 3: Properties of AMSE(c), CCIC(c) & RMSC(c)Model 3: selection over all possible combinationsqmax = 8 T=100 T=500R21 = 0.5; σue = 0.1.AMSE CCIC RMSC AMSE CCIC RMSCMed Bias 0.008 0.004 0.004 0.002 0.000 0.001Cov Rate 0.892 0.897 0.900 0.899 0.895 0.897Interdecile Range 0.257 0.259 0.262 0.114 0.116 0.116Med AD 0.066 0.068 0.068 0.030 0.031 0.030R21 = 0.5; σue = 0.5.AMSE CCIC RMSC AMSE CCIC RMSCMed Bias 0.020 0.012 0.017 0.004 0.002 0.001Cov Rate 0.874 0.895 0.890 0.895 0.904 0.902Interdecile Range 0.256 0.255 0.261 0.116 0.114 0.113Med AD 0.067 0.067 0.067 0.030 0.030 0.030R21 = 0.5; σue = 0.9.AMSE CCIC RMSC AMSE CCIC RMSCMed Bias 0.022 0.020 0.028 0.003 0.003 0.003Cov Rate 0.868 0.867 0.869 0.894 0.896 0.901Interdecile Range 0.261 0.256 0.265 0.117 0.115 0.115Med AD 0.068 0.067 0.068 0.030 0.031 0.031Notes: See Table 1.11

A comparative study of three data-based methods of instrument selection

https://pure.manchester.ac.uk/ws/files/19790986/POST-PEER-REVIEW-NON-PUBLISHERS-DOCUMENT.PDF

A comparative study of three data-based methods of instrument selection

Abstract

Similar works

Full text

Available Versions

The University of Manchester - Institutional Repository