By William Sullivan,2014-06-12 00:31
25 views 0

    Are Nearly “Exogenous Instruments” Reliable?

     Daniel Berkowitz

     Department of Economics, University of Pittsburgh,

    Pittsburgh, PA 15260, USA

     Mehmet Caner*

    Department of Economics, North Carolina State University,

     PO Box 8110

     4168 Nelson Hall

    Raleigh, NC, 27695, USA

    Tele: 919-513-0853

    Fax: 919-515-5613

     Ying Fang

     Wang Yanan Institute for Studies in Economics, WISE,

     Xiamen University, China,



    We show that when instruments are nearly exogenous, the two stage least squares t-statistic unpredictably over-rejects or under-rejects the null hypothesis that the endogenous regressor is insignificant and Anderson-Rubin test over-rejects the null. We

    prove that in the limit these tests are no longer nuisance parameter free.

JEL Code: C30.

    Keywords: Valid Instruments, Weak Identification, Inference.

* Corresponding Author

I. Introduction

     Instrumental variable methods are used to identify causal relationships. Researchers pick relevant instruments that should be related to the endogenous 1explanatory variable both on the basis of a priori argument and statistically.

     Instruments must also be exogenous; that is, they are not related to the outcome variable after controlling for relevant explanatory variables. Just whether or not the exclusion restriction is satisfied is controversial for many other seemingly exogenous instruments. For example, Angrist (1990) argues that draft lottery numbers are instruments for testing whether serving in Vietnam affects the earnings of men in the civilian sector because these numbers influence earnings purely through military service. However, Wooldridge (2002, p.88) argues that because civilian employers are more likely to invest in job training for employees who have high draft numbers, these numbers could also influence earnings through job training, which is unobservable.

    We show that the standard t-test statistic is unreliable: even when the instrument is very close to being exogenous, the t-test grossly and unpredictably over-rejects or under-rejects the null that the endogenous regressor is insignificant, and the Anderson-Rubin test over-rejects the null. We prove these results in the limit and in small samples. And, to our knowledge, these are new theoretical results.

     2. Inference Using the Standard Test Statistics

    In this section we relax the assumption that instruments must be exogenous and introduce a definition of “near exogeneity.” Suppose we want to check for whether not

    an institution, say property rights enforcement, influences long term growth in a sample 2of countries. If we suspect that institutions are endogenous and we also believe that a linear specification is appropriate, we would estimate and compute test-statistics for the following simple linear simultaneous equations model (Hausman, 1983; Phillips, 1983):

     (1) LRGr?;?INST;u01

     (2) INST(;Z(;V01

    Equation (1) is the structural equation, where LRGr is an nx1 vector of long run growth, INST is an nx1 vector of institutions, and u is an nx1 vector of structural error terms that

    2have zero mean and finite variance . Equation (2) is the reduced form, Z is an (?u

    nxk matrix of instruments and V is an nx1 vector of reduced form errors that have zero

     1 Instruments that marginally satisfy this requirement are denoted weak and are the subject of a large and growing literature (see Staiger and Stock, 1997; Stock et al, 2000).

     2 We just consider one kind of institution and, hence, one endogenous variable for expositional simplicity. Our method also works for multiple endogenous variables. See Acemoglu and Johnson (2006) for an analysis of how instrumental variables can be used to identify how two endogenous institutions, property rights (measured by a survey of risk of expropriation) and efficiency of contracts (measured by an index of legal formalism), can affect long run growth.


    2. The error terms u and V may be correlated and n means and finite variance. (?V

    represents the number of countries. The parameters, are unknowns, ?,?,(and(0101

    and, for notational conventional, we denote . Other ?{?,?},({(,(}0101

    covariates, for example, population, latitude and education, can be added to the system in 3equations (1) and (2) without loss of generality.

    In order to determine whether or not institutions matter, we estimate the unknown parameter β and use test-statistics to check whether β = 0. To do this properly, we need 11

    valid instruments that are both relevant and exogenous. As previously discussed, relevant instruments are picked on the basis of a theoretical, institutional and/or historical argument, and are validated ex post by estimating the reduced form. The second criterion for validity is that instruments are exogenous, which implies they are orthogonal to the error term in the structural equation:

    ' (3) Exogenous?CovZu0ii

    It is generally difficult, as we have previously argued, to find instruments that satisfy this strong condition. In particular, while these instruments influence long run growth in the structural equation primarily through institutions, they may also be weakly correlated with unobserved factors that can also influence long term growth. We model this potential small correlation as “nearly exogenous” which is a local to zero setup:

    ' (4) NearlyExogenous?CovZuC/nii

    where C is an nx1 vector of constants that is contained in compact set.

    'If we choose to capture near exogeneity, then the test statistics CovZuCii

    always diverge in the limit. Thus, this assumption does not provide any guidance for finite sample behavior when there is some mild correlation between the instrument and error.

    In what follows, small sample simulation methods are used to show that even a slight relaxation of the exogeneity assumption in equation (3) makes the standard test statistics unreliable. Suppose we employ the TSLS t-test to determine whether or not


    institutions matter. Denoting the H and Has the null and the alternative and as ?01 1,TSLS

    the TSLS estimator of we use the t-statistic to test ?,1

    , against H:?001

    , where the t-statistic is given by H:??011


    t?/avar? (5) 1,TSLS1,TSLS

     3 By the Frisch-Waugh-Lovell Theorem, we can always project out these covariates and obtain the system in equations (1) and (2) (see Davidson and McKinnnon, 1993, p.19).


     We generate i.i.d. data for the one instrument, the structural error term and reduced form, (Z,u,V), from a joint normal distribution N(0, Λ) and

    CovZu10??ii??CovZuCovVu 1.(6)??iiii


where Cov Z‟u measures the correlation between the instrument Z and the error term u, ii

    and Cov V‟u measures the endogeneity of institutions, which is set to 0.25 in all ii

    simulations. When the iid data (Z,u,V) are generated, we can derive the observation of and INST and LRGr by using equations (1) and (2) and specified true values of

    . Based on the information of (LRGr, INST, Z), we compute the t-statistic and ?and(11

    then test whether the null of β= 0 can be rejected at the 5% level by using the critical 1

    value 1.95. We replicate the simulation by 1000 times to derive the distribution of the t-statistic and calculate the actual rejection probability which is reported in Table 1.

     Table 1 reports rates of right hand side and left hand false rejection when the instrument is more weakly correlated with the error term: Cov Z‟u= 0.06 or -0.06 and ii

    illustrates that as the absolute value of the correlation decreases, the size problems of the two-sided t-test are mitigated. When the correlation is positive there is a 9.4% false rejection rate on the right hand side, a conservative 0.4% rate from the left hand side and an overall 9.8% false rejection rate. When, the correlation is negative, the rates of false rejection on the right hand and left hand sides are 0.6% and 7.2%, respectively, and the overall false rejection rate is 7.9%.

     Suppose we test the null against the alternative using Anderson-Rubin (Anderson and Rubin, 1949) test:

     4 (7) AR(?0)LRGr'PLRGr/(LRGr'MLRGr)/(n2)1zz

    1Here, is the test statistic for the null, is the projection matrix AR(?0)PZ(Z'Z)Z1z

    and . MIPzz


    Table 1 illustrates that the small sample problems associated with the Anderson-Rubin test (for herein, denoted the AR-test) are also diminished when the instrument is less endogenous. When the correlation decreases to 0.06, the AR-test falsely rejects 9.4% of the time. Since it is not possible to calculate the absolute value of the correlation between the instruments and structural error, it is not possible to adjust for this small sample distortion and the AR-test is also unreliable.

     4 We can generalize this test statistic to allow for multiple endogenous explanatory variables and as least as many instruments.


3. Large Sample Distributions

     This section adds to the bad news: we show that the shifts in test statistic distributions observed in the small sample simulations also hold in limit. For the next three sections of the paper, we generalize the simultaneous equations system equations (1) and (2) to model a more general system with m ? 1 endogenous explanatory variables,

    and k ? m instruments:



    where y and Y are respectively and nx1 vector and nxm matrix of endogenous explanatory variables, Z is an nxk matrix of instruments, u is an nx1 vector of structural errors, V is an nxm matrix of reduced form errors, and the errors have zero means and finite variance, and u and V are correlated with each other. As noted before, other exogenous covariates can be added to the system.

     In the next theorem, we show that near exogeneity shifts the asymptotic distribution of the t-statistic to a normal distribution with non-zero mean.

    Theorem 1: Suppose that the instrument is nearly exogenous according to (4), and the standard Assumption 2 in the appendix holds. Then,

    d11/2 (8) tN[((('Q()('C,1]uzz

    2Q is the second moment matrix of where(isthesquarerootof(,andzzuu


    Proof. See the Appendix.

     According to Theorem 1, the mean of the distribution depends upon the parameter C, which, by equation (4), is related to the small correlation between structural error and instruments. When C=0 and the instruments are exogenous, the t-statistic converges to the standard normal distribution. When C>0 (given ), the distribution shifts to the (0

    right. When C<0 (given ), the distribution shifts to the left. Since we cannot (0

    consistently estimate C let alone know its sign, we cannot use this large sample theorem to improve inference.

     The next theorem characterizes the impact of near exogeneity on the distribution of the AR-test, which is now more generally defined from equation (7) for k instruments and m endogenous explanatory variables:

     (7*) AR(?)(yY?)'P(yY?)/(yY?)'M(yY?)/(nkm)00z00z0

We use this statistic to test against where is the true value. H:??H:????10000

    Theorem 2: Suppose that the instrument is nearly exogenous according to (4), and the standard Assumption 2 in the appendix holds. If the null hypothesis is ??,then0


    d2 (9) AR(?)?()K0

    2is a non-central chi-square distribution with k degrees of freedom and the where?()K

    12non-centrality parameter . C'C,where(Quzz

    Proof. See the Appendix.

     According to Theorem 2, the mean of the non-centrality parameter is quadratic in parameter C. Therefore, when C=0 the AR-test converges to the centered chi-square distribution, and when C?0 the distribution shifts to the right. Again, since we do not know C, we cannot use these theorems to obtain appropriate critical values. The convergence is uniform.

4. Conclusion

    This article analyzes the limit theory when there are both weakly identified as well as nearly exogenous instruments. We show that Anderson-Rubin test is no longer asymptotically pivotal. In future research we consider how to remedy this problem by using a delete-d jackknife bootstrap procedure.


    In the beginning of this appendix, we first describe the near exogeneity assumption and some moment conditions that are required to obtain the theorems in the paper. Assumptions 1 and 2 are sufficient for Lemma 1, Theorem 1 and Theorem 2.

    Assumption 1: Near Exogeneity ~? , where is a fixed K1EZuC/NCii


    Assumption 2: The following limits hold jointly when the sample size N

    converges to infinity:

    p22(a) , where , and are ((uu/N,Vu/N,VV/N)((,,)VuVVuuVuVV

    respectively a 11 scalar, an vector and an matrix. mmm1


    (b) KK where is a positive definite, finite matrix. QZZ/NQZZZZ


    (c) , and (Z'u/N,Z'V/N)(,)ZuZV

    ????C??Zu???? ?N,?Q??zz????0????ZV??

    2??(uVu??where . ????VuVV


    These convergences in Assumption 2 are not primitive assumptions but hold under weak primitive conditions. Parts (a) and (b) follow from the weak law of large numbers, and Part (c) follows from triangular arrays central limit theorem. Instead of a mean zero normal distribution in Staiger and Stock (1997), the in (c) is a normal Zu

    distribution with nonzero mean, which is a drift term C coming from the near exogeneity

    2;?assumption. For any independent sequence , if for some ~?EZu?Zuiiii

     for all , then Liapunov's theorem leads to the limiting results in i1,2,3,...,N?0

    (c); see James Davidson (1994).

     **Lemma 1. If Assumptions 1 and 2 hold for the model defined by (1) and (2), then


    the TSLS estimator is consistent and ?TSLS

    d?121 N(??)N(((Q()(C,(((Q())TSLSZZuZZ0pp22where , . uu/NE(u)(ZZ/NE(ZZ)QiuiiZZ

    Proof of Lemma 1: We know that

    ?1 ?(YPY)(YPy).TSLSZZ

    So we have




    By Assumption 2 and equation (2*), we can obtain that

    YZZZZY11[()()()]NNN p1((Q()ZZ

    Now, we consider NNZu11[ZuE(Zu)];E(Zu) iiiiii??NNNi1i1

    Combining Assumptions (1) and (2), we obtain

    dZu2N[C,(Q]. uZZN

    Then the result in the lemma follows directly. Q.E.D.

    Lemma 1 summarizes the limiting results of the TSLS estimator under near exogeneity. The reason why we can obtain a consistent estimator under near exogeneity is because the correlation between instruments and structural errors shrinks toward zero asymptotically. When C=0, we can obtain the regular results of the TSLS estimator under the orthogonality condition. Instead of a normal distribution with a zero mean, near exogeneity can shift the distribution away from mean zero. The nonzero mean depends on an unknown local to zero parameter C which is impossible to be estimated consistently (Donald W.K. Andrews, 2000).


    Proof of Theorem 1: The result in the theorem directly follows from Lemma 1.


     Proof of Theorem 2: The Anderson-Rubin test is given by

    1 AR?()(yY?)P(yY?)/(yY?)M(yY?)00Z00Z0NK

    We first observe that


    1/2Define . Parts (b) and (c) in Assumption 2 implies: )PuZ

    d1/21/22 )Q?N(QC,().ZuZZZZu

Next, note that


    1uMu ZNK


    2By part (a) in Assumption 2, the first term converges in probability to , and the last (uterm tends to zero. We have

    p12 (yY?)M(y?)(.0Z0uNK

    So d21??AR()(CC),where0K 2(Q Q.E.D.uZZ



    Acemoglu, D. and S. Johnson, 2006, Unbundling Institutions, Journal of Political Economy 113, 949-995.

    Anderson, T.W. and H. Rubin, 1949, Estimators of the Parameters of a Single Equation in a Complete Set of Stochastic Equations, The Annals of Mathematical Statistics 20, 46-63.

    Andrews, D. W.K. 2000, Inconsistency of the Bootstrap with a Parameter Is on the Boundary of the Parameter Space, Econometrica 68, 399-405.

    Angrist, J. D. 1990, Lifetime Earnings and the Vietnam Era Draft Lottery: Evidence from Social Security Administrative Records, American Economic Review 80, 313-336.

    Davidson, J. 1994, Stochastic Limit Theory: An Introduction for Econometricians, (Oxford University Press, Cambridge).

Davidson, R.. and J. G. McKinnnon 1993, Estimation and Inference in Econometrics,

    (Oxford University Press, New York).

    Hausman, J. A. 1983, Specification and Estimation of Simultaneous Equations Models, in: Zvi Grilliches and Michael D. Intrilligator, eds., Handbook of Econometrics, Volume 1 (Amsterdam, North Holland) Chapter 7.

    Phillips, P. C.B. 1983, Exact Small Sample Theory in the Simultaneous Equations Model, in: Zvi Grilliches and Michael D. Intrilligator, eds., Handbook of Econometrics, Volume

     (Amsterdam, North Holland) Chapter 8. 1

    Staiger, D. and J. H. Stock, 1997, Instrumental Variables Regression with Weak Instruments, Econometrica, 65, 557-586.

Stock, J. H., J. H. Wright, and M. Yogo 2002, A Survey of Weak

    Instruments and Weak Identification in Generalized Method of Moments, Journal of Business and Economic Statistics, 20, 518-529.

    Wooldridge, J. M. 2002, Econometric Analysis of Cross Section and Panel Data. (MIT Press, London and Cambridge).


    Table 1: Test Statistics

    Sample Size = 100, and 1,000 simulations

    Truth is that Institutions Do Not Matter

    u Actual Actual Actual Test-Nominal Cov Zii

    rejection rejection rejection statistic 5% Critical

    rate rate (RHS) rate (LHS) Values

    t-statistic ?1.95 0.06 9.8% 9.4% 0.4% t-statistic ?1.95 -0.06 7.9% 0.6% 7.2% AR test 3.85 ?.0.? 9.4% n.a. n.a.

    t-statistic ?1.95 0.10 19.4% 19.2% 0.2% t-statistic ?1.95 -0.10 14.3% 0.3% 14.0% AR test 3.85 ?.0?. 17.7% n.a. n.a.




     ?记者 孙彬 管建涛 连振祥 吉哲鹏 娄辰 李松

     南京 哈尔滨 兰州 昆明 济南 重庆报道






Report this document

For any questions or suggestions please email