Advanced Econometrics - Lecture 3 Instrumental Variable and GMM Estimator Advanced Econometrics - Lecture 3 n The OLS Estimator: With Error Correlated Regressors n Correlated Regressors: Some Cases n Instrumental Variables (IV) Estimator: The Concept n IV Estimator: The Method n Calculation of the IV Estimator n An Example n The GIV Estimator n The Generalized Method of Moments n Some Tests OLS Estimator OLS Estimator, cont’d The Assumption (A7): E{x[t ]ε[t]} = 0 for all t Implication of (A7): for all t, each of the regressors is uncorrelated with the current error term, no contemporary correlation Stronger assumptions – (A2), (A8), (A10) – have same consequences (A7) is required for unbiasedness and consistency of the OLS estimator In reality, the assumption E{x[t] ε[t]} = 0 is not always true Examples of situations with E{x[t] ε[t]} ≠ 0: n Regression on the lagged dependent variable with autocorrelated error term n Observations of a regressor with measurement errors n Endogeneity of regressors n Simultaneity Advanced Econometrics - Lecture 3 n The OLS Estimator: With Error Correlated Regressors n Correlated Regressors: Some Cases n Instrumental Variables (IV) Estimator: The Concept n IV Estimator: The Method n Calculation of the IV Estimator n An Example n The GIV Estimator n The Generalized Method of Moments n Some Tests Regressor with Measurement Error y[t] = β[1] + β[2]w[t] + v[t] with white noise v[t], V{v[t]} = σ[v]², and E{v[t]|w[t]} = 0; conditional expectation of y[t] given w[t] : E{y[t]|w[t]} = β[1] + β[2]w[t] e.g., w[t] is household income, y[t] is household saving Measurement process: x[t] = w[t] + u[t] where u[t ]is (i) white noise with V{u[t]} = σ[u]², (ii) independent of v[t], and (iii) independent of w[t] The model to be analyzed is y[t] = β[1] + β[2]x[t] + ε[t] with ε[t] = v[t] - β[2]u[t] n E{x[t] ε[t]} = - β[2 ]σ[u]² ≠ 0: requirement for consistency is violated n x[t] and ε[t] are negatively (positively) correlated if β[2] > 0 (β[2] < 0) Measurement Error, cont‘d Inconsistency of b[2] plim b[2 ]= β[2] + E{x[t] ε[t]} / V{x[t]} β[2] is underestimated Inconsistency of b[1] plim (b[1 ]- β[1])= - plim (b[2 ]- β[2]) E{x[t]} given E{x[t]} > 0 for the reported income: β[1] is overestimated; inconsistency carries over The model does not correspond to the conditional expectation of y[t] given x[t]: E{y[t]|x[t]} = β[1] + β[2]x[t] - β[2] E{u[t]|x[t]} Dynamic Regression Allows to model dynamic effects of changes of x on y: y[t] = β[1] + β[2]x[t] + β[3]y[t-1] + ε[t] OLS estimators are consistent if E{x[t] ε[t]} = 0 and E{y[t-1] ε[t]} = 0 AR(1) model for ε[t]: ε[t] = ρε[t-1] + v[t] v[t] white noise with σ[v]² From y[t] = β[1] + β[2]x[t] + β[3]y[t-1] + ρε[t-1] + v[t] follows E{y[t-1]ε[t]} = β[3] E{y[t-2]ε[t]} + ρ²σ[v]²(1 - ρ²)^-1^ which indicates that y[t-1] is correlated with ε[t] OLS estimators not consistent The model does not correspond to the conditional expectation of y[t] given the regressors x[t] and y[t-1]: E{y[t]|x[t], y[t-1]} = β[1] + β[2]x[t] + β[3]y[t-1] + E{ε[t] |x[t], y[t-1]} Omission of a Relevant Regressor Endogenous Regressors: Consequences Model y[i] = x[i]‘β + ε[i] with white noise ε[i], V{ε[i]} = σ[ε]² or in matrix notation: y = Xβ+ ε Violation of (A7): E{X‘ε} ≠ 0 OLS estimator b = β + (X‘X)^-1X‘ε n E{b} ≠ β, b is biased; bias E{(X‘X)^-1X‘ε} difficult to assess n plim b = β + Σ[xx]^-1 q with q = plim(T^-1X‘ε) q For q = 0 (regressors and error terms are asymptotically uncorrelated), OLS estimators b are consistent also in case of non-exogenous regressors q For q ≠ 0 (error terms and at least one regressor are asymptotically correlated): plim b ≠ β, the OLS estimators b are not consistent Simultaneity The regressor x[t] has an impact on y[t]; at the same time y[t] has an impact on x[t ] Example: Consumption function n x[t] per capita income; y[t] per capita consumption y[t] = β[1] + β[2]x[t] + ε[i ](A) β[2]: marginal propensity to consume, 0 < β[2] < 1 n z[t]: per capita investment (exogenous, E{z[t] ε[i]} = 0) x[t] = y[t] + z[t] (B) n Both y[t] and x[t] are endogenous: E{y[t] ε[i]} = E{x[t] ε[i]} = σ[ε]²(1 – β[2])^-1 n Equations (A) and (B) are the structural equations, the coefficients are behavioral parameters n OLS estimator b[2] from (A) is inconsistent plim b[2] = β[2] + Cov{x[t] ε[i]} / V{x[t]} = β[2] + (1 – β[2]) σ[ε]²(V{z[t]} + σ[ε]²)^-1 Advanced Econometrics - Lecture 3 n The OLS Estimator: With Error Correlated Regressors n Correlated Regressors: Some Cases n Instrumental Variables (IV) Estimator: The Concept n IV Estimator: The Method n Calculation of the IV Estimator n An Example n The GIV Estimator n The Generalized Method of Moments n Some Tests Example: Consumption Function Data: annual differences of the logarithmic series PCR (c, Private Consumption) and PYR (y, Disposable Income of Households) from the AWM database (1970:1 to 2003:4) Fitted model ĉ = 0.011 + 0.718 y with t = 15.55, R^2 = 0.65, DW = 0.50 Attention! Income y[t] in c[t] = β[1] + β[2]y[t] + ε[t] might be correlated with the errors: income is used for consumption and other expenditures y[t] = c[t] + z[t] where z[t] includes all income components besides consumption Risk of inconsistency due to correlated y[t] and ε[t] Consumption Function, cont’d Alternative model: c[t] = β[1] + β[2]y[t-1] + ε[t] n y[t-1] and ε[t] are certainly uncorrelated n No risk of inconsistency due to correlated y[t] and ε[t] [n ]y[t-1] is certainly highly correlated with y[t], is almost as good as regressor as y[t] Fitted model: ĉ = 0.012 + 0.660 y[-1] with t = 12.86, R^2 = 0.56, DW = 0.79 Deterioration of t-statistic and R^2 are price for improvement of the estimator IV Estimator: The Idea Alternative to OLS estimator n Avoids bias and inconsistency in case of endogenous regressors Instrumental variable estimator (IV estimator): Replace with error terms correlated regressors by regressors n which are uncorrelated with the error terms n which are (highly) correlated with the regressors that are to be replaced and use OLS estimation The hope is that the IV estimator is not biased and consistent or at least less than the OLS estimator Price: Deteriorated model fit, e.g., t-statistic, R^2 IV Estimator: A Simple Case The model y[t] = β[1] + β[2]x[t] + ε[t] with endogenous regressor, E{x[t ]ε[t]} ≠ 0 the OLS estimator is inconsistent Find an instrumental variable z[t] satisfying • E{z[t ]ε[t]} = 0, i.e., instrument is uncorrelated with error term (exogeneity) • cov{x[t], z[t ]} ≠ 0, i.e., instrument is correlated with endogenous regressor and not linearly dependent of x’s (relevance) Covariance of y[t ] with z[t] Cov{y[t],z[t][ ]} = β[2] Cov{x[t],z[t][ ]} + Cov{ε[t],z[t][ ]} This gives β[2] = Cov{ y[t],z[t][ ]} / Cov{ x[t],z[t][ ]} IV Estimator: Simple Case, cont’d The IV estimator is obtained by replacing the population covariances by the sample covariances Properties: n The IV estimator is a consistent estimator for β[2] provided that the instruments are valid, i.e., they are exogenous and relevant n Typically, it cannot not be shown that the IV estimator is unbiased; small sample properties are not known n The IV estimator coincides with the OLS estimator if z[t] = x[t] Advanced Econometrics - Lecture 3 n The OLS Estimator: With Error Correlated Regressors n Correlated Regressors: Some Cases n Instrumental Variables (IV) Estimator: The Concept n IV Estimator: The Method n Calculation of the IV Estimator n An Example n The GIV Estimator n The Generalized Method of Moments n Some Tests IV Estimator: General Case The model is y[t] = x[t]‘β + ε[t], ε[t] white noise, V{ε[i]} = σ[ε]² with E{ε[t] x[t]} ≠ 0 at least one component x[k] of x[ ]is correlated with the error term The vector of instruments z[t] (with the same dimension as x[t]) fulfills E{ε[t] z[t]} = 0 IV estimator based on the instruments z[t] IV Estimator: General Case, cont’d The (asymptotic) covariance matrix of is given by In the estimated covariance matrix, σ² is substituted by The asymptotic distribution of IV estimators, given IID(0, σ[ε]²) error terms, leads to the approximate distribution with the estimated covariance matrix Derivation of IV Estimators The model is y[t] = x[t]‘β + ε[t] = x[0t]‘β[0] + β[K]x[Kt] + ε[t] with x[0t] = (x[1t], …, x[K-1,t])’ containing the first K-1 components of x[t], and E{ε[t] x[0t]} = 0 K-the component is endogenous: E{ε[t] x[Kt]} ≠ 0 The instrumental variable z[Kt] fulfills E{ε[t] z[Kt]} = 0 Moment conditions: K conditions to be satisfied by the coefficients, the K-th condition with z[Kt] instead of x[Kt]: E{ε[t] x[0t]} = E{(y[t] – x[0t]‘β[0] – β[K]x[Kt]) x[0t]} = 0 (K-1 conditions) E{ε[t] z[t]} = E{(y[t] – x[0t]‘β[0] – β[K]x[Kt]) z[Kt]} = 0 Number of conditions – and corresponding linear equations – equals the number of coefficients to be estimated Derivation of IV Estimators, cont’d The system of linear equations for the K coefficients β to be estimated can be uniquely solved for the coefficients β: the coefficients β are identified To derive the IV estimators from the moment conditions, the expectations are replaced by sample averages The solution of the linear equation system – with z[t]’ = (x[0t]‘, z[Kt]) – is Identification requires that the KxK matrix Σ[t ]z[t] x[t]’ is finite and invertible; instrument z is relevant when this is fulfilled Advanced Econometrics - Lecture 3 n The OLS Estimator: With Error Correlated Regressors n Correlated Regressors: Some Cases n Instrumental Variables (IV) Estimator: The Concept n IV Estimator: The Method n Calculation of the IV Estimator n An Example n The GIV Estimator n The Generalized Method of Moments n Some Tests Calculation of IV Estimators In matrix notation, the model is y = Xβ + ε The IV estimator is with z[t] obtained from x[t] by substituting values of the instrumental variable(s) for endogenous regressors Calculation in two steps: • Regression of the explanatory variables x[1], …, x[K] – including the endogenous ones – on the columns of Z gives fitted values • Regression of y on the fitted explanatory variables gives Calculation of IV Estimators, cont’d Remarks: n The KxK matrix Z’X = Σ[t ]z[t]x[t]’ is required to be finite and invertible n From it is obvious that the estimator obtained in the second step is the IV estimator n However, the estimator obtained in the second step is more general; see below Choice of Instrumental Variables Instrumental variable are required to be n exogenous, i.e., uncorrelated with the error terms n Relevant, i.e., correlated with the endogenous regressors Choice must be based on subject matter arguments, e.g., arguments from economic theory Often not easy Advanced Econometrics - Lecture 3 n The OLS Estimator: With Error Correlated Regressors n Correlated Regressors: Some Cases n Instrumental Variables (IV) Estimator: The Concept n IV Estimator: The Method n Calculation of the IV Estimator n An Example n The GIV Estimator n The Generalized Method of Moments n Some Tests Example: Returns to Schooling Wage equation with x[2i]: years of schooling, and u[i]: abilities (intelligence, family background, etc. ; unobservable) y[i] = x[1i]‘β[1] + x[2i]β[2] + u[i]γ + v[i] Empirically, more education implies higher income Question: Is this effect causal? [n ]If yes, one year more at school increases wage by β[2] n Otherwise, abilities may cause higher income and also more years at school Model for analysis: y[i] = x[i]‘β + ε[i] with ε[i] = u[i]γ + v[i] x[2] with E{x[2i]u[i]} ≠ 0 is endogenous: OLS estimators b are inconsistent ; “ability bias” Returns to Schooling: Data n National Longitudinal Survey of Young Men (Card, 1995) n Data from 3010 males, survey 1976 n Individual characteristics, incl. experience, race, region, family background etc. n Human capital function log(wage[i]) = β[1 ]+ β[2] ed[i] + β[3] exp[i] + β[3] exp[i]² + ε[i] with ed: years of schooling, exp: years of experience n Further explanatory variables: black: dummy for afro-american, smsa: dummy for living in metropolitan area, south: dummy for living in the south Returns to Schooling: OLS Estimation Returns to Schooling: Instrumental Variables Instrumental variable n Factors which affect schooling but is uncorrelated with error terms, in particular with unobserved abilities that are determining wage n Costs of schooling, e.g., distance to school, number of siblings; parents’ education; quarter of birth General remarks: n The choice of instruments that should be explained and motivated n Models that explain endogenous regressors from exogenous regressors and instruments (Verbeek: reduced form) should show significant effect of the instruments n Number of instruments can be larger than K Returns to Schooling: Step 1 of IV Estimation Returns to Schooling: Step 2 of IV Estimation Returns to Schooling: TSLS Estimation Returns to Schooling: Summary of Estimates Estimated regression coefficients and t-statistics 1) The model differs from that used by Verbeek Advanced Econometrics - Lecture 3 n The OLS Estimator: With Error Correlated Regressors n Correlated Regressors: Some Cases n Instrumental Variables (IV) Estimator: The Concept n IV Estimator: The Method n Calculation of the IV Estimator n An Example n The GIV Estimator n The Generalized Method of Moments n Some Tests Arbitrary Number of Instruments Moment conditions E{ε[i] x[i]} = E{(y[i] – x[i]‘β) z[i]} = 0 one equation for each component of z[i] General case: R moment conditions Substitution of expectations by sample averages gives • R < K: infinite number of solutions, not enough instruments; unidentified model • R = K: one unique solution, the IV estimator; identified model Generalized IV (GIV) Estimator • R > K: more instruments than necessary for identification; overidentified model Mimimizing the quadratic form in the sample moments with a RxR positive definite weighting matrix W[N] gives the generalized instrumental variable (GIV) estimator “two stage least squares (TSLS) estimator” The optimal weighting matrix correspond to the most efficient IV estimator Instrumental Variables: Remarks The instrumental variables are required to be n exogenous, i.e., uncorrelated with the error terms n relevant, i.e., correlated with the regressors that they are supposed to be instrumenting, not linear combinations of regressors First step of the TSLS procedure: the (endogenous) variables are regressed on the instrumental variables (reduced form regression) n The instruments for explaining x[i] should be “sufficiently important”; check the t-statistics n “Weak instruments”: if instruments correlate only weakly with the endogenous regressor, the IV estimator may be biased, have large standard error, bad approximation to normal distribution Advanced Econometrics - Lecture 3 n The OLS Estimator: With Error Correlated Regressors n Correlated Regressors: Some Cases n Instrumental Variables (IV) Estimator: The Concept n IV Estimator: The Method n Calculation of the IV Estimator n An Example n The GIV Estimator n The Generalized Method of Moments n Some Tests Generalized Method of Moments (GMM) GMM generalizes the IV estimation concept to parameters of models which are not necessarily linear The model is characterized by R moment conditions E{f(w[t], z[t], θ)} = 0 q f: R-vector function q w[t]: vector of observable variables, exogenous or endogenous q z[t]: vector of instrument variables q θ: K-vector of unknown parameters Example: For linear model y[t] = x[t]‘β + ε[t], w[t]‘ = (y[t], x[t]‘) GMM Estimator Substitution of the moment conditions by sample equivalents: g[T](θ) = (1/T) Σ[t] f(w[t], z[t], θ) = 0 • R = K: solve for θ to derive a unique consistent estimator • R > K: minimization wrt θ of the quadratic form Q[T](θ ) = g[T](θ)‘ W[T] g[T](θ) with the positive definite weighting matrix W[T] GMM estimator corresponds to the optimal weighting matrix ^ and is the most efficient estimator For a nonlinear f(.), W[T] depends of θ; iterative optimization algorithms Advanced Econometrics - Lecture 3 n The OLS Estimator: With Error Correlated Regressors n Correlated Regressors: Some Cases n Instrumental Variables (IV) Estimator: The Concept n IV Estimator: The Method n Calculation of the IV Estimator n An Example n The GIV Estimator n The Generalized Method of Moments n Some Tests Some Tests For testing n Exogeneity of regressors: Hausman-Wu test n Suitability of variables to be used as instrumental variable: overidentifying restrictions or Sargan test Hausman-Wu Test For testing whether one or more regressors are endogenous (correlated with the error term) Based on the assumption that the instruments are valid; i.e., given that E{ε[i] z[i]} = 0, E{ε[i]x[i]} = 0 can be tested The idea of the test: n Under the null, both the OLS and IV estimator are consistent; they should differ by sampling error only Hausman-Wu Test, cont’d Hausman–Wu is testing whether the residuals v[i] from the reduced form equation of potentially endogenous regressors contribute to explaining y[i] = x[1i]’b[1] + x[2i]b[2] + v[i]γ + ε[i] n the OLS estimators for b[1 ]and b[2] are the IV estimators n γ = 0: x[2i ]is exogenous For testing the null hypothesis: n t-test of H[0]: γ = 0 n F-test if more than 1 regressors are tested for exogeneity Attention! Test has little power if instruments are weak or unsuitable Sargan Test For testing whether the instruments are valid The validity of the instruments requires that all moment conditions are fulfilled; the R values of the sums must be close to zero Test statistic has under the null hypothesis an asymptotic Chi-squared distribution with R-K df Sargan Test, cont’d Remarks n Only R-K of the R moment conditions are free on account of the first order conditions of the minimization problem n The test is also called overidentifying restrictions test n Rejection implies: the joint validity of all moment conditions and hence of all instruments is not acceptable n The Sargan test gives no indication which instruments are invalid n Test whether a subset of R-R[1] instruments is valid; R[1] (>K) instruments are out of doubt: q Calculate ξ for all R moment conditions q Calculate ξ[1] for the R[1] moment conditions q Under H[0], ξ - ξ[1] has a Chi-squared distribution with R-R[1] df Exercise • Answer questions a. to e. of Exercise 5.2 of Verbeek.